Timescales of inference in visual adaptation

Power-law adaptation in the presynaptic vesicle cycle

After synaptic transmission, fused synaptic vesicles are recycled, enabling the synapse to recover its capacity for renewed release. The recovery steps, which range from endocytosis to vesicle docking and priming, have been studied individually, but it is not clear what their impact on the overall dynamics of synaptic recycling is, and how they influence signal transmission. Here we model the dynamics of vesicle recycling and find that the multiple timescales of the recycling steps are reflected in synaptic recovery. This leads to multi-timescale synapse dynamics, which can be described by a simplified synaptic model with 'power-law' adaptation. Using cultured hippocampal neurons, we test this model experimentally, and show that the duration of synaptic exhaustion changes the effective synaptic recovery timescale, as predicted by the model. Finally, we show that this adaptation could implement a specific function in the hippocampus, namely enabling efficient communication between neurons through the temporal whitening of hippocampal spike trains.

Efficient coding in biophysically realistic excitatory-inhibitory spiking networks

The principle of efficient coding posits that sensory cortical networks are designed to encode maximal sensory information with minimal metabolic cost. Despite the major influence of efficient coding in neuroscience, it has remained unclear whether fundamental empirical properties of neural network activity can be explained solely based on this normative principle. Here, we derive the structural, coding, and biophysical properties of excitatory-inhibitory recurrent networks of spiking neurons that emerge directly from imposing that the network minimizes an instantaneous loss function and a time-averaged performance measure enacting efficient coding. We assumed that the network encodes a number of independent stimulus features varying with a time scale equal to the membrane time constant of excitatory and inhibitory neurons. The optimal network has biologically plausible biophysical features, including realistic integrate-and-fire spiking dynamics, spike-triggered adaptation, and a non-specific excitatory external input. The excitatory-inhibitory recurrent connectivity between neurons with similar stimulus tuning implements feature-specific competition, similar to that recently found in visual cortex. Networks with unstructured connectivity cannot reach comparable levels of coding efficiency. The optimal ratio of excitatory vs inhibitory neurons and the ratio of mean inhibitory-to-inhibitory vs excitatory-to-inhibitory connectivity are comparable to those of cortical sensory networks. The efficient network solution exhibits an instantaneous balance between excitation and inhibition. The network can perform efficient coding even when external stimuli vary over multiple time scales. Together, these results suggest that key properties of biological neural networks may be accounted for by efficient coding.

Efficient coding in biophysically realistic excitatory-inhibitory spiking networks

The principle of efficient coding posits that sensory cortical networks are designed to encode maximal sensory information with minimal metabolic cost. Despite the major influence of efficient coding in neuroscience, it has remained unclear whether fundamental empirical properties of neural network activity can be explained solely based on this normative principle. Here, we derive the structural, coding, and biophysical properties of excitatory-inhibitory recurrent networks of spiking neurons that emerge directly from imposing that the network minimizes an instantaneous loss function and a time-averaged performance measure enacting efficient coding. We assumed that the network encodes a number of independent stimulus features varying with a time scale equal to the membrane time constant of excitatory and inhibitory neurons. The optimal network has biologically-plausible biophysical features, including realistic integrate-and-fire spiking dynamics, spike-triggered adaptation, and a non-specific excitatory external input. The excitatory-inhibitory recurrent connectivity between neurons with similar stimulus tuning implements feature-specific competition, similar to that recently found in visual cortex. Networks with unstructured connectivity cannot reach comparable levels of coding efficiency. The optimal ratio of excitatory vs inhibitory neurons and the ratio of mean inhibitory-to-inhibitory vs excitatory-to-inhibitory connectivity are comparable to those of cortical sensory networks. The efficient network solution exhibits an instantaneous balance between excitation and inhibition. The network can perform efficient coding even when external stimuli vary over multiple time scales. Together, these results suggest that key properties of biological neural networks may be accounted for by efficient coding.

Computational and Neural Evidence for Altered Fast and Slow Learning from Losses in Problem Gambling

Learning occurs across multiple timescales, with fast learning crucial for adapting to sudden environmental changes, and slow learning beneficial for extracting robust knowledge from multiple events. Here, we asked if miscalibrated fast vs slow learning can lead to maladaptive decision-making in individuals with problem gambling. We recruited participants with problem gambling (PG; N = 20; 9 female and 11 male) and a recreational gambling control group without any symptoms associated with PG (N = 20; 10 female and 10 male) from the community in Los Angeles, CA. Participants performed a decision-making task involving reward-learning and loss-avoidance while being scanned with fMRI. Using computational model fitting, we found that individuals in the PG group showed evidence for an excessive dependence on slow timescales and a reduced reliance on fast timescales during learning. fMRI data implicated the putamen, an area associated with habit, and medial prefrontal cortex (PFC) in slow loss-value encoding, with significantly more robust encoding in medial PFC in the PG group compared to controls. The PG group also exhibited stronger loss prediction error encoding in the insular cortex. These findings suggest that individuals with PG have an impaired ability to adjust their predictions following losses, manifested by a stronger influence of slow value learning. This impairment could contribute to the behavioral inflexibility of problem gamblers, particularly the persistence in gambling behavior typically observed in those individuals after incurring loss outcomes.

Optical Bio-Inspired Synaptic Devices

The traditional computer with von Neumann architecture has the characteristics of separate storage and computing units, which leads to sizeable time and energy consumption in the process of data transmission, which is also the famous "von Neumann storage wall" problem. Inspired by neural synapses, neuromorphic computing has emerged as a promising solution to address the von Neumann problem due to its excellent adaptive learning and parallel capabilities. Notably, in 2016, researchers integrated light into neuromorphic computing, which inspired the extensive exploration of optoelectronic and all-optical synaptic devices. These optical synaptic devices offer obvious advantages over traditional all-electric synaptic devices, including a wider bandwidth and lower latency. This review provides an overview of the research background on optoelectronic and all-optical devices, discusses their implementation principles in different scenarios, presents their application scenarios, and concludes with prospects for future developments.

Localized and Long-Lasting Adaptation in Dragonfly Target-Detecting Neurons

Some visual neurons in the dragonfly (Hemicordulia tau) optic lobe respond to small, moving targets, likely underlying their fast pursuit of prey and conspecifics. In response to repetitive targets presented at short intervals, the spiking activity of these "small target motion detector" (STMD) neurons diminishes over time. Previous experiments limited this adaptation by including intertrial rest periods of varying durations. However, the characteristics of this effect have never been quantified. Here, using extracellular recording techniques lasting for several hours, we quantified both the spatial and temporal properties of STMD adaptation. We found that the time course of adaptation was variable across STMD units. In any one STMD, a repeated series led to more rapid adaptation, a minor accumulative effect more akin to habituation. Following an adapting stimulus, responses recovered quickly, though the rate of recovery decreased nonlinearly over time. We found that the region of adaptation is highly localized, with targets displaced by ∼2.5° eliciting a naive response. Higher frequencies of target stimulation converged to lower levels of sustained response activity. We determined that adaptation itself is a target-tuned property, not elicited by moving bars or luminance flicker. As STMD adaptation is a localized phenomenon, dependent on recent history, it is likely to play an important role in closed-loop behavior where a target is foveated in a localized region for extended periods of the pursuit duration.

Adapting and facilitating responses in mouse somatosensory cortex are dynamic and shaped by experience

Sensory adaptation is the process whereby brain circuits adjust neuronal activity in response to redundant sensory stimuli. Although sensory adaptation has been extensively studied for individual neurons on timescales of tens of milliseconds to a few seconds, little is known about it over longer timescales or at the population level. We investigated population-level adaptation in the barrel field of the mouse somatosensory cortex (S1BF) using in vivo two-photon calcium imaging and Neuropixels recordings in awake mice. Among stimulus-responsive neurons, we found both adapting and facilitating neurons, which decreased or increased their firing, respectively, with repetitive whisker stimulation. The former outnumbered the latter by 2:1 in layers 2/3 and 4; hence, the overall population response of mouse S1BF was slightly adapting. We also discovered that population adaptation to one stimulus frequency (5 Hz) does not necessarily generalize to a different frequency (12.5 Hz). Moreover, responses of individual neurons to repeated rounds of stimulation over tens of minutes were strikingly heterogeneous and stochastic, such that their adapting or facilitating response profiles were not stable across time. Such representational drift was particularly striking when recording longitudinally across 8-9 days, as adaptation profiles of most whisker-responsive neurons changed drastically from one day to the next. Remarkably, repeated exposure to a familiar stimulus paradoxically shifted the population away from strong adaptation and toward facilitation. Thus, the adapting vs. facilitating response profile of S1BF neurons is not a fixed property of neurons but rather a highly dynamic feature that is shaped by sensory experience across days.

A dynamic scale-mixture model of motion in natural scenes

Some of the most important tasks of visual and motor systems involve estimating the motion of objects and tracking them over time. Such systems evolved to meet the behavioral needs of the organism in its natural environment, and may therefore be adapted to the statistics of motion it is likely to encounter. By tracking the movement of individual points in movies of natural scenes, we begin to identify common properties of natural motion across scenes. As expected, objects in natural scenes move in a persistent fashion, with velocity correlations lasting hundreds of milliseconds. More subtly, but crucially, we find that the observed velocity distributions are heavy-tailed and can be modeled as a Gaussian scale-mixture. Extending this model to the time domain leads to a dynamic scale-mixture model, consisting of a Gaussian process multiplied by a positive scalar quantity with its own independent dynamics. Dynamic scaling of velocity arises naturally as a consequence of changes in object distance from the observer, and may approximate the effects of changes in other parameters governing the motion in a given scene. This modeling and estimation framework has implications for the neurobiology of sensory and motor systems, which need to cope with these fluctuations in scale in order to represent motion efficiently and drive fast and accurate tracking behavior.

Encoding surprise by retinal ganglion cells

The efficient coding hypothesis posits that early sensory neurons transmit maximal information about sensory stimuli, given internal constraints. A central prediction of this theory is that neurons should preferentially encode stimuli that are most surprising. Previous studies suggest this may be the case in early visual areas, where many neurons respond strongly to rare or surprising stimuli. For example, previous research showed that when presented with a rhythmic sequence of full-field flashes, many retinal ganglion cells (RGCs) respond strongly at the instance the flash sequence stops, and when another flash would be expected. This phenomenon is called the 'omitted stimulus response'. However, it is not known whether the responses of these cells varies in a graded way depending on the level of stimulus surprise. To investigate this, we presented retinal neurons with extended sequences of stochastic flashes. With this stimulus, the surprise associated with a particular flash/silence, could be quantified analytically, and varied in a graded manner depending on the previous sequences of flashes and silences. Interestingly, we found that RGC responses could be well explained by a simple normative model, which described how they optimally combined their prior expectations and recent stimulus history, so as to encode surprise. Further, much of the diversity in RGC responses could be explained by the model, due to the different prior expectations that different neurons had about the stimulus statistics. These results suggest that even as early as the retina many cells encode surprise, relative to their own, internally generated expectations.

Functional maturation of the rod bipolar to AII-amacrine cell ribbon synapse in the mouse retina

Retinal ribbon synapses undergo functional changes after eye opening that remain uncharacterized. Using light-flash stimulation and paired patch-clamp recordings, we examined the maturation of the ribbon synapse between rod bipolar cells (RBCs) and AII-amacrine cells (AII-ACs) after eye opening (postnatal day 14) in the mouse retina at near physiological temperatures. We find that light-evoked excitatory postsynaptic currents (EPSCs) in AII-ACs exhibit a slow sustained component that increases in magnitude with advancing age, whereas a fast transient component remains unchanged. Similarly, paired recordings reveal a dual-component EPSC with a slower sustained component that increases during development, even though the miniature EPSC (mEPSC) amplitude and kinetics do not change significantly. We thus propose that the readily releasable pool of vesicles from RBCs increases after eye opening, and we estimate that a short light flash can evoke the release of ∼4,000 vesicles onto a single mature AII-AC.

Short-term neuronal and synaptic plasticity act in synergy for deviance detection in spiking networks

Sensory areas of cortex respond more strongly to infrequent stimuli when these violate previously established regularities, a phenomenon known as deviance detection (DD). Previous modeling work has mainly attempted to explain DD on the basis of synaptic plasticity. However, a large fraction of cortical neurons also exhibit firing rate adaptation, an underexplored potential mechanism. Here, we investigate DD in a spiking neuronal network model with two types of short-term plasticity, fast synaptic short-term depression (STD) and slower threshold adaptation (TA). We probe the model with an oddball stimulation paradigm and assess DD by evaluating the network responses. We find that TA is sufficient to elicit DD. It achieves this by habituating neurons near the stimulation site that respond earliest to the frequently presented standard stimulus (local fatigue), which diminishes the response and promotes the recovery (global fatigue) of the wider network. Further, we find a synergy effect between STD and TA, where they interact with each other to achieve greater DD than the sum of their individual effects. We show that this synergy is caused by the local fatigue added by STD, which inhibits the global response to the frequently presented stimulus, allowing greater recovery of TA-mediated global fatigue and making the network more responsive to the deviant stimulus. Finally, we show that the magnitude of DD strongly depends on the timescale of stimulation. We conclude that highly predictable information can be encoded in strong local fatigue, which allows greater global recovery and subsequent heightened sensitivity for DD.

Two Sides of the Same Coin: Efficient and Predictive Neural Coding

Some visual properties are consistent across a wide range of environments, while other properties are more labile. The efficient coding hypothesis states that many of these regularities in the environment can be discarded from neural representations, thus allocating more of the brain's dynamic range to properties that are likely to vary. This paradigm is less clear about how the visual system prioritizes different pieces of information that vary across visual environments. One solution is to prioritize information that can be used to predict future events, particularly those that guide behavior. The relationship between the efficient coding and future prediction paradigms is an area of active investigation. In this review, we argue that these paradigms are complementary and often act on distinct components of the visual input. We also discuss how normative approaches to efficient coding and future prediction can be integrated.

Subcortico-amygdala pathway processes innate and learned threats

Behavioral flexibility and timely reactions to salient stimuli are essential for survival. The subcortical thalamic-basolateral amygdala (BLA) pathway serves as a shortcut for salient stimuli ensuring rapid processing. Here, we show that BLA neuronal and thalamic axonal activity in mice mirror the defensive behavior evoked by an innate visual threat as well as an auditory learned threat. Importantly, perturbing this pathway compromises defensive responses to both forms of threats, in that animals fail to switch from exploratory to defensive behavior. Despite the shared pathway between the two forms of threat processing, we observed noticeable differences. Blocking β-adrenergic receptors impairs the defensive response to the innate but not the learned threats. This reduced defensive response, surprisingly, is reflected in the suppression of the activity exclusively in the BLA as the thalamic input response remains intact. Our side-by-side examination highlights the similarities and differences between innate and learned threat-processing, thus providing new fundamental insights.

Self-powered and broadband opto-sensor with bionic visual adaptation function based on multilayer γ-InSe flakes

Visual adaptation that can autonomously adjust the response to light stimuli is a basic function of artificial visual systems for intelligent bionic robots. To improve efficiency and reduce complexity, artificial visual systems with integrated visual adaptation functions based on a single device should be developed to replace traditional approaches that require complex circuitry and algorithms. Here, we have developed a single two-terminal opto-sensor based on multilayer γ-InSe flakes, which successfully emulated the visual adaptation behaviors with a new working mechanism combining the photo-pyroelectric and photo-thermoelectric effect. The device can operate in self-powered mode and exhibit good human-eye-like adaptation behaviors, which include broadband light-sensing image adaptation (from ultraviolet to near-infrared), near-complete photosensitivity recovery (99.6%), and synergetic visual adaptation, encouraging the advancement of intelligent opto-sensors and machine vision systems.

Multifaceted luminance gain control beyond photoreceptors in Drosophila

Animals navigating in natural environments must handle vast changes in their sensory input. Visual systems, for example, handle changes in luminance at many timescales, from slow changes across the day to rapid changes during active behavior. To maintain luminance-invariant perception, visual systems must adapt their sensitivity to changing luminance at different timescales. We demonstrate that luminance gain control in photoreceptors alone is insufficient to explain luminance invariance at both fast and slow timescales and reveal the algorithms that adjust gain past photoreceptors in the fly eye. We combined imaging and behavioral experiments with computational modeling to show that downstream of photoreceptors, circuitry taking input from the single luminance-sensitive neuron type L3 implements gain control at fast and slow timescales. This computation is bidirectional in that it prevents the underestimation of contrasts in low luminance and overestimation in high luminance. An algorithmic model disentangles these multifaceted contributions and shows that the bidirectional gain control occurs at both timescales. The model implements a nonlinear interaction of luminance and contrast to achieve gain correction at fast timescales and a dark-sensitive channel to improve the detection of dim stimuli at slow timescales. Together, our work demonstrates how a single neuronal channel performs diverse computations to implement gain control at multiple timescales that are together important for navigation in natural environments.

The effect of a short-wave filtering contact lens on color appearance

We assessed the effect of a contact lens that filters short-wavelength (SW) visible light on color appearance. These effects were modeled and measured by direct comparison to a clear contact lens. Sixty-one subjects were enrolled, and 58 completed as cohort; 31 were 18 to 39 years old (mean ± SD, 29.6 ± 5.6), 27 were 40 to 65 years old (50.1 ± 8.1). A double-masked contralateral design was used; participants randomly wore a SW-filtering contact lens on one eye and a clear control lens on the other eye. Subjects then mixed three primaries (including a short-wave primary, strongly within the absorbance of the test lens) until a perceived perfect neutral white was achieved with each eye. Color appearance was quantified using chromaticity coordinates measured with a spectral radiometer within a custom-built tricolorimeter. Color vision in natural scenes was simulated using hyperspectral images and cone fundamentals based on a standard observer. Overall, the chromaticity coordinates of matches that were set using the SW-filtering contact lens (n = 58; x = 0.345, y = 0.325, u' = 0.222, v' = 0.470) and clear contact lens (n = 58; x = 0.344, y = 0.325, u' = 0.223, v' = 0.471) were not significantly different, regardless of age group. Simulations indicated that, for natural scenes, the SW-filtering contact lens that was evaluated changes L/(L+M) and S/(L+M) chromatic contrast by no more than -1.4% to +1.1% and -36.9% to +5.0%, respectively. Tricolorimetry was used to measure color appearance in subjects wearing a SW-filtering lens in one eye and a clear lens in the other, and the results indicate that imparting a subtle tint to a contact lens, as in the SW-filtering lens that was evaluated, does not alter color appearance for younger or older subjects. A model of color vision predicted little effect of the lens on chromatic contrast for natural scenes.

Calibrating Vision: Concepts and Questions

The idea that visual coding and perception are shaped by experience and adjust to changes in the environment or the observer is universally recognized as a cornerstone of visual processing, yet the functions and processes mediating these calibrations remain in many ways poorly understood. In this article we review a number of facets and issues surrounding the general notion of calibration, with a focus on plasticity within the encoding and representational stages of visual processing. These include how many types of calibrations there are - and how we decide; how plasticity for encoding is intertwined with other principles of sensory coding; how it is instantiated at the level of the dynamic networks mediating vision; how it varies with development or between individuals; and the factors that may limit the form or degree of the adjustments. Our goal is to give a small glimpse of an enormous and fundamental dimension of vision, and to point to some of the unresolved questions in our understanding of how and why ongoing calibrations are a pervasive and essential element of vision.

Nonequilibrium dynamics of adaptation in sensory systems

Adaptation is used by biological sensory systems to respond to a wide range of environmental signals, by adapting their response properties to the statistics of the stimulus in order to maximize information transmission. We derive rules of optimal adaptation to changes in the mean and variance of a continuous stimulus in terms of Bayesian filters and map them onto stochastic equations that couple the state of the environment to an internal variable controlling the response function. We calculate numerical and exact results for the speed and accuracy of adaptation and its impact on information transmission. We find that, in the regime of efficient adaptation, the speed of adaptation scales sublinearly with the rate of change of the environment. Finally, we exploit the mathematical equivalence between adaptation and stochastic thermodynamics to quantitatively relate adaptation to the irreversibility of the adaptation time course, defined by the rate of entropy production. Our results suggest a means to empirically quantify adaptation in a model-free and nonparametric way.

Retinal Encoding of Natural Scenes

An ultimate goal in retina science is to understand how the neural circuit of the retina processes natural visual scenes. Yet most studies in laboratories have long been performed with simple, artificial visual stimuli such as full-field illumination, spots of light, or gratings. The underlying assumption is that the features of the retina thus identified carry over to the more complex scenario of natural scenes. As the application of corresponding natural settings is becoming more commonplace in experimental investigations, this assumption is being put to the test and opportunities arise to discover processing features that are triggered by specific aspects of natural scenes. Here, we review how natural stimuli have been used to probe, refine, and complement knowledge accumulated under simplified stimuli, and we discuss challenges and opportunities along the way toward a comprehensive understanding of the encoding of natural scenes. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Visual Stimulation Induces Distinct Forms of Sensitization of On-Off Direction-Selective Ganglion Cell Responses in the Dorsal and Ventral Retina

Experience-dependent modulation of neuronal responses is a key attribute in sensory processing. In the mammalian retina, the On-Off direction-selective ganglion cell (DSGC) is well known for its robust direction selectivity. However, how the On-Off DSGC light responsiveness dynamically adjusts to the changing visual environment is underexplored. Here, we report that On-Off DSGCs tuned to posterior motion direction [i.e. posterior DSGCs (pDSGCs)] in mice of both sexes can be transiently sensitized by prior stimuli. Notably, distinct sensitization patterns are found in dorsal and ventral pDSGCs. Although responses of both dorsal and ventral pDSGCs to dark stimuli (Off responses) are sensitized, only dorsal cells show the sensitization of responses to bright stimuli (On responses). Visual stimulation to the dorsal retina potentiates a sustained excitatory input from Off bipolar cells, leading to tonic depolarization of pDSGCs. Such tonic depolarization propagates from the Off to the On dendritic arbor of the pDSGC to sensitize its On response. We also identified a previously overlooked feature of DSGC dendritic architecture that can support dendritic integration between On and Off dendritic layers bypassing the soma. By contrast, ventral pDSGCs lack a sensitized tonic depolarization and thus do not exhibit sensitization of their On responses. Our results highlight a topographic difference in Off bipolar cell inputs underlying divergent sensitization patterns of dorsal and ventral pDSGCs. Moreover, substantial crossovers between dendritic layers of On-Off DSGCs suggest an interactive dendritic algorithm for processing On and Off signals before they reach the soma.SIGNIFICANCE STATEMENT Visual neuronal responses are dynamically influenced by the prior visual experience. This form of plasticity reflects the efficient coding of the naturalistic environment by the visual system. We found that a class of retinal output neurons, On-Off direction-selective ganglion cells, transiently increase their responsiveness after visual stimulation. Cells located in dorsal and ventral retinas exhibit distinct sensitization patterns because of different adaptive properties of Off bipolar cell signaling. A previously overlooked dendritic morphologic feature of the On-Off direction-selective ganglion cell is implicated in the cross talk between On and Off pathways during sensitization. Together, these findings uncover a topographic difference in the adaptive encoding of upper and lower visual fields and the underlying neural mechanism in the dorsal and ventral retinas.

When the Whole Is Less Than the Sum of Its Parts: Maximum Object Category Information and Behavioral Prediction in Multiscale Activation Patterns

Neural codes are reflected in complex neural activation patterns. Conventional electroencephalography (EEG) decoding analyses summarize activations by averaging/down-sampling signals within the analysis window. This diminishes informative fine-grained patterns. While previous studies have proposed distinct statistical features capable of capturing variability-dependent neural codes, it has been suggested that the brain could use a combination of encoding protocols not reflected in any one mathematical feature alone. To check, we combined 30 features using state-of-the-art supervised and unsupervised feature selection procedures (n = 17). Across three datasets, we compared decoding of visual object category between these 17 sets of combined features, and between combined and individual features. Object category could be robustly decoded using the combined features from all of the 17 algorithms. However, the combination of features, which were equalized in dimension to the individual features, were outperformed across most of the time points by the multiscale feature of Wavelet coefficients. Moreover, the Wavelet coefficients also explained the behavioral performance more accurately than the combined features. These results suggest that a single but multiscale encoding protocol may capture the EEG neural codes better than any combination of protocols. Our findings put new constraints on the models of neural information encoding in EEG.

Predicting and Manipulating Cone Responses to Naturalistic Inputs

Primates explore their visual environment by making frequent saccades, discrete and ballistic eye movements that direct the fovea to specific regions of interest. Saccades produce large and rapid changes in input. The magnitude of these changes and the limited signaling range of visual neurons mean that effective encoding requires rapid adaptation. Here, we explore how macaque cone photoreceptors maintain sensitivity under these conditions. Adaptation makes cone responses to naturalistic stimuli highly nonlinear and dependent on stimulus history. Such responses cannot be explained by linear or linear-nonlinear models but are well explained by a biophysical model of phototransduction based on well-established biochemical interactions. The resulting model can predict cone responses to a broad range of stimuli and enables the design of stimuli that elicit specific (e.g., linear) cone photocurrents. These advances will provide a foundation for investigating the contributions of cone phototransduction and post-transduction processing to visual function.SIGNIFICANCE STATEMENT We know a great deal about adaptational mechanisms that adjust sensitivity to slow changes in visual inputs such as the rising or setting sun. We know much less about the rapid adaptational mechanisms that are essential for maintaining sensitivity as gaze shifts around a single visual scene. We characterize how phototransduction in cone photoreceptors adapts to rapid changes in input similar to those encountered during natural vision. We incorporate these measurements into a quantitative model that can predict cone responses across a broad range of stimuli. This model not only shows how cone phototransduction aids the encoding of natural inputs but also provides a tool to identify the role of the cone responses in shaping those of downstream visual neurons.

Advances in materials and devices for mimicking sensory adaptation

Adaptive devices, which aim to adjust electrical behaviors autonomically to external stimuli, are considered to be attractive candidates for next-generation artificial perception systems. Compared with typical electronic devices with stable signal output, adaptive devices possess unique features in exhibiting dynamic fitness to varying environments. To meet this requirement, increasing efforts have been made focusing on developing new materials, functional interfaces and novel device geometry for sensory perception applications. In this review, we summarize the recent advances in materials and devices for mimicking sensory adaptation. Keeping this in mind, we first introduce the fundamentals of biological sensory adaptation. Thereafter, the recent progress in mimicking sensory adaptation, such as tactile and visual adaptive systems, is overviewed. Moreover, we suggest five strategies to construct adaptive devices. Finally, challenges and perspectives are proposed to highlight the directions that deserve focused attention in this flourishing field.

Gated recurrence enables simple and accurate sequence prediction in stochastic, changing, and structured environments

From decision making to perception to language, predicting what is coming next is crucial. It is also challenging in stochastic, changing, and structured environments; yet the brain makes accurate predictions in many situations. What computational architecture could enable this feat? Bayesian inference makes optimal predictions but is prohibitively difficult to compute. Here, we show that a specific recurrent neural network architecture enables simple and accurate solutions in several environments. This architecture relies on three mechanisms: gating, lateral connections, and recurrent weight training. Like the optimal solution and the human brain, such networks develop internal representations of their changing environment (including estimates of the environment’s latent variables and the precision of these estimates), leverage multiple levels of latent structure, and adapt their effective learning rate to changes without changing their connection weights. Being ubiquitous in the brain, gated recurrence could therefore serve as a generic building block to predict in real-life environments.

Changes of tuning but not dynamics of contrast adaptation with age

Normal aging results in pronounced optical and neural changes in the visual system. Processes of adaptation are thought to help compensate for many of these changes in order to maintain perceptual constancy, but it is uncertain how stable adaptation itself remains with aging. We compared the dynamics of adaptation in young (aged 19-24 years) and older (aged 66-74) adults. Contrast thresholds for Gabor patterns were tracked during and after 300 s adaptation to vertical and horizontal Gabor patches. The time course of contrast adaptation and asymptotic adaptation magnitude were similar between older and young adults when normalized for their respective baseline thresholds. Older adults showed stronger transfer of adaptation to the orthogonal orientation and there was an asymmetry between the transfer of adaptation between the horizontal and vertical orientations for both groups. These results suggest age-related changes in orientation tuning while the processes of cortical contrast adaptation remain largely intact with aging.

Binocular rivalry reveals an out-of-equilibrium neural dynamics suited for decision-making

In ambiguous or conflicting sensory situations, perception is often 'multistable' in that it perpetually changes at irregular intervals, shifting abruptly between distinct alternatives. The interval statistics of these alternations exhibits quasi-universal characteristics, suggesting a general mechanism. Using binocular rivalry, we show that many aspects of this perceptual dynamics are reproduced by a hierarchical model operating out of equilibrium. The constitutive elements of this model idealize the metastability of cortical networks. Independent elements accumulate visual evidence at one level, while groups of coupled elements compete for dominance at another level. As soon as one group dominates perception, feedback inhibition suppresses supporting evidence. Previously unreported features in the serial dependencies of perceptual alternations compellingly corroborate this mechanism. Moreover, the proposed out-of-equilibrium dynamics satisfies normative constraints of continuous decision-making. Thus, multistable perception may reflect decision-making in a volatile world: integrating evidence over space and time, choosing categorically between hypotheses, while concurrently evaluating alternatives.

Spontaneous recovery of adaptation aftereffects of natural facial categories

Adaptation to a natural face attribute such as a happy face can bias the perception of a subsequent face in this dimension such as a neutral face. Such face adaptation aftereffects have been widely found in many natural facial categories. However, how temporally tuned mechanisms could control the temporal dynamics of natural face adaptation aftereffects remains unknown. To address the question, we used a deadaptation paradigm to examine whether the spontaneous recovery of natural facial aftereffects would emerge in four natural facial categories including variable categories (emotional expressions in Experiment 1 and eye gaze in Experiment 2) and invariable categories (facial gender in Experiment 3 and facial identity in Experiment 4). In the deadaptation paradigm, participants adapted to a face with an extreme attribute (such as a 100% angry face in Experiment 1) for a relatively long duration, and then deadapted to a face with an opposite extreme attribute (such as a 100% happy face in Experiment 1) for a relatively short duration. The time courses of face adaptation aftereffects were measured using a top-up manner. Deadaptation only masked the effects of initial longer-lasting adaptation, and the spontaneous recovery of adaptation aftereffects was observed at the post-test stage for all four natural facial categories. These results likely indicate that the temporal dynamics of adaptation aftereffects of natural facial categories may be controlled by multiple temporally tuned mechanisms.

Feedback from retinal ganglion cells to the inner retina

Retinal ganglion cells (RGCs) are thought to be strictly postsynaptic within the retina. They carry visual signals from the eye to the brain, but do not make chemical synapses onto other retinal neurons. Nevertheless, they form gap junctions with other RGCs and amacrine cells, providing possibilities for RGC signals to feed back into the inner retina. Here we identified such feedback circuitry in the salamander and mouse retinas. First, using biologically inspired circuit models, we found mutual inhibition among RGCs of the same type. We then experimentally determined that this effect is mediated by gap junctions with amacrine cells. Finally, we found that this negative feedback lowers RGC visual response gain without affecting feature selectivity. The principal neurons of the retina therefore participate in a recurrent circuit much as those in other brain areas, not being a mere collector of retinal signals, but are actively involved in visual computations.

Remembering the Past to See the Future

In addition to the role that our visual system plays in determining what we are seeing right now, visual computations contribute in important ways to predicting what we will see next. While the role of memory in creating future predictions is often overlooked, efficient predictive computation requires the use of information about the past to estimate future events. In this article, we introduce a framework for understanding the relationship between memory and visual prediction and review the two classes of mechanisms that the visual system relies on to create future predictions. We also discuss the principles that define the mapping from predictive computations to predictive mechanisms and how downstream brain areas interpret the predictive signals computed by the visual system. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Matching visual induction effects on screens of different size

In the film industry, the same movie is expected to be watched on displays of vastly different sizes, from cinema screens to mobile phones. But visual induction, the perceptual phenomenon by which the appearance of a scene region is affected by its surroundings, will be different for the same image shown on two displays of different dimensions. This phenomenon presents a practical challenge for the preservation of the artistic intentions of filmmakers, because it can lead to shifts in image appearance between viewing destinations. In this work, we show that a neural field model based on the efficient representation principle is able to predict induction effects and how, by regularizing its associated energy functional, the model is still able to represent induction but is now invertible. From this finding, we propose a method to preprocess an image in a screen-size dependent way so that its perception, in terms of visual induction, may remain constant across displays of different size. The potential of the method is demonstrated through psychophysical experiments on synthetic images and qualitative examples on natural images.

Efficient and adaptive sensory codes

The ability to adapt to changes in stimulus statistics is a hallmark of sensory systems. Here, we developed a theoretical framework that can account for the dynamics of adaptation from an information processing perspective. We use this framework to optimize and analyze adaptive sensory codes, and we show that codes optimized for stationary environments can suffer from prolonged periods of poor performance when the environment changes. To mitigate the adversarial effects of these environmental changes, sensory systems must navigate tradeoffs between the ability to accurately encode incoming stimuli and the ability to rapidly detect and adapt to changes in the distribution of these stimuli. We derive families of codes that balance these objectives, and we demonstrate their close match to experimentally observed neural dynamics during mean and variance adaptation. Our results provide a unifying perspective on adaptation across a range of sensory systems, environments, and sensory tasks.

High Contrast Allows the Retina to Compute More Than Just Contrast

The goal of sensory processing is to represent the environment of an animal. All sensory systems share a similar constraint: they need to encode a wide range of stimulus magnitudes within their narrow neuronal response range. The most efficient way, exploited by even the simplest nervous systems, is to encode relative changes in stimulus magnitude rather than the absolute magnitudes. For instance, the retina encodes contrast, which are the variations of light intensity occurring in time and in space. From this perspective, it is easy to understand why the bright plumage of a moving bird gains a lot of attention, while an octopus remains motionless and mimics its surroundings for concealment. Stronger contrasts simply cause stronger visual signals. However, the gains in retinal performance associated with higher contrast are far more than what can be attributed to just a trivial linear increase in signal strength. Here we discuss how this improvement in performance is reflected throughout different parts of the neural circuitry, within its neural code and how high contrast activates many non-linear mechanisms to unlock several sophisticated retinal computations that are virtually impossible in low contrast conditions.

Visual mode switching learned through repeated adaptation to color

When the environment changes, vision adapts to maintain accurate perception. For repeatedly encountered environments, learning to adjust more rapidly would be beneficial, but past work remains inconclusive. We tested if the visual system can learn such visual mode switching for a strongly color-tinted environment, where adaptation causes the dominant hue to fade over time. Eleven observers wore bright red glasses for five 1-hr periods per day, for 5 days. Color adaptation was measured by asking observers to identify 'unique yellow', appearing neither reddish nor greenish. As expected, the world appeared less and less reddish during the 1-hr periods of glasses wear. Critically, across days the world also appeared significantly less reddish immediately upon donning the glasses. These results indicate that the visual system learned to rapidly adjust to the reddish environment, switching modes to stabilize color vision. Mode switching likely provides a general strategy to optimize perceptual processes.

Color illusions also deceive CNNs for low-level vision tasks: Analysis and implications

The study of visual illusions has proven to be a very useful approach in vision science. In this work we start by showing that, while convolutional neural networks (CNNs) trained for low-level visual tasks in natural images may be deceived by brightness and color illusions, some network illusions can be inconsistent with the perception of humans. Next, we analyze where these similarities and differences may come from. On one hand, the proposed linear eigenanalysis explains the overall similarities: in simple CNNs trained for tasks like denoising or deblurring, the linear version of the network has center-surround receptive fields, and global transfer functions are very similar to the human achromatic and chromatic contrast sensitivity functions in human-like opponent color spaces. These similarities are consistent with the long-standing hypothesis that considers low-level visual illusions as a by-product of the optimization to natural environments. Specifically, here human-like features emerge from error minimization. On the other hand, the observed differences must be due to the behavior of the human visual system not explained by the linear approximation. However, our study also shows that more 'flexible' network architectures, with more layers and a higher degree of nonlinearity, may actually have a worse capability of reproducing visual illusions. This implies, in line with other works in the vision science literature, a word of caution on using CNNs to study human vision: on top of the intrinsic limitations of the L + NL formulation of artificial networks to model vision, the nonlinear behavior of flexible architectures may easily be markedly different from that of the visual system.

Dynamic Contextual Modulation in Superior Colliculus of Awake Mouse

The responses of neurons in the visual pathway depend on the context in which a stimulus is presented. Responses to predictable stimuli are usually suppressed, highlighting responses to unexpected stimuli that might be important for behavior. Here, we established how context modulates the response of neurons in the superior colliculus (SC), a region important in orienting toward or away from visual stimuli. We made extracellular recordings from single units in the superficial layers of SC in awake mice. We found strong suppression of visual response by spatial context (surround suppression) and temporal context (adaptation). Neurons showing stronger surround suppression also showed stronger adaptation effects. In neurons where it was present, surround suppression was dynamic and was reduced by adaptation. Adaptation's effects further revealed two components to surround suppression: one component that was weakly tuned for orientation and adaptable, and another component that was more strongly tuned but less adaptable. The selectivity of the tuned component was flexible, such that suppression was stronger when the stimulus over the surround matched that over the receptive field. Our results therefore reveal strong interactions between spatial and temporal context in regulating the flow of signals through mouse SC, and suggest the presence of a subpopulation of neurons that might signal novelty in either space or time.

The sifting of visual information in the superior colliculus

Much of the early visual system is devoted to sifting the visual scene for the few bits of behaviorally relevant information. In the visual cortex of mammals, a hierarchical system of brain areas leads eventually to the selective encoding of important features, like faces and objects. Here, we report that a similar process occurs in the other major visual pathway, the superior colliculus. We investigate the visual response properties of collicular neurons in the awake mouse with large-scale electrophysiology. Compared to the superficial collicular layers, neuronal responses in the deeper layers become more selective for behaviorally relevant stimuli; more invariant to location of stimuli in the visual field; and more suppressed by repeated occurrence of a stimulus in the same location. The memory of familiar stimuli persists in complete absence of the visual cortex. Models of these neural computations lead to specific predictions for neural circuitry in the superior colliculus.

Short- and long-term forms of neural adaptation: An ERP investigation of dynamic motion aftereffects

Adaptation is essential to interact with a dynamic and changing environment, and can be observed on different timescales. Previous studies on a motion paradigm called dynamic motion aftereffect (dMAE) showed that neural adaptation can establish even in very short timescales. However, the neural mechanisms underlying such rapid form of neural plasticity is still debated. In the present study, short- and long-term forms of neural plasticity were investigated using dynamic motion aftereffect combined with EEG (Electroencephalogram). Participants were adapted to directional drifting gratings for either short (640 msec) or long (6.4 sec) durations. Both adaptation durations led to motion aftereffects on the perceived direction of a dynamic and directionally ambiguous test pattern, but the long adaptation produced stronger dMAE. In line with behavioral results, we found robust changes in the event-related potentials elicited by the dynamic test pattern within 64-112 msec time range. These changes were mainly clustered over occipital and parieto-occipital scalp sites. Within this time range, the aftereffects induced by long adaptation were stronger than those by short adaptation. Moreover, the aftereffects by each adaptation duration were in the opposite direction. Overall, these EEG findings suggest that dMAEs reflect changes in cortical areas mediating low- and mid-level visual motion processing. They further provide evidence that short- and long-term forms of motion adaptation lead to distinct changes in neural activity, and hence support the view that adaptation is an active time-dependent process which involves different neural mechanisms.

Mice Reach Higher Visual Sensitivity at Night by Using a More Efficient Behavioral Strategy

Circadian clocks predictively adjust the physiology of organisms to the day/night cycle. The retina has its own clock, and many diurnal changes in its physiology have been reported. However, their implications for retinal functions and visually guided behavior are largely unresolved. Here, we study the impact of diurnal rhythm on the sensitivity limit of mouse vision. A simple photon detection task allowed us to link well-defined retinal output signals directly to visually guided behavior. We show that visually guided behavior at its sensitivity limit is strongly under diurnal control, reaching the highest sensitivity and stability at night. The diurnal differences in visual sensitivity did not arise in the retina, as assessed by spike recordings from the most sensitive retinal ganglion cell types: ON sustained, OFF sustained, and OFF transient alpha ganglion cells. Instead, we found that mice, as nocturnal animals, use a more efficient search strategy for visual cues at night. Intriguingly, they can switch to the more efficient night strategy even at their subjective day after first having performed the task at night. Our results exemplify that the shape of visual psychometric functions depends robustly on the diurnal state of the animal, its search strategy, and even its diurnal history of performing the task. The results highlight the impact of the day/night cycle on high-level sensory processing, demonstrating a direct diurnal impact on the behavioral strategy of the animal.

Mimicking Sensory Adaptation with Dielectric Engineered Organic Transistors

Mimicking sensory adaptation with transistors is essential for developing next-generation smart circuits. A key challenge is how to obtain controllable and reversible short-term signal decay while simultaneously maintaining long-term electrical stability. By introducing a buried dynamic-trapping interface within the dielectric layer, an organic adaptive transistor (OAT) with sensory adaptation functionality is developed. The device induces self-adaptive interfacial trapping to enable volatile shielding of the gating field, thereby leading to rapid and temporary carrier concentration decay in the conductive channel without diminishing the mobility upon a fixed voltage bias. More importantly, the device exhibits a fine-tuned decay constant ranging from 50 ms to 5 s, accurately matching the adaptation timescales in bio-systems. This not only suggests promising applications of OATs in flexible artificial intelligent elements, but also provides a strategy for engineering organic devices toward novel biomimetic functions.

Population adaptation in efficient balanced networks

Adaptation is a key component of efficient coding in sensory neurons. However, it remains unclear how neurons can provide a stable representation of external stimuli given their history-dependent responses. Here we show that a stable representation is maintained if efficiency is optimized by a population of neurons rather than by neurons individually. We show that spike-frequency adaptation and E/I balanced recurrent connectivity emerge as solutions to a global cost-accuracy tradeoff. The network will redistribute sensory responses from highly excitable neurons to less excitable neurons as the cost of neural activity increases. This does not change the representation at the population level despite causing dynamic changes in individual neurons. By applying this framework to an orientation coding network, we reconcile neural and behavioral findings. Our approach underscores the common mechanisms behind the diversity of neural adaptation and its role in producing a reliable representation of the stimulus while minimizing metabolic cost.

Humans see, hear, feel, taste and smell the world as spiking electrical signals in the brain encoded by sensory neurons. Sensory neurons learn from experience to adjust their activity when exposed repeatedly to the same stimuli. A loud noise or that strange taste in your mouth might be alarming at first but soon sensory neurons dial down their response as the sensations become familiar, saving energy.

This neural adaptation has been observed experimentally in individual cells, but it raises questions about how the brain deciphers signals from sensory neurons. How do downstream neurons learn whether the reduced activity from sensory neurons is a result of getting used to a feeling, or a signal encoding a weaker stimulus? The energy saved through neural adaptation cannot come at the expense of sensing the world less accurately. Neural networks in our brain have evidently evolved to code information in a way that is both efficient and accurate, and computational neuroscientists want to know how. There has been great interest in reproducing neural networks for machine learning, but computer models have not yet captured the mechanisms of neural coding with the same eloquence as the brain.

Gutierrez and Denève used computational models to test how networks of sensory neurons encode a sensible signal whilst adapting to new or repeated stimuli. The experiments showed that optimal neural networks are highly cooperative and share the load when encoding information. Individual neurons are more sensitive to certain stimuli but the information is encoded across the network so that if one neuron becomes fatigued, others receptive to the same stimuli can respond. In this way, the network is both responsive and reliable, producing a steady output which can be readily interpreted by downstream neurons.

Exploring how stimuli are encoded in the brain, Gutierrez and Denève have shown that the activity of one neuron does not represent the whole picture of neural adaptation. The brain has evolved to adapt to continuous stimuli for efficiency at both the level of individual neurons and across balanced networks of interconnected neurons. It takes many neurons to accurately represent the world, but only as a network can the brain sustain a steady picture.

Paradoxical Rules of Spike Train Decoding Revealed at the Sensitivity Limit of Vision

All sensory information is encoded in neural spike trains. It is unknown how the brain utilizes this neural code to drive behavior. Here, we unravel the decoding rules of the brain at the most elementary level by linking behavioral decisions to retinal output signals in a single-photon detection task. A transgenic mouse line allowed us to separate the two primary retinal outputs, ON and OFF pathways, carrying information about photon absorptions as increases and decreases in spiking, respectively. We measured the sensitivity limit of rods and the most sensitive ON and OFF ganglion cells and correlated these results with visually guided behavior using markerless head and eye tracking. We show that behavior relies only on the ON pathway even when the OFF pathway would allow higher sensitivity. Paradoxically, behavior does not rely on the spike code with maximal information but instead relies on a decoding strategy based on increases in spiking.

Neural sensitization improves encoding fidelity in the primate retina

An animal’s motion through the environment can induce large and frequent fluctuations in light intensity on the retina. These fluctuations pose a major challenge to neural circuits tasked with encoding visual information, as they can cause cells to adapt and lose sensitivity. Here, we report that sensitization, a short-term plasticity mechanism, solves this difficult computational problem by maintaining neuronal sensitivity in the face of these fluctuations. The numerically dominant output pathway in the macaque monkey retina, the midget (parvocellular-projecting) pathway, undergoes sensitization under specific conditions, including simulated eye movements. Sensitization is present in the excitatory synaptic inputs from midget bipolar cells and is mediated by presynaptic disinhibition from a wide-field mechanism extending >0.5 mm along the retinal surface. Direct physiological recordings and a computational model indicate that sensitization in the midget pathway supports accurate sensory encoding and prevents a loss of responsiveness during dynamic visual processing.

Light intensity on the retina can fluctuate rapidly during natural vision, posing a challenge for encoding visual information. Here, the authors report that mechanisms of sensitization/facilitation maintain the sensitivity of the numerically dominant neural pathway in the primate retina during dynamic vision.

Coding Principles in Adaptation

Adaptation is a common principle that recurs throughout the nervous system at all stages of processing. This principle manifests in a variety of phenomena, from spike frequency adaptation, to apparent changes in receptive fields with changes in stimulus statistics, to enhanced responses to unexpected stimuli. The ubiquity of adaptation leads naturally to the question: What purpose do these different types of adaptation serve? A diverse set of theories, often highly overlapping, has been proposed to explain the functional role of adaptive phenomena. In this review, we discuss several of these theoretical frameworks, highlighting relationships among them and clarifying distinctions. We summarize observations of the varied manifestations of adaptation, particularly as they relate to these theoretical frameworks, focusing throughout on the visual system and making connections to other sensory systems.

Dynamics of contrast adaptation in central and peripheral vision

Adaptation aftereffects are generally stronger for peripheral than for foveal viewing. We examined whether there are also differences in the dynamics of visual adaptation in central and peripheral vision. We tracked the time course of contrast adaptation to binocularly presented Gabor patterns in both the central visual field (within 5°) and in the periphery (beyond 10° eccentricity) using a yes/no detection task to monitor contrast thresholds. Consistent with previous studies, sensitivity losses were stronger in the periphery than in the center when adapting to equivalent high contrast (90% contrast) patterns. The time course of the threshold changes was fitted with separate exponential functions to estimate the time constants during the adapt and post-adapt phases. When adapting to equivalent high contrast, adaptation effects built up and decayed more slowly in the periphery compared with central adaptation. Surprisingly, the aftereffect in the periphery did not decay completely to the baseline within the monitored post-adapt period (400 s), and instead asymptoted to a higher level than for central adaptation. Even when contrast was reduced to one-third (30% contrast) of the central contrast, peripheral adaptation remained stronger and decayed more slowly. This slower dynamic was also confirmed at suprathreshold test contrasts by tracking tilt-aftereffects with a 2AFC orientation discrimination task. Our results indicate that the dynamics of contrast adaptation differ between central and peripheral vision, with the periphery adapting not only more strongly but also more slowly, and provide another example of potential qualitative processing differences between central and peripheral vision.

How a well-adapting immune system remembers

The adaptive immune system is able to protect us from a large variety of pathogens, even ones it has not seen yet. Can predicting the future pathogen distribution help in protection? We find that a combination of probabilistic forecasting and occasional sampling of the current environment reduces infection costs—a scheme easily implemented by the memory repertoire. The proposed theoretical framework offers a modular recipe for updating the memory repertoire, which quantitatively predicts the strength of the immune response in flu-vaccination experiments, unlike other update schemes. It also links the observed early life dynamics of the memory pool to the sparseness properties of the pathogen distribution and competitive receptor dynamics for pathogens.

An adaptive agent predicting the future state of an environment must weigh trust in new observations against prior experiences. In this light, we propose a view of the adaptive immune system as a dynamic Bayesian machinery that updates its memory repertoire by balancing evidence from new pathogen encounters against past experience of infection to predict and prepare for future threats. This framework links the observed initial rapid increase of the memory pool early in life followed by a midlife plateau to the ease of learning salient features of sparse environments. We also derive a modulated memory pool update rule in agreement with current vaccine-response experiments. Our results suggest that pathogenic environments are sparse and that memory repertoires significantly decrease infection costs, even with moderate sampling. The predicted optimal update scheme maps onto commonly considered competitive dynamics for antigen receptors.

Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales

Behavior deviating from our normative expectations often appears irrational. For example, even though behavior following the so-called matching law can maximize reward in a stationary foraging task, actual behavior commonly deviates from matching. Such behavioral deviations are interpreted as a failure of the subject; however, here we instead suggest that they reflect an adaptive strategy, suitable for uncertain, non-stationary environments. To prove it, we analyzed the behavior of primates that perform a dynamic foraging task. In such nonstationary environment, learning on both fast and slow timescales is beneficial: fast learning allows the animal to react to sudden changes, at the price of large fluctuations (variance) in the estimates of task relevant variables. Slow learning reduces the fluctuations but costs a bias that causes systematic behavioral deviations. Our behavioral analysis shows that the animals solved this bias-variance tradeoff by combining learning on both fast and slow timescales, suggesting that learning on multiple timescales can be a biologically plausible mechanism for optimizing decisions under uncertainty.

Recent experience can only provide limited information to guide decisions in a volatile environment. Here, the authors report that the choices made by a monkey in a dynamic foraging task can be better explained by a model that combines learning on both fast and slow timescales.

Olfactory Navigation and the Receptor Nonlinearity

The demands on a sensory system depend not only on the statistics of its inputs but also on the task. In olfactory navigation, for example, the task is to find the plume source; allocation of sensory resources may therefore be driven by aspects of the plume that are informative about source location, rather than concentration per se. Here we explore the implications of this idea for encoding odor concentration. To formalize the notion that sensory resources are limited, we considered coding strategies that partitioned the odor concentration range into a set of discriminable intervals. We developed a dynamic programming algorithm that, given the distribution of odor concentrations at several locations, determines the partitioning that conveys the most information about location. We applied this analysis to planar laser-induced fluorescence measurements of spatiotemporal odor fields with realistic advection speeds (5-20 cm/s), with or without a nearby boundary or obstacle. Across all environments, the optimal coding strategy allocated more resources (i.e., more and finer discriminable intervals) to the upper end of the concentration range than would be expected from histogram equalization, the optimal strategy if the goal were to reconstruct the plume, rather than to navigate. Finally, we show that ligand binding, as captured by the Hill equation, transforms odorant concentration into response levels in a way that approximates information maximization for navigation. This behavior occurs when the Hill dissociation constant is near the mean odor concentration, an adaptive set-point that has been observed in the olfactory system of flies.SIGNIFICANCE STATEMENT The first step of olfactory processing is receptor binding, and the resulting relationship between odorant concentration and the bound receptor fraction is a saturating one. While this Hill nonlinearity can be viewed as a distortion that is imposed by the biophysics of receptor binding, here we show that it also plays an important information-processing role in olfactory navigation. Specifically, by combining a novel dynamic-programming algorithm with physical measurements of turbulent plumes, we determine the optimal strategy for encoding odor concentration when the goal is to determine location. This strategy is distinct from histogram equalization, the strategy that maximizes information about plume concentration, and is closely approximated by the Hill nonlinearity when the binding constant is near the ambient mean.

Contrast and luminance adaptation alter neuronal coding and perception of stimulus orientation

Sensory systems face a barrage of stimulation that continually changes along multiple dimensions. These simultaneous changes create a formidable problem for the nervous system, as neurons must dynamically encode each stimulus dimension, despite changes in other dimensions. Here, we measured how neurons in visual cortex encode orientation following changes in luminance and contrast, which are critical for visual processing, but nuisance variables in the context of orientation coding. Using information theoretic analysis and population decoding approaches, we find that orientation discriminability is luminance and contrast dependent, changing over time due to firing rate adaptation. We also show that orientation discrimination in human observers changes during adaptation, in a manner consistent with the neuronal data. Our results suggest that adaptation does not maintain information rates per se, but instead acts to keep sensory systems operating within the limited dynamic range afforded by spiking activity, despite a wide range of possible inputs.

Sensory systems produce stable stimulus representations despite constant changes across multiple stimulus dimensions. Here, the authors reveal dynamic neural coding mechanisms by testing how coding of one dimension (orientation) changes with adaptations to other dimensions (luminance and contrast).

The Role of Bottom-Up and Top-Down Cortical Interactions in Adaptation to Natural Scene Statistics

Adaptation is a mechanism by which cortical neurons adjust their responses according to recently viewed stimuli. Visual information is processed in a circuit formed by feedforward (FF) and feedback (FB) synaptic connections of neurons in different cortical layers. Here, the functional role of FF-FB streams and their synaptic dynamics in adaptation to natural stimuli is assessed in psychophysics and neural model. We propose a cortical model which predicts psychophysically observed motion adaptation aftereffects (MAE) after exposure to geometrically distorted natural image sequences. The model comprises direction selective neurons in V1 and MT connected by recurrent FF and FB dynamic synapses. Psychophysically plausible model MAEs were obtained from synaptic changes within neurons tuned to salient direction signals of the broadband natural input. It is conceived that, motion disambiguation by FF-FB interactions is critical to encode this salient information. Moreover, only FF-FB dynamic synapses operating at distinct rates predicted psychophysical MAEs at different adaptation time-scales which could not be accounted for by single rate dynamic synapses in either of the streams. Recurrent FF-FB pathways thereby play a role during adaptation in a natural environment, specifically in inducing multilevel cortical plasticity to salient information and in mediating adaptation at different time-scales.

Variance adaptation in navigational decision making

Sensory systems relay information about the world to the brain, which enacts behaviors through motor outputs. To maximize information transmission, sensory systems discard redundant information through adaptation to the mean and variance of the environment. The behavioral consequences of sensory adaptation to environmental variance have been largely unexplored. Here, we study how larval fruit flies adapt sensory-motor computations underlying navigation to changes in the variance of visual and olfactory inputs. We show that variance adaptation can be characterized by rescaling of the sensory input and that for both visual and olfactory inputs, the temporal dynamics of adaptation are consistent with optimal variance estimation. In multisensory contexts, larvae adapt independently to variance in each sense, and portions of the navigational pathway encoding mixed odor and light signals are also capable of variance adaptation. Our results suggest multiplication as a mechanism for odor-light integration.