Modeling convergent ON and OFF pathways in the early visual system

Filter-based models of suppression in retinal ganglion cells: Comparison and generalization across species and stimuli

The dichotomy of excitation and suppression is one of the canonical mechanisms explaining the complexity of neural activity. Computational models of the interplay of excitation and suppression in single neurons aim at investigating how this interaction affects a neuron's spiking responses and shapes the encoding of sensory stimuli. Here, we compare the performance of three filter-based stimulus-encoding models for predicting retinal ganglion cell responses recorded from axolotl, mouse, and marmoset retina to different types of temporally varying visual stimuli. Suppression in these models is implemented via subtractive or divisive interactions of stimulus filters or by a response-driven feedback module. For the majority of ganglion cells, the subtractive and divisive models perform similarly and outperform the feedback model as well as a linear-nonlinear (LN) model with no suppression. Comparison between the subtractive and the divisive model depends on cell type, species, and stimulus components, with the divisive model generalizing best across temporal stimulus frequencies and visual contrast and the subtractive model capturing in particular responses for slow temporal stimulus dynamics and for slow axolotl cells. Overall, we conclude that the divisive and subtractive models are well suited for capturing interactions of excitation and suppression in ganglion cells and perform best for different temporal regimes of these interactions.

An actor-model framework for visual sensory encoding

A fundamental challenge in neuroengineering is determining a proper artificial input to a sensory system that yields the desired perception. In neuroprosthetics, this process is known as artificial sensory encoding, and it holds a crucial role in prosthetic devices restoring sensory perception in individuals with disabilities. For example, in visual prostheses, one key aspect of artificial image encoding is to downsample images captured by a camera to a size matching the number of inputs and resolution of the prosthesis. Here, we show that downsampling an image using the inherent computation of the retinal network yields better performance compared to learning-free downsampling methods. We have validated a learning-based approach (actor-model framework) that exploits the signal transformation from photoreceptors to retinal ganglion cells measured in explanted mouse retinas. The actor-model framework generates downsampled images eliciting a neuronal response in-silico and ex-vivo with higher neuronal reliability than the one produced by a learning-free approach. During the learning process, the actor network learns to optimize contrast and the kernel's weights. This methodological approach might guide future artificial image encoding strategies for visual prostheses. Ultimately, this framework could be applicable for encoding strategies in other sensory prostheses such as cochlear or limb.

Model-Based Approach Shows ON Pathway Afferents Elicit a Transient Decrease of V1 Responses

Neurons in the primary visual cortex (V1) receive excitation and inhibition from distinct parallel pathways processing lightness (ON) and darkness (OFF). V1 neurons overall respond more strongly to dark than light stimuli, consistent with a preponderance of darker regions in natural images, as well as human psychophysics. However, it has been unclear whether this "dark-dominance" is because of more excitation from the OFF pathway or more inhibition from the ON pathway. To understand the mechanisms behind dark-dominance, we record electrophysiological responses of individual simple-type V1 neurons to natural image stimuli and then train biologically inspired convolutional neural networks to predict the neurons' responses. Analyzing a sample of 71 neurons (in anesthetized, paralyzed cats of either sex) has revealed their responses to be more driven by dark than light stimuli, consistent with previous investigations. We show that this asymmetry is predominantly because of slower inhibition to dark stimuli rather than to stronger excitation from the thalamocortical OFF pathway. Consistent with dark-dominant neurons having faster responses than light-dominant neurons, we find dark-dominance to solely occur in the early latencies of neurons' responses. Neurons that are strongly dark-dominated also tend to be less orientation-selective. This novel approach gives us new insight into the dark-dominance phenomenon and provides an avenue to address new questions about excitatory and inhibitory integration in cortical neurons.SIGNIFICANCE STATEMENT Neurons in the early visual cortex respond on average more strongly to dark than to light stimuli, but the mechanisms behind this bias have been unclear. Here we address this issue by combining single-unit electrophysiology with a novel machine learning model to analyze neurons' responses to natural image stimuli in primary visual cortex. Using these techniques, we find slower inhibition to light than to dark stimuli to be the leading mechanism behind stronger dark responses. This slower inhibition to light might help explain other empirical findings, such as why orientation selectivity is weaker at earlier response latencies. These results demonstrate how imbalances in excitation versus inhibition can give rise to response asymmetries in cortical neuron responses.

A new grid stimulus with subtle flicker perception for user-friendly SSVEP-based BCIs

Objective.The traditional uniform flickering stimulation pattern shows strong steady-state visual evoked potential (SSVEP) responses and poor user experience with intense flicker perception. To achieve a balance between performance and comfort in SSVEP-based brain-computer interface (BCI) systems, this study proposed a new grid stimulation pattern with reduced stimulation area and low spatial contrast.Approach.A spatial contrast scanning experiment was conducted first to clarify the relationship between the SSVEP characteristics and the signs and values of spatial contrast. Four stimulation patterns were involved in the experiment: the ON and OFF grid stimulation patterns that separately activated the positive or negative contrast information processing pathways, the ON-OFF grid stimulation pattern that simultaneously activated both pathways, and the uniform flickering stimulation pattern that served as a control group. The contrast-intensity and contrast-user experience curves were obtained for each stimulation pattern. Accordingly, the optimized stimulation schemes with low spatial contrast (the ON-50% grid stimulus, the OFF-50% grid stimulus, and the Flicker-30% stimulus) were applied in a 12-target and a 40-target BCI speller and compared with the traditional uniform flickering stimulus (the Flicker-500% stimulus) in the evaluation of BCI performance and subjective experience.Main results.The OFF-50% grid stimulus showed comparable online performance (12-target, 2 s: 69.87 ± 0.74 vs. 69.76 ± 0.58 bits min^-1, 40-target, 4 s: 57.02 ± 2.53 vs. 60.79 ± 1.08 bits min^-1) and improved user experience (better comfortable level, weaker flicker perception and higher preference level) compared to the traditional Flicker-500% stimulus in both multi-targets BCI spellers.Significance.Selective activation of the negative contrast information processing pathway using the new OFF-50% grid stimulus evoked robust SSVEP responses. On this basis, high-performance and user-friendly SSVEP-based BCIs have been developed and implemented, which has important theoretical significance and application value in promoting the development of the visual BCI technology.

Neural System Identification With Spike-Triggered Non-Negative Matrix Factorization

Neuronal circuits formed in the brain are complex with intricate connection patterns. Such complexity is also observed in the retina with a relatively simple neuronal circuit. A retinal ganglion cell (GC) receives excitatory inputs from neurons in previous layers as driving forces to fire spikes. Analytical methods are required to decipher these components in a systematic manner. Recently a method called spike-triggered non-negative matrix factorization (STNMF) has been proposed for this purpose. In this study, we extend the scope of the STNMF method. By using retinal GCs as a model system, we show that STNMF can detect various computational properties of upstream bipolar cells (BCs), including spatial receptive field, temporal filter, and transfer nonlinearity. In addition, we recover synaptic connection strengths from the weight matrix of STNMF. Furthermore, we show that STNMF can separate spikes of a GC into a few subsets of spikes, where each subset is contributed by one presynaptic BC. Taken together, these results corroborate that STNMF is a useful method for deciphering the structure of neuronal circuits.

Simple model for encoding natural images by retinal ganglion cells with nonlinear spatial integration

A central goal in sensory neuroscience is to understand the neuronal signal processing involved in the encoding of natural stimuli. A critical step towards this goal is the development of successful computational encoding models. For ganglion cells in the vertebrate retina, the development of satisfactory models for responses to natural visual scenes is an ongoing challenge. Standard models typically apply linear integration of visual stimuli over space, yet many ganglion cells are known to show nonlinear spatial integration, in particular when stimulated with contrast-reversing gratings. We here study the influence of spatial nonlinearities in the encoding of natural images by ganglion cells, using multielectrode-array recordings from isolated salamander and mouse retinas. We assess how responses to natural images depend on first- and second-order statistics of spatial patterns inside the receptive field. This leads us to a simple extension of current standard ganglion cell models. We show that taking not only the weighted average of light intensity inside the receptive field into account but also its variance over space can partly account for nonlinear integration and substantially improve response predictions of responses to novel images. For salamander ganglion cells, we find that response predictions for cell classes with large receptive fields profit most from including spatial contrast information. Finally, we demonstrate how this model framework can be used to assess the spatial scale of nonlinear integration. Our results underscore that nonlinear spatial stimulus integration translates to stimulation with natural images. Furthermore, the introduced model framework provides a simple, yet powerful extension of standard models and may serve as a benchmark for the development of more detailed models of the nonlinear structure of receptive fields.

For understanding how sensory systems operate in the natural environment, an important goal is to develop models that capture neuronal responses to natural stimuli. For retinal ganglion cells, which connect the eye to the brain, current standard models often fail to capture responses to natural visual scenes. This shortcoming is at least partly rooted in the fact that ganglion cells may combine visual signals over space in a nonlinear fashion. We here show that a simple model, which not only considers the average light intensity inside a cell’s receptive field but also the variance of light intensity over space, can partly account for these nonlinearities and thereby improve current standard models. This provides an easy-to-obtain benchmark for modeling ganglion cell responses to natural images.

What the salamander eye has been telling the vision scientist's brain

Salamanders have been habitual residents of research laboratories for more than a century, and their history in science is tightly interwoven with vision research. Nevertheless, many vision scientists - even those working with salamanders - may be unaware of how much our knowledge about vision, and particularly the retina, has been shaped by studying salamanders. In this review, we take a tour through the salamander history in vision science, highlighting the main contributions of salamanders to our understanding of the vertebrate retina. We further point out specificities of the salamander visual system and discuss the perspectives of this animal system for future vision research.

New Features of Receptive Fields in Mouse Retina through Spike-triggered Covariance

Retinal ganglion cells (RGCs) encode various spatiotemporal features of visual information into spiking patterns. The receptive field (RF) of each RGC is usually calculated by spike-triggered average (STA), which is fast and easy to understand, but limited to simple and unimodal RFs. As an alternative, spike-triggered covariance (STC) has been proposed to characterize more complex patterns in RFs. This study compares STA and STC for the characterization of RFs and demonstrates that STC has an advantage over STA for identifying novel spatiotemporal features of RFs in mouse RGCs. We first classified mouse RGCs into ON, OFF, and ON/OFF cells according to their response to full-field light stimulus, and then investigated the spatiotemporal patterns of RFs with random checkerboard stimulation, using both STA and STC analysis. We propose five sub-types (T1–T5) in the STC of mouse RGCs together with their physiological implications. In particular, the relatively slow biphasic pattern (T1) could be related to excitatory inputs from bipolar cells. The transient biphasic pattern (T2) allows one to characterize complex patterns in RFs of ON/OFF cells. The other patterns (T3–T5), which are contrasting, alternating, and monophasic patterns, could be related to inhibitory inputs from amacrine cells. Thus, combining STA and STC and considering the proposed sub-types unveil novel characteristics of RFs in the mouse retina and offer a more holistic understanding of the neural coding mechanisms of mouse RGCs.

From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction

Recently, deep feedforward neural networks have achieved considerable success in modeling biological sensory processing, in terms of reproducing the input-output map of sensory neurons. However, such models raise profound questions about the very nature of explanation in neuroscience. Are we simply replacing one complex system (a biological circuit) with another (a deep network), without understanding either? Moreover, beyond neural representations, are the deep network's computational mechanisms for generating neural responses the same as those in the brain? Without a systematic approach to extracting and understanding computational mechanisms from deep neural network models, it can be difficult both to assess the degree of utility of deep learning approaches in neuroscience, and to extract experimentally testable hypotheses from deep networks. We develop such a systematic approach by combining dimensionality reduction and modern attribution methods for determining the relative importance of interneurons for specific visual computations. We apply this approach to deep network models of the retina, revealing a conceptual understanding of how the retina acts as a predictive feature extractor that signals deviations from expectations for diverse spatiotemporal stimuli. For each stimulus, our extracted computational mechanisms are consistent with prior scientific literature, and in one case yields a new mechanistic hypothesis. Thus overall, this work not only yields insights into the computational mechanisms underlying the striking predictive capabilities of the retina, but also places the framework of deep networks as neuroscientific models on firmer theoretical foundations, by providing a new roadmap to go beyond comparing neural representations to extracting and understand computational mechanisms.

Data-Driven Approaches to Understanding Visual Neuron Activity

With modern neurophysiological methods able to record neural activity throughout the visual pathway in the context of arbitrarily complex visual stimulation, our understanding of visual system function is becoming limited by the available models of visual neurons that can be directly related to such data. Different forms of statistical models are now being used to probe the cellular and circuit mechanisms shaping neural activity, understand how neural selectivity to complex visual features is computed, and derive the ways in which neurons contribute to systems-level visual processing. However, models that are able to more accurately reproduce observed neural activity often defy simple interpretations. As a result, rather than being used solely to connect with existing theories of visual processing, statistical modeling will increasingly drive the evolution of more sophisticated theories.

Functional characterization of retinal ganglion cells using tailored nonlinear modeling

The mammalian retina encodes the visual world in action potentials generated by 20-50 functionally and anatomically-distinct types of retinal ganglion cell (RGC). Individual RGC types receive synaptic input from distinct presynaptic circuits; therefore, their responsiveness to specific features in the visual scene arises from the information encoded in synaptic input and shaped by postsynaptic signal integration and spike generation. Unfortunately, there is a dearth of tools for characterizing the computations reflected in RGC spike output. Therefore, we developed a statistical model, the separable Nonlinear Input Model, to characterize the excitatory and suppressive components of RGC receptive fields. We recorded RGC responses to a correlated noise ("cloud") stimulus in an in vitro preparation of mouse retina and found that our model accurately predicted RGC responses at high spatiotemporal resolution. It identified multiple receptive fields reflecting the main excitatory and suppressive components of the response of each neuron. Significantly, our model accurately identified ON-OFF cells and distinguished their distinct ON and OFF receptive fields, and it demonstrated a diversity of suppressive receptive fields in the RGC population. In total, our method offers a rich description of RGC computation and sets a foundation for relating it to retinal circuitry.

The dynamic receptive fields of retinal ganglion cells

Retinal ganglion cells (RGCs) were one of the first classes of sensory neurons to be described in terms of a receptive field (RF). Over the last six decades, our understanding of the diversity of RGC types and the nuances of their response properties has grown exponentially. We will review the current understanding of RGC RFs mostly from studies in mammals, but including work from other vertebrates as well. We will argue for a new paradigm that embraces the fluidity of RGC RFs with an eye toward the neuroethology of vision. Specifically, we will focus on (1) different methods for measuring RGC RFs, (2) RF models, (3) feature selectivity and the distinction between fluid and stable RF properties, and (4) ideas about the future of understanding RGC RFs.

A single retinal circuit model for multiple computations

Vision is dependent on extracting intricate features of the visual information from the outside world, and complex visual computations begin to take place as soon as at the retinal level. In multiple studies on salamander retinas, the responses of a subtype of retinal ganglion cells, i.e., fast/biphasic-OFF ganglion cells, have been shown to be able to realize multiple functions, such as the segregation of a moving object from its background, motion anticipation, and rapid encoding of the spatial features of a new visual scene. For each of these visual functions, modeling approaches using extended linear-nonlinear cascade models suggest specific preceding retinal circuitries merging onto fast/biphasic-OFF ganglion cells. However, whether multiple visual functions can be accommodated together in a certain retinal circuitry and how specific mechanisms for each visual function interact with each other have not been investigated. Here, we propose a physiologically consistent, detailed computational model of the retinal circuit based on the spatiotemporal dynamics and connections of each class of retinal neurons to implement object motion sensitivity, motion anticipation, and rapid coding in the same circuit. Simulations suggest that multiple computations can be accommodated together, thereby implying that the fast/biphasic-OFF ganglion cell has potential to output a train of spikes carrying multiple pieces of information on distinct features of the visual stimuli.

Bioinspired Approach to Modeling Retinal Ganglion Cells Using System Identification Techniques

The processing capabilities of biological vision systems are still vastly superior to artificial vision, even though this has been an active area of research for over half a century. Current artificial vision techniques integrate many insights from biology yet they remain far-off the capabilities of animals and humans in terms of speed, power, and performance. A key aspect to modeling the human visual system is the ability to accurately model the behavior and computation within the retina. In particular, we focus on modeling the retinal ganglion cells (RGCs) as they convey the accumulated data of real world images as action potentials onto the visual cortex via the optic nerve. Computational models that approximate the processing that occurs within RGCs can be derived by quantitatively fitting the sets of physiological data using an input-output analysis where the input is a known stimulus and the output is neuronal recordings. Currently, these input-output responses are modeled using computational combinations of linear and nonlinear models that are generally complex and lack any relevance to the underlying biophysics. In this paper, we illustrate how system identification techniques, which take inspiration from biological systems, can accurately model retinal ganglion cell behavior, and are a viable alternative to traditional linear-nonlinear approaches.

Multiplexed computations in retinal ganglion cells of a single type

In the early visual system, cells of the same type perform the same computation in different places of the visual field. How these cells code together a complex visual scene is unclear. A common assumption is that cells of a single-type extract a single-stimulus feature to form a feature map, but this has rarely been observed directly. Using large-scale recordings in the rat retina, we show that a homogeneous population of fast OFF ganglion cells simultaneously encodes two radically different features of a visual scene. Cells close to a moving object code quasilinearly for its position, while distant cells remain largely invariant to the object's position and, instead, respond nonlinearly to changes in the object's speed. We develop a quantitative model that accounts for this effect and identify a disinhibitory circuit that mediates it. Ganglion cells of a single type thus do not code for one, but two features simultaneously. This richer, flexible neural map might also be present in other sensory systems.

Rapid and coordinated processing of global motion images by local clusters of retinal ganglion cells

Even when the body is stationary, the whole retinal image is always in motion by fixational eye movements and saccades that move the eye between fixation points. Accumulating evidence indicates that the brain is equipped with specific mechanisms for compensating for the global motion induced by these eye movements. However, it is not yet fully understood how the retina processes global motion images during eye movements. Here we show that global motion images evoke novel coordinated firing in retinal ganglion cells (GCs). We simultaneously recorded the firing of GCs in the goldfish isolated retina using a multi-electrode array, and classified each GC based on the temporal profile of its receptive field (RF). A moving target that accompanied the global motion (simulating a saccade following a period of fixational eye movements) modulated the RF properties and evoked synchronized and correlated firing among local clusters of the specific GCs. Our findings provide a novel concept for retinal information processing during eye movements.

Neurons in primary visual cortex represent distribution of luminance

To efficiently detect a wide range of light-intensity changes, visual neurons must adapt to ambient luminance. However, how neurons in the primary visual cortex (V1) code the distribution of luminance remains unknown. We designed stimuli that represent rapid changes in luminance under different luminance distributions and investigated V1 neuron responses to these novel stimuli. We demonstrate that V1 neurons represent luminance changes by dynamically adjusting their responses when the luminance distribution changes. Many cells (35%) detected luminance changes by responding to dark stimuli when the distribution was dominated by bright stimuli, bright stimuli when dominated by dark stimuli, and both dark and bright stimuli when dominated by intermediate luminance stimuli; 13% of cells signaled the mean luminance that was varied with different distributions; the remaining 52% of cells gradually shifted the responses that were most sensitive to luminance changes when the luminance distribution varied. The remarkable response changes of the former two cell groups suggest their crucial roles in detecting luminance changes. These response characteristics demonstrate that V1 neurons are not only sensitive to luminance change, but also luminance distribution change. They encode luminance changes according to the luminance distribution. Mean cells represent the prevailing luminance and reversal cells represent the salient stimuli in the environment.

Analysis of Neuronal Spike Trains, Deconstructed

As information flows through the brain, neuronal firing progresses from encoding the world as sensed by the animal to driving the motor output of subsequent behavior. One of the more tractable goals of quantitative neuroscience is to develop predictive models that relate the sensory or motor streams with neuronal firing. Here we review and contrast analytical tools used to accomplish this task. We focus on classes of models in which the external variable is compared with one or more feature vectors to extract a low-dimensional representation, the history of spiking and other variables are potentially incorporated, and these factors are nonlinearly transformed to predict the occurrences of spikes. We illustrate these techniques in application to datasets of different degrees of complexity. In particular, we address the fitting of models in the presence of strong correlations in the external variable, as occurs in natural sensory stimuli and in movement. Spectral correlation between predicted and measured spike trains is introduced to contrast the relative success of different methods.

Neural correlations enable invariant coding and perception of natural stimuli in weakly electric fish

Neural representations of behaviorally relevant stimulus features displaying invariance with respect to different contexts are essential for perception. However, the mechanisms mediating their emergence and subsequent refinement remain poorly understood in general. Here, we demonstrate that correlated neural activity allows for the emergence of an invariant representation of natural communication stimuli that is further refined across successive stages of processing in the weakly electric fish Apteronotus leptorhynchus. Importantly, different patterns of input resulting from the same natural communication stimulus occurring in different contexts all gave rise to similar behavioral responses. Our results thus reveal how a generic neural circuit performs an elegant computation that mediates the emergence and refinement of an invariant neural representation of natural stimuli that most likely constitutes a neural correlate of perception.

DOI:http://dx.doi.org/10.7554/eLife.12993.001

We can effortlessly recognize an object – a car, for example – in many different contexts such as when seen from behind, under different lighting levels or even from different viewpoints. This phenomenon is known as perceptual invariance: objects are correctly recognized, despite variations in exactly what is seen (or otherwise sensed). However, it is still not clear how the brain processes perceptual information to recognize the same object under a wide variety of contexts.

Some fish, such as the brown ghost knifefish, produce a weak electric signal that they can alter to communicate with other members of their species. A communication call may be produced in a variety of contexts that alter which aspects of the signal nearby fish detect. Despite this, fish tend to respond to a given communication call in the same way regardless of its context; this suggests that these fish also have perceptual invariance.

The communication calls of weakly electric fish can be easily mimicked in a laboratory and produce reliable behavioral responses, which makes these fish a good model for understanding how perceptual invariance might be coded in the brain. Therefore, Metzen et al. recorded the activity of the receptor neurons that first respond to communication calls in weakly electric fish. The results revealed that a given communication signal made the firing patterns of all receptor neurons in the fish’s brain more similar to each other, regardless of the signal’s context. This occurs despite the changes in context causing single receptor neurons to respond in different ways. At each stage of the process by which information is transmitted from the receptor neurons to neurons deeper in the brain, the similarity in the neurons’ firing patterns is refined, thereby giving rise to perceptual invariance.

While perceptual invariance to a given object in different contexts is desirable, it is also important to be able to distinguish between different objects. This implies that neurons should respond similarly to stimuli associated with the same object and differently to stimuli associated with different objects. Further studies are now needed to confirm whether this is the case.

DOI:http://dx.doi.org/10.7554/eLife.12993.002

Ribbon Synapses and Visual Processing in the Retina

The first synapses transmitting visual information contain an unusual organelle, the ribbon, which is involved in the transport and priming of vesicles to be released at the active zone. The ribbon is one of many design features that allow efficient refilling of the active zone, which in turn enables graded changes in membrane potential to be transmitted using a continuous mode of neurotransmitter release. The ribbon also plays a key role in supplying vesicles for rapid and transient bursts of release that signal fast changes, such as the onset of light. We increasingly understand how the physiological properties of ribbon synapses determine basic transformations of the visual signal and, in particular, how the process of refilling the active zone regulates the gain and adaptive properties of the retinal circuit. The molecular basis of ribbon function is, however, far from clear.

Electrosensory Midbrain Neurons Display Feature Invariant Responses to Natural Communication Stimuli

Neurons that respond selectively but in an invariant manner to a given feature of natural stimuli have been observed across species and systems. Such responses emerge in higher brain areas, thereby suggesting that they occur by integrating afferent input. However, the mechanisms by which such integration occurs are poorly understood. Here we show that midbrain electrosensory neurons can respond selectively and in an invariant manner to heterogeneity in behaviorally relevant stimulus waveforms. Such invariant responses were not seen in hindbrain electrosensory neurons providing afferent input to these midbrain neurons, suggesting that response invariance results from nonlinear integration of such input. To test this hypothesis, we built a model based on the Hodgkin-Huxley formalism that received realistic afferent input. We found that multiple combinations of parameter values could give rise to invariant responses matching those seen experimentally. Our model thus shows that there are multiple solutions towards achieving invariant responses and reveals how subthreshold membrane conductances help promote robust and invariant firing in response to heterogeneous stimulus waveforms associated with behaviorally relevant stimuli. We discuss the implications of our findings for the electrosensory and other systems.

Spike-Triggered Covariance Analysis Reveals Phenomenological Diversity of Contrast Adaptation in the Retina

When visual contrast changes, retinal ganglion cells adapt by adjusting their sensitivity as well as their temporal filtering characteristics. The latter has classically been described by contrast-induced gain changes that depend on temporal frequency. Here, we explored a new perspective on contrast-induced changes in temporal filtering by using spike-triggered covariance analysis to extract multiple parallel temporal filters for individual ganglion cells. Based on multielectrode-array recordings from ganglion cells in the isolated salamander retina, we found that contrast adaptation of temporal filtering can largely be captured by contrast-invariant sets of filters with contrast-dependent weights. Moreover, differences among the ganglion cells in the filter sets and their contrast-dependent contributions allowed us to phenomenologically distinguish three types of filter changes. The first type is characterized by newly emerging features at higher contrast, which can be reproduced by computational models that contain response-triggered gain-control mechanisms. The second type follows from stronger adaptation in the Off pathway as compared to the On pathway in On-Off-type ganglion cells. Finally, we found that, in a subset of neurons, contrast-induced filter changes are governed by particularly strong spike-timing dynamics, in particular by pronounced stimulus-dependent latency shifts that can be observed in these cells. Together, our results show that the contrast dependence of temporal filtering in retinal ganglion cells has a multifaceted phenomenology and that a multi-filter analysis can provide a useful basis for capturing the underlying signal-processing dynamics.

Our sensory systems have to process stimuli under a wide range of environmental conditions. To cope with this challenge, the involved neurons adapt by adjusting their signal processing to the recently encountered intensity range. In the visual system, one finds, for example, that higher visual contrast leads to changes in how visual signals are temporally filtered, making signal processing faster and more band-pass-like at higher contrast. By analyzing signals from neurons in the retina of salamanders, we here found that these adaptation effects can be described by a fixed set of filters, independent of contrast, whose relative contributions change with contrast. Also, we found that different phenomena contribute to this adaptation. In particular, some cells change their relative sensitivity to light increments and light decrements, whereas other cells are influenced by a strong contrast-dependence of the exact timing of their responses. Our results show that contrast adaptation in the retina is not an entirely homogeneous phenomenon, and that models with multiple filters can help in characterizing sensory adaptation.

Ambient illumination switches contrast preference of specific retinal processing streams

Contrast, a fundamental feature of visual scenes, is encoded in a distributed manner by ∼ 20 retinal ganglion cell (RGC) types, which stream visual information to the brain. RGC types respond preferentially to positive (ON(pref)) or negative (OFF(pref)) contrast and differ in their sensitivity to preferred contrast and responsiveness to nonpreferred stimuli. Vision operates over an enormous range of mean light levels. The influence of ambient illumination on contrast encoding across RGC types is not well understood. Here, we used large-scale multielectrode array recordings to characterize responses of mouse RGCs under lighting conditions spanning five orders in brightness magnitude. We identify three functional RGC types that switch contrast preference in a luminance-dependent manner (Sw1-, Sw2-, and Sw3-RGCs). As ambient illumination increases, Sw1- and Sw2-RGCs shift from ON(pref) to OFF(pref) and Sw3-RGCs from OFF(pref) to ON(pref). In all cases, transitions in contrast preference are reversible and track light levels. By mapping spatiotemporal receptive fields at different mean light levels, we find that changes in input from ON and OFF pathways in receptive field centers underlie shifts in contrast preference. Sw2-RGCs exhibit direction-selective responses to motion stimuli. Despite changing contrast preference, direction selectivity of Sw2-RGCs and other RGCs as well as orientation-selective responses of RGCs remain stable across light levels.

Real-time emulator for reproducing graded potentials in vertebrate retina

In most parts of the retina, neuronal circuits process visual signals represented by slowly changing membrane potentials, or so-called graded potentials. A feasible approach to speculate about the functional roles of retinal neuronal circuits is to reproduce the graded potentials of retinal neurons in response to natural scenes. In this study, we developed a simulation platform for reproducing graded potentials with the following features: real-time reproduction of retinal neural activities in response to natural scenes, a configurable model structure, and compact hardware. The spatio-temporal properties of neurons were emulated efficiently by a mixed analog-digital architecture that consisted of analog resistive networks and a field-programmable gate array. The neural activities on sustained and transient pathways were emulated from 128 × 128 inputs at 200 frames per second.

The neural circuit mechanisms underlying the retinal response to motion reversal

To make up for delays in visual processing, retinal circuitry effectively predicts that a moving object will continue moving in a straight line, allowing retinal ganglion cells to anticipate the object's position. However, a sudden reversal of motion triggers a synchronous burst of firing from a large group of ganglion cells, possibly signaling a violation of the retina's motion prediction. To investigate the neural circuitry underlying this response, we used a combination of multielectrode array and whole-cell patch recordings to measure the responses of individual retinal ganglion cells in the tiger salamander to reversing stimuli. We found that different populations of ganglion cells were responsible for responding to the reversal of different kinds of objects, such as bright versus dark objects. Using pharmacology and designed stimuli, we concluded that ON and OFF bipolar cells both contributed to the reversal response, but that amacrine cells had, at best, a minor role. This allowed us to formulate an adaptive cascade model (ACM), similar to the one previously used to describe ganglion cell responses to motion onset. By incorporating the ON pathway into the ACM, we were able to reproduce the time-varying firing rate of fast OFF ganglion cells for all experimentally tested stimuli. Analysis of the ACM demonstrates that bipolar cell gain control is primarily responsible for generating the synchronized retinal response, as individual bipolar cells require a constant time delay before recovering from gain control.

Interacting linear and nonlinear characteristics produce population coding asymmetries between ON and OFF cells in the retina

The early visual system is a model for understanding the roles of cell populations in parallel processing. Cells in this system can be classified according to their responsiveness to different stimuli; a prominent example is the division between cells that respond to stimuli of opposite contrasts (ON vs OFF cells). These two cell classes display many asymmetries in their physiological characteristics (including temporal characteristics, spatial characteristics, and nonlinear characteristics) that, individually, are known to have important roles in population coding. Here we describe a novel distinction between the information that ON and OFF ganglion cell populations carry in mouse--that OFF cells are able to signal motion information about both light and dark objects, while ON cells have a selective deficit at signaling the motion of dark objects. We found that none of the previously reported asymmetries in physiological characteristics could account for this distinction. We therefore analyzed its basis via a recently developed linear-nonlinear-Poisson model that faithfully captures input/output relationships for a broad range of stimuli (Bomash et al., 2013). While the coding differences between ON and OFF cell populations could not be ascribed to the linear or nonlinear components of the model individually, they had a simple explanation in the way that these components interact. Sensory transformations in other systems can likewise be described by these models, and thus our findings suggest that similar interactions between component properties may help account for the roles of cell classes in population coding more generally.

Inferring nonlinear neuronal computation based on physiologically plausible inputs

The computation represented by a sensory neuron's response to stimuli is constructed from an array of physiological processes both belonging to that neuron and inherited from its inputs. Although many of these physiological processes are known to be nonlinear, linear approximations are commonly used to describe the stimulus selectivity of sensory neurons (i.e., linear receptive fields). Here we present an approach for modeling sensory processing, termed the Nonlinear Input Model (NIM), which is based on the hypothesis that the dominant nonlinearities imposed by physiological mechanisms arise from rectification of a neuron's inputs. Incorporating such 'upstream nonlinearities' within the standard linear-nonlinear (LN) cascade modeling structure implicitly allows for the identification of multiple stimulus features driving a neuron's response, which become directly interpretable as either excitatory or inhibitory. Because its form is analogous to an integrate-and-fire neuron receiving excitatory and inhibitory inputs, model fitting can be guided by prior knowledge about the inputs to a given neuron, and elements of the resulting model can often result in specific physiological predictions. Furthermore, by providing an explicit probabilistic model with a relatively simple nonlinear structure, its parameters can be efficiently optimized and appropriately regularized. Parameter estimation is robust and efficient even with large numbers of model components and in the context of high-dimensional stimuli with complex statistical structure (e.g. natural stimuli). We describe detailed methods for estimating the model parameters, and illustrate the advantages of the NIM using a range of example sensory neurons in the visual and auditory systems. We thus present a modeling framework that can capture a broad range of nonlinear response functions while providing physiologically interpretable descriptions of neural computation.

Alert response to motion onset in the retina

Previous studies have shown that motion onset is very effective at capturing attention and is more salient than smooth motion. Here, we find that this salience ranking is present already in the firing rate of retinal ganglion cells. By stimulating the retina with a bar that appears, stays still, and then starts moving, we demonstrate that a subset of salamander retinal ganglion cells, fast OFF cells, responds significantly more strongly to motion onset than to smooth motion. We refer to this phenomenon as an alert response to motion onset. We develop a computational model that predicts the time-varying firing rate of ganglion cells responding to the appearance, onset, and smooth motion of a bar. This model, termed the adaptive cascade model, consists of a ganglion cell that receives input from a layer of bipolar cells, represented by individual rectified subunits. Additionally, both the bipolar and ganglion cells have separate contrast gain control mechanisms. This model captured the responses to our different motion stimuli over a wide range of contrasts, speeds, and locations. The alert response to motion onset, together with its computational model, introduces a new mechanism of sophisticated motion processing that occurs early in the visual system.

Computing complex visual features with retinal spike times

Neurons in sensory systems can represent information not only by their firing rate, but also by the precise timing of individual spikes. For example, certain retinal ganglion cells, first identified in the salamander, encode the spatial structure of a new image by their first-spike latencies. Here we explore how this temporal code can be used by downstream neural circuits for computing complex features of the image that are not available from the signals of individual ganglion cells. To this end, we feed the experimentally observed spike trains from a population of retinal ganglion cells to an integrate-and-fire model of post-synaptic integration. The synaptic weights of this integration are tuned according to the recently introduced tempotron learning rule. We find that this model neuron can perform complex visual detection tasks in a single synaptic stage that would require multiple stages for neurons operating instead on neural spike counts. Furthermore, the model computes rapidly, using only a single spike per afferent, and can signal its decision in turn by just a single spike. Extending these analyses to large ensembles of simulated retinal signals, we show that the model can detect the orientation of a visual pattern independent of its phase, an operation thought to be one of the primitives in early visual processing. We analyze how these computations work and compare the performance of this model to other schemes for reading out spike-timing information. These results demonstrate that the retina formats spatial information into temporal spike sequences in a way that favors computation in the time domain. Moreover, complex image analysis can be achieved already by a simple integrate-and-fire model neuron, emphasizing the power and plausibility of rapid neural computing with spike times.

Features and functions of nonlinear spatial integration by retinal ganglion cells

Ganglion cells in the vertebrate retina integrate visual information over their receptive fields. They do so by pooling presynaptic excitatory inputs from typically many bipolar cells, which themselves collect inputs from several photoreceptors. In addition, inhibitory interactions mediated by horizontal cells and amacrine cells modulate the structure of the receptive field. In many models, this spatial integration is assumed to occur in a linear fashion. Yet, it has long been known that spatial integration by retinal ganglion cells also incurs nonlinear phenomena. Moreover, several recent examples have shown that nonlinear spatial integration is tightly connected to specific visual functions performed by different types of retinal ganglion cells. This work discusses these advances in understanding the role of nonlinear spatial integration and reviews recent efforts to quantitatively study the nature and mechanisms underlying spatial nonlinearities. These new insights point towards a critical role of nonlinearities within ganglion cell receptive fields for capturing responses of the cells to natural and behaviorally relevant visual stimuli. In the long run, nonlinear phenomena of spatial integration may also prove important for implementing the actual neural code of retinal neurons when designing visual prostheses for the eye.

Spike-triggered covariance: geometric proof, symmetry properties, and extension beyond Gaussian stimuli

The space of sensory stimuli is complex and high-dimensional. Yet, single neurons in sensory systems are typically affected by only a small subset of the vast space of all possible stimuli. A proper understanding of the input–output transformation represented by a given cell therefore requires the identification of the subset of stimuli that are relevant in shaping the neuronal response. As an extension to the commonly-used spike-triggered average, the analysis of the spike-triggered covariance matrix provides a systematic methodology to detect relevant stimuli. As originally designed, the consistency of this method is guaranteed only if stimuli are drawn from a Gaussian distribution. Here we present a geometric proof of consistency, which provides insight into the foundations of the method, in particular, into the crucial role played by the geometry of stimulus space and symmetries in the stimulus–response relation. This approach leads to a natural extension of the applicability of the spike-triggered covariance technique to arbitrary spherical or elliptic stimulus distributions. The extension only requires a subtle modification of the original prescription. Furthermore, we present a new resampling method for assessing statistical significance of identified relevant stimuli, applicable to spherical and elliptic stimulus distributions. Finally, we exemplify the modified method and compare it to other prescriptions given in the literature.

Parallel coding of first- and second-order stimulus attributes by midbrain electrosensory neurons

Natural stimuli often have time-varying first-order (i.e., mean) and second-order (i.e., variance) attributes that each carry critical information for perception and can vary independently over orders of magnitude. Experiments have shown that sensory systems continuously adapt their responses based on changes in each of these attributes. This adaptation creates ambiguity in the neural code as multiple stimuli may elicit the same neural response. While parallel processing of first- and second-order attributes by separate neural pathways is sufficient to remove this ambiguity, the existence of such pathways and the neural circuits that mediate their emergence have not been uncovered to date. We recorded the responses of midbrain electrosensory neurons in the weakly electric fish Apteronotus leptorhynchus to stimuli with first- and second-order attributes that varied independently in time. We found three distinct groups of midbrain neurons: the first group responded to both first- and second-order attributes, the second group responded selectively to first-order attributes, and the last group responded selectively to second-order attributes. In contrast, all afferent hindbrain neurons responded to both first- and second-order attributes. Using computational analyses, we show how inputs from a heterogeneous population of ON- and OFF-type afferent neurons are combined to give rise to response selectivity to either first- or second-order stimulus attributes in midbrain neurons. Our study thus uncovers, for the first time, generic and widely applicable mechanisms by which parallel processing of first- and second-order stimulus attributes emerges in the brain.

Neural adaptation facilitates oscillatory responses to static inputs in a recurrent network of ON and OFF cells

We investigate the role of adaptation in a neural field model, composed of ON and OFF cells, with delayed all-to-all recurrent connections. As external spatially profiled inputs drive the network, ON cells receive inputs directly, while OFF cells receive an inverted image of the original signals. Via global and delayed inhibitory connections, these signals can cause the system to enter states of sustained oscillatory activity. We perform a bifurcation analysis of our model to elucidate how neural adaptation influences the ability of the network to exhibit oscillatory activity. We show that slow adaptation encourages input-induced rhythmic states by decreasing the Andronov-Hopf bifurcation threshold. We further determine how the feedback and adaptation together shape the resonant properties of the ON and OFF cell network and how this affects the response to time-periodic input. By introducing an additional frequency in the system, adaptation alters the resonance frequency by shifting the peaks where the response is maximal. We support these results with numerical experiments of the neural field model. Although developed in the context of the circuitry of the electric sense, these results are applicable to any network of spontaneously firing cells with global inhibitory feedback to themselves, in which a fraction of these cells receive external input directly, while the remaining ones receive an inverted version of this input via feedforward di-synaptic inhibition. Thus the results are relevant beyond the many sensory systems where ON and OFF cells are usually identified, and provide the backbone for understanding dynamical network effects of lateral connections and various forms of ON/OFF responses.

Responses of recurrent nets of asymmetric ON and OFF cells

A neural field model of ON and OFF cells with all-to-all inhibitory feedback is investigated. External spatiotemporal stimuli drive the ON and OFF cells with, respectively, direct and inverted polarity. The dynamic differences between networks built of ON and OFF cells ("ON/OFF") and those having only ON cells ("ON/ON") are described for the general case where ON and OFF cells can have different spontaneous firing rates; this asymmetric case is generic. Neural responses to nonhomogeneous static and time-periodic inputs are analyzed in regimes close to and away from self-oscillation. Static stimuli can cause oscillatory behavior for certain asymmetry levels. Time-periodic stimuli expose dynamical differences between ON/OFF and ON/ON nets. Outside the stimulated region, we show that ON/OFF nets exhibit frequency doubling, while ON/ON nets cannot. On the other hand, ON/ON networks show antiphase responses between stimulated and unstimulated regions, an effect that does not rely on specific receptive field circuitry. An analysis of the resonance properties of both net types reveals that ON/OFF nets exhibit larger response amplitude. Numerical simulations of the neural field models agree with theoretical predictions for localized static and time-periodic forcing. This is also the case for simulations of a network of noisy integrate-and-fire neurons. We finally discuss the application of the model to the electrosensory system and to frequency-doubling effects in retina.

Non-centered spike-triggered covariance analysis reveals neurotrophin-3 as a developmental regulator of receptive field properties of ON-OFF retinal ganglion cells

The functional separation of ON and OFF pathways, one of the fundamental features of the visual system, starts in the retina. During postnatal development, some retinal ganglion cells (RGCs) whose dendrites arborize in both ON and OFF sublaminae of the inner plexiform layer transform into RGCs with dendrites that monostratify in either the ON or OFF sublamina, acquiring final dendritic morphology in a subtype-dependent manner. Little is known about how the receptive field (RF) properties of ON, OFF, and ON-OFF RGCs mature during this time because of the lack of a reliable and efficient method to classify RGCs into these subtypes. To address this deficiency, we developed an innovative variant of Spike Triggered Covariance (STC) analysis, which we term Spike Triggered Covariance – Non-Centered (STC-NC) analysis. Using a multi-electrode array (MEA), we recorded the responses of a large population of mouse RGCs to a Gaussian white noise stimulus. As expected, the Spike-Triggered Average (STA) fails to identify responses driven by symmetric static nonlinearities such as those that underlie ON-OFF center RGC behavior. The STC-NC technique, in contrast, provides an efficient means to identify ON-OFF responses and quantify their RF center sizes accurately. Using this new tool, we find that RGCs gradually develop sensitivity to focal stimulation after eye opening, that the percentage of ON-OFF center cells decreases with age, and that RF centers of ON and ON-OFF cells become smaller. Importantly, we demonstrate for the first time that neurotrophin-3 (NT-3) regulates the development of physiological properties of ON-OFF center RGCs. Overexpression of NT-3 leads to the precocious maturation of RGC responsiveness and accelerates the developmental decrease of RF center size in ON-OFF cells. In summary, our study introduces STC-NC analysis which successfully identifies subtype RGCs and demonstrates how RF development relates to a neurotrophic driver in the retina.

The developmental separation of ON and OFF pathways is one of the fundamental features of the visual system. In the mouse retina, some bi-stratified ON-OFF RGCs are refined into mono-stratified ON or OFF RGCs during the first postnatal month. However, the process by which the RGCs' physiological receptive field properties mature remains incompletely characterized, mainly due to the lack of a reliable and efficient method to classify RGCs into different subtypes. Here we have developed an innovative analysis, Spike Triggered Covariance – Non-Centered (STC-NC), and demonstrated that this technique can accurately characterize the receptive field properties of ON, OFF and ON-OFF center cells. We show that, in wildtype mouse, RGCs gradually develop sensitivity to focal stimulation after eye opening, and the development of ON-OFF receptive field center properties correlates well with their dendritic laminar refinement. Furthermore, overexpression of NT-3 accelerates the developmental decrease of receptive field center size in ON-OFF cells. Our study is the first to establish the STC-NC analysis which can successfully identify ON-OFF subtype RGCs and to demonstrate how receptive field development relates to a neurotrophic driver in the retina.

Retina is structured to process an excess of darkness in natural scenes

Retinal ganglion cells that respond selectively to a dark spot on a brighter background (OFF cells) have smaller dendritic fields than their ON counterparts and are more numerous. OFF cells also branch more densely, and thus collect more synapses per visual angle. That the retina devotes more resources to processing dark contrasts predicts that natural images contain more dark information. We confirm this across a range of spatial scales and trace the origin of this phenomenon to the statistical structure of natural scenes. We show that the optimal mosaics for encoding natural images are also asymmetric, with OFF elements smaller and more numerous, matching retinal structure. Finally, the concentration of synapses within a dendritic field matches the information content, suggesting a simple principle to connect a concrete fact of neuroanatomy with the abstract concept of information: equal synapses for equal bits.

Eye smarter than scientists believed: neural computations in circuits of the retina

We rely on our visual system to cope with the vast barrage of incoming light patterns and to extract features from the scene that are relevant to our well-being. The necessary reduction of visual information already begins in the eye. In this review, we summarize recent progress in understanding the computations performed in the vertebrate retina and how they are implemented by the neural circuitry. A new picture emerges from these findings that helps resolve a vexing paradox between the retina's structure and function. Whereas the conventional wisdom treats the eye as a simple prefilter for visual images, it now appears that the retina solves a diverse set of specific tasks and provides the results explicitly to downstream brain areas.

The firing statistics of Poisson neuron models driven by slow stimuli

The coding properties of cells with different types of receptive fields have been studied for decades. ON-type neurons fire in response to positive fluctuations of the time-dependent stimulus, whereas OFF cells are driven by negative stimulus segments. Biphasic cells, in turn, are selective to up/down or down/up stimulus upstrokes. In this article, we explore the way in which different receptive fields affect the firing statistics of Poisson neuron models, when driven with slow stimuli. We find analytical expressions for the time-dependent peri-stimulus time histogram and the inter-spike interval distribution in terms of the incoming signal. Our results enable us to understand the interplay between the intrinsic and extrinsic factors that regulate the statistics of spike trains. The former depend on biophysical neural properties, whereas the latter hinge on the temporal characteristics of the input signal.

Ideal observer analysis of signal quality in retinal circuits

The function of the retina is crucial, for it must encode visual signals so the brain can detect objects in the visual world. However, the biological mechanisms of the retina add noise to the visual signal and therefore reduce its quality and capacity to inform about the world. Because an organism's survival depends on its ability to unambiguously detect visual stimuli in the presence of noise, its retinal circuits must have evolved to maximize signal quality, suggesting that each retinal circuit has a specific functional role. Here we explain how an ideal observer can measure signal quality to determine the functional roles of retinal circuits. In a visual discrimination task the ideal observer can measure from a neural response the increment threshold, the number of distinguishable response levels, and the neural code, which are fundamental measures of signal quality relevant to behavior. It can compare the signal quality in stimulus and response to determine the optimal stimulus, and can measure the specific loss of signal quality by a neuron's receptive field for non-optimal stimuli. Taking into account noise correlations, the ideal observer can track the signal-to-noise ratio available from one stage to the next, allowing one to determine each stage's role in preserving signal quality. A comparison between the ideal performance of the photon flux absorbed from the stimulus and actual performance of a retinal ganglion cell shows that in daylight a ganglion cell and its presynaptic circuit loses a factor of approximately 10-fold in contrast sensitivity, suggesting specific signal-processing roles for synaptic connections and other neural circuit elements. The ideal observer is a powerful tool for characterizing signal processing in single neurons and arrays along a neural pathway.