Spike-triggered covariance: geometric proof, symmetry properties, and extension beyond Gaussian stimuli

Filter-based models of suppression in retinal ganglion cells: Comparison and generalization across species and stimuli

The dichotomy of excitation and suppression is one of the canonical mechanisms explaining the complexity of neural activity. Computational models of the interplay of excitation and suppression in single neurons aim at investigating how this interaction affects a neuron's spiking responses and shapes the encoding of sensory stimuli. Here, we compare the performance of three filter-based stimulus-encoding models for predicting retinal ganglion cell responses recorded from axolotl, mouse, and marmoset retina to different types of temporally varying visual stimuli. Suppression in these models is implemented via subtractive or divisive interactions of stimulus filters or by a response-driven feedback module. For the majority of ganglion cells, the subtractive and divisive models perform similarly and outperform the feedback model as well as a linear-nonlinear (LN) model with no suppression. Comparison between the subtractive and the divisive model depends on cell type, species, and stimulus components, with the divisive model generalizing best across temporal stimulus frequencies and visual contrast and the subtractive model capturing in particular responses for slow temporal stimulus dynamics and for slow axolotl cells. Overall, we conclude that the divisive and subtractive models are well suited for capturing interactions of excitation and suppression in ganglion cells and perform best for different temporal regimes of these interactions.

KV1 Channels Enable Myelinated Axons to Transmit Spikes Reliably without Spiking Ectopically

Action potentials (spikes) are regenerated at each node of Ranvier during saltatory transmission along a myelinated axon. The high density of voltage-gated sodium channels required by nodes to reliably transmit spikes increases the risk of ectopic spike generation in the axon. Here we show that ectopic spiking is avoided because KV1 channels prevent nodes from responding to slow depolarization; instead, axons respond selectively to rapid depolarization because KV1 channels implement a high-pass filter. To characterize this filter, we compared spike initiation properties in the soma and axon of CA1 pyramidal neurons from mice of both sexes, using spatially restricted photoactivation of channelrhodopsin-2 (ChR2) to evoke spikes in either region while simultaneously recording at the soma. Somatic photostimulation evoked repetitive spiking whereas axonal photostimulation evoked transient spiking. Blocking KV1 channels converted the axon photostimulation response to repetitive spiking and encouraged spontaneous ectopic spike initiation in the axon. According to computational modeling, the high-pass filter implemented by KV1 channels matches the axial current waveform associated with saltatory conduction, enabling axons to faithfully transmit digital signals by maximizing their signal-to-noise ratio for this task. Specifically, a node generates a single spike only when rapidly depolarized, which is precisely what occurs during saltatory conduction when a pulse of axial current (triggered by a spike occurring at the upstream node) reaches the next node. The soma and axon use distinct spike initiation mechanisms (filters) appropriate for the task required of each region, namely, analog-to-digital transduction in the soma versus digital signal transmission in the axon.

Spike-triggered Clustering for Retinal Ganglion Cell Classification

Retinal ganglion cells (RGCs), the retina's output neurons, encode visual information through spiking. The RGC receptive field (RF) represents the basic unit of visual information processing in the retina. RFs are commonly estimated using the spike-triggered average (STA), which is the average of the stimulus patterns to which a given RGC is sensitive. Whereas STA, based on the concept of the average, is simple and intuitive, it leaves more complex structures in the RFs undetected. Alternatively, spike-triggered covariance (STC) analysis provides information on second-order RF statistics. However, STC is computationally cumbersome and difficult to interpret. Thus, the objective of this study was to propose and validate a new computational method, called spike-triggered clustering (STCL), specific for multimodal RFs. Specifically, RFs were fit with a Gaussian mixture model, which provides the means and covariances of multiple RF clusters. The proposed method recovered bipolar stimulus patterns in the RFs of ON-OFF cells, while the STA identified only ON and OFF RGCs, and the remaining RGCs were labeled as unknown types. In contrast, our new STCL analysis distinguished ON-OFF RGCs from the ON, OFF, and unknown RGC types classified by STA. Thus, the proposed method enables us to include ON-OFF RGCs prior to retinal information analysis.

Time- and frequency-resolved covariance analysis for detection and characterization of seizures from intracraneal EEG recordings

The amount of power in different frequency bands of the electroencephalogram (EEG) carries information about the behavioral state of a subject. Hence, neurologists treating epileptic patients monitor the temporal evolution of the different bands. We propose a covariance-based method to detect and characterize epileptic seizures operating on the band-filtered EEG signal. The algorithm is unsupervised and performs a principal component analysis of intra-cranial EEG recordings, detecting transient fluctuations of the power in each frequency band. Its simplicity makes it suitable for online implementation. Good sampling of the non-ictal periods is required, while no demands are imposed on the amount of data during ictal activity. We tested the method with 32 seizures registered in 5 patients. The area below the resulting receiver-operating characteristic curves was 87% for the detection of seizures and 91% for the detection of recruited electrodes. To identify the behaviorally relevant correlates of the physiological signal, we identified transient changes in the variance of each band that were correlated with the degree of loss of consciousness, the latter assessed by the so-called Consciousness Seizure Scale, summarizing the performance of the subject in a number of behavioral tests requested during seizures. We concluded that those crisis with maximal impairment of consciousness tended to exhibit an increase in variance approximately 40 s after seizure onset, with predominant power in the theta and alpha bands and reduced delta and beta activity.

The statistics of how natural images drive the responses of neurons

To model the responses of neurons in the early visual system, at least three basic components are required: a receptive field, a normalization term, and a specification of encoding noise. Here, we examine how the receptive field, the normalization factor, and the encoding noise affect the drive to model-neuron responses when stimulated with natural images. We show that when these components are modeled appropriately, the response drives elicited by natural stimuli are Gaussian-distributed and scale invariant, and very nearly maximize the sensitivity (d') for natural-image discrimination. We discuss the statistical models of natural stimuli that can account for these response statistics, and we show how some commonly used modeling practices may distort these results. Finally, we show that normalization can equalize important properties of neural response across different stimulus types. Specifically, narrowband (stimulus- and feature-specific) normalization causes model neurons to yield Gaussian response-drive statistics when stimulated with natural stimuli, 1/f noise stimuli, and white-noise stimuli. The current work makes recommendations for best practices and lays a foundation, grounded in the response statistics to natural stimuli, upon which to build principled models of more complex visual tasks.

New Features of Receptive Fields in Mouse Retina through Spike-triggered Covariance

Retinal ganglion cells (RGCs) encode various spatiotemporal features of visual information into spiking patterns. The receptive field (RF) of each RGC is usually calculated by spike-triggered average (STA), which is fast and easy to understand, but limited to simple and unimodal RFs. As an alternative, spike-triggered covariance (STC) has been proposed to characterize more complex patterns in RFs. This study compares STA and STC for the characterization of RFs and demonstrates that STC has an advantage over STA for identifying novel spatiotemporal features of RFs in mouse RGCs. We first classified mouse RGCs into ON, OFF, and ON/OFF cells according to their response to full-field light stimulus, and then investigated the spatiotemporal patterns of RFs with random checkerboard stimulation, using both STA and STC analysis. We propose five sub-types (T1–T5) in the STC of mouse RGCs together with their physiological implications. In particular, the relatively slow biphasic pattern (T1) could be related to excitatory inputs from bipolar cells. The transient biphasic pattern (T2) allows one to characterize complex patterns in RFs of ON/OFF cells. The other patterns (T3–T5), which are contrasting, alternating, and monophasic patterns, could be related to inhibitory inputs from amacrine cells. Thus, combining STA and STC and considering the proposed sub-types unveil novel characteristics of RFs in the mouse retina and offer a more holistic understanding of the neural coding mechanisms of mouse RGCs.

Functional characterization of retinal ganglion cells using tailored nonlinear modeling

The mammalian retina encodes the visual world in action potentials generated by 20-50 functionally and anatomically-distinct types of retinal ganglion cell (RGC). Individual RGC types receive synaptic input from distinct presynaptic circuits; therefore, their responsiveness to specific features in the visual scene arises from the information encoded in synaptic input and shaped by postsynaptic signal integration and spike generation. Unfortunately, there is a dearth of tools for characterizing the computations reflected in RGC spike output. Therefore, we developed a statistical model, the separable Nonlinear Input Model, to characterize the excitatory and suppressive components of RGC receptive fields. We recorded RGC responses to a correlated noise ("cloud") stimulus in an in vitro preparation of mouse retina and found that our model accurately predicted RGC responses at high spatiotemporal resolution. It identified multiple receptive fields reflecting the main excitatory and suppressive components of the response of each neuron. Significantly, our model accurately identified ON-OFF cells and distinguished their distinct ON and OFF receptive fields, and it demonstrated a diversity of suppressive receptive fields in the RGC population. In total, our method offers a rich description of RGC computation and sets a foundation for relating it to retinal circuitry.

Activity Correlations between Direction-Selective Retinal Ganglion Cells Synergistically Enhance Motion Decoding from Complex Visual Scenes

Neurons in sensory systems are often tuned to particular stimulus features. During complex naturalistic stimulation, however, multiple features may simultaneously affect neuronal responses, which complicates the readout of individual features. To investigate feature representation under complex stimulation, we studied how direction-selective ganglion cells in salamander retina respond to texture motion where direction, velocity, and spatial pattern inside the receptive field continuously change. We found that the cells preserve their direction preference under this stimulation, yet their direction encoding becomes ambiguous due to simultaneous activation by luminance changes. The ambiguities can be resolved by considering populations of direction-selective cells with different preferred directions. This gives rise to synergistic motion decoding, yielding more information from the population than the summed information from single-cell responses. Strong positive response correlations between cells with different preferred directions amplify this synergy. Our results show how correlated population activity can enhance feature extraction in complex visual scenes.

•

Direction-selective ganglion cells respond to motion as well as luminance changes

•

This obscures the readout of direction from single cells under complex texture motion

•

Population decoding improves direction readout supralinearly over individual cells

•

Strong spike correlations further enhance readout through increased synergy

Nonlinear Modeling of Neural Interaction for Spike Prediction Using the Staged Point-Process Model

Neurons communicate nonlinearly through spike activities. Generalized linear models (GLMs) describe spike activities with a cascade of a linear combination across inputs, a static nonlinear function, and an inhomogeneous Bernoulli or Poisson process, or Cox process if a self-history term is considered. This structure considers the output nonlinearity in spike generation but excludes the nonlinear interaction among input neurons. Recent studies extend GLMs by modeling the interaction among input neurons with a quadratic function, which considers the interaction between every pair of input spikes. However, quadratic effects may not fully capture the nonlinear nature of input interaction. We therefore propose a staged point-process model to describe the nonlinear interaction among inputs using a few hidden units, which follows the idea of artificial neural networks. The output firing probability conditioned on inputs is formed as a cascade of two linear-nonlinear (a linear combination plus a static nonlinear function) stages and an inhomogeneous Bernoulli process. Parameters of this model are estimated by maximizing the log likelihood on output spike trains. Unlike the iterative reweighted least squares algorithm used in GLMs, where the performance is guaranteed by the concave condition, we propose a modified Levenberg-Marquardt (L-M) algorithm, which directly calculates the Hessian matrix of the log likelihood, for the nonlinear optimization in our model. The proposed model is tested on both synthetic data and real spike train data recorded from the dorsal premotor cortex and primary motor cortex of a monkey performing a center-out task. Performances are evaluated by discrete-time rescaled Kolmogorov-Smirnov tests, where our model statistically outperforms a GLM and its quadratic extension, with a higher goodness-of-fit in the prediction results. In addition, the staged point-process model describes nonlinear interaction among input neurons with fewer parameters than quadratic models, and the modified L-M algorithm also demonstrates fast convergence.

Information-Theoretical Analysis of the Neural Code in the Rodent Temporal Lobe

In the study of the neural code, information-theoretical methods have the advantage of making no assumptions about the probabilistic mapping between stimuli and responses. In the sensory domain, several methods have been developed to quantify the amount of information encoded in neural activity, without necessarily identifying the specific stimulus or response features that instantiate the code. As a proof of concept, here we extend those methods to the encoding of kinematic information in a navigating rodent. We estimate the information encoded in two well-characterized codes, mediated by the firing rate of neurons, and by the phase-of-firing with respect to the theta-filtered local field potential. In addition, we also consider a novel code, mediated by the delta-filtered local field potential. We find that all three codes transmit significant amounts of kinematic information, and informative neurons tend to employ a combination of codes. Cells tend to encode conjunctions of kinematic features, so that most of the informative neurons fall outside the traditional cell types employed to classify spatially-selective units. We conclude that a broad perspective on the candidate stimulus and response features expands the repertoire of strategies with which kinematic information is encoded.

Describing complex cells in primary visual cortex: a comparison of context and multifilter LN models

Receptive field (RF) models are an important tool for deciphering neural responses to sensory stimuli. The two currently popular RF models are multifilter linear-nonlinear (LN) models and context models. Models are, however, never correct, and they rely on assumptions to keep them simple enough to be interpretable. As a consequence, different models describe different stimulus-response mappings, which may or may not be good approximations of real neural behavior. In the current study, we take up two tasks: 1) we introduce new ways to estimate context models with realistic nonlinearities, that is, with logistic and exponential functions, and 2) we evaluate context models and multifilter LN models in terms of how well they describe recorded data from complex cells in cat primary visual cortex. Our results, based on single-spike information and correlation coefficients, indicate that context models outperform corresponding multifilter LN models of equal complexity (measured in terms of number of parameters), with the best increase in performance being achieved by the novel context models. Consequently, our results suggest that the multifilter LN-model framework is suboptimal for describing the behavior of complex cells: the context-model framework is clearly superior while still providing interpretable quantizations of neural behavior. NEW & NOTEWORTHY We used data from complex cells in primary visual cortex to estimate a wide variety of receptive field models from two frameworks that have previously not been compared with each other. The models included traditionally used multifilter linear-nonlinear models and novel variants of context models. Using mutual information and correlation coefficients as performance measures, we showed that context models are superior for describing complex cells and that the novel context models performed the best.

Coding Schemes in the Archerfish Optic Tectum

Many studies have yielded valuable knowledge on the early visual system but it is biased since the studies have focused on terrestrial mammals alone. Here, to better account for visual systems in different environments and animal classes, we studied the structure of early visual processing in the archerfish which harnesses its extreme visual ability to hunt by shooting water jets at prey hanging on vegetation above the water. Thus, the archerfish provides a unique opportunity to study visual processing in a vertebrate which is an expert vision-guided predator with a very different brain structure than mammals. The receptive field structures in the archerfish (both sexes) optic tectum, the main visual processing region in the fish brain, were measured and linear non-linear cascades were used to analyze their properties. The findings indicate that the spatial receptive field structures lie on a continuum between circular and elliptical shapes. In addition, the cells' functional properties display a richness of response characteristics, since many cells could be captured by more than a single linear filter. Finally, the non-linear response functions that link linear filters and neuronal responses were found to be similar to the non-linear functions of models that describe terrestrial mammalian single cell activity. Overall our results help to better understand the early visual processing system across vertebrates.

Electrical receptive fields of retinal ganglion cells: Influence of presynaptic neurons

Implantable retinal stimulators activate surviving neurons to restore a sense of vision in people who have lost their photoreceptors through degenerative diseases. Complex spatial and temporal interactions occur in the retina during multi-electrode stimulation. Due to these complexities, most existing implants activate only a few electrodes at a time, limiting the repertoire of available stimulation patterns. Measuring the spatiotemporal interactions between electrodes and retinal cells, and incorporating them into a model may lead to improved stimulation algorithms that exploit the interactions. Here, we present a computational model that accurately predicts both the spatial and temporal nonlinear interactions of multi-electrode stimulation of rat retinal ganglion cells (RGCs). The model was verified using in vitro recordings of ON, OFF, and ON-OFF RGCs in response to subretinal multi-electrode stimulation with biphasic pulses at three stimulation frequencies (10, 20, 30 Hz). The model gives an estimate of each cell’s spatiotemporal electrical receptive fields (ERFs); i.e., the pattern of stimulation leading to excitation or suppression in the neuron. All cells had excitatory ERFs and many also had suppressive sub-regions of their ERFs. We show that the nonlinearities in observed responses arise largely from activation of presynaptic interneurons. When synaptic transmission was blocked, the number of sub-regions of the ERF was reduced, usually to a single excitatory ERF. This suggests that direct cell activation can be modeled accurately by a one-dimensional model with linear interactions between electrodes, whereas indirect stimulation due to summated presynaptic responses is nonlinear.

Implantable neural stimulation devices are being widely used and clinically tested for the restoration of lost function (e.g. cochlear implants) and the treatment of neurological disorders. Smart devices that can combine sensing and stimulation will dramatically improve future patient outcomes. To this end, mathematical models that can accurately predict neural responses to electrical stimulation will be critical for the development of smart stimulation devices. Here, we demonstrate a model that predicts neural responses to simultaneous stimulation across multiple electrodes in the retina. We show that the activation of presynaptic neurons leads to nonlinearities in the responses of postsynaptic retinal ganglion cells. The model is accurate and is applicable to a wide range of neural stimulation devices.

Diversity in spatial scope of contrast adaptation among mouse retinal ganglion cells

Retinal ganglion cells adapt to changes in visual contrast by adjusting their response kinetics and sensitivity. While much work has focused on the time scales of these adaptation processes, less is known about the spatial scale of contrast adaptation. For example, do small, localized contrast changes affect a cell's signal processing across its entire receptive field? Previous investigations have provided conflicting evidence, suggesting that contrast adaptation occurs either locally within subregions of a ganglion cell's receptive field or globally over the receptive field in its entirety. Here, we investigated the spatial extent of contrast adaptation in ganglion cells of the isolated mouse retina through multielectrode-array recordings. We applied visual stimuli so that ganglion cell receptive fields contained regions where the average contrast level changed periodically as well as regions with constant average contrast level. This allowed us to analyze temporal stimulus integration and sensitivity separately for stimulus regions with and without contrast changes. We found that the spatial scope of contrast adaptation depends strongly on cell identity, with some ganglion cells displaying clear local adaptation, whereas others, in particular large transient ganglion cells, adapted globally to contrast changes. Thus, the spatial scope of contrast adaptation in mouse retinal ganglion cells appears to be cell-type specific. This could reflect differences in mechanisms of contrast adaptation and may contribute to the functional diversity of different ganglion cell types.NEW & NOTEWORTHY Understanding whether adaptation of a neuron in a sensory system can occur locally inside the receptive field or whether it always globally affects the entire receptive field is important for understanding how the neuron processes complex sensory stimuli. For mouse retinal ganglion cells, we here show that both local and global contrast adaptation exist and that this diversity in spatial scope can contribute to the functional diversity of retinal ganglion cell types.

Inference of neuronal functional circuitry with spike-triggered non-negative matrix factorization

Neurons in sensory systems often pool inputs over arrays of presynaptic cells, giving rise to functional subunits inside a neuron’s receptive field. The organization of these subunits provides a signature of the neuron’s presynaptic functional connectivity and determines how the neuron integrates sensory stimuli. Here we introduce the method of spike-triggered non-negative matrix factorization for detecting the layout of subunits within a neuron’s receptive field. The method only requires the neuron’s spiking responses under finely structured sensory stimulation and is therefore applicable to large populations of simultaneously recorded neurons. Applied to recordings from ganglion cells in the salamander retina, the method retrieves the receptive fields of presynaptic bipolar cells, as verified by simultaneous bipolar and ganglion cell recordings. The identified subunit layouts allow improved predictions of ganglion cell responses to natural stimuli and reveal shared bipolar cell input into distinct types of ganglion cells.

How a neuron integrates sensory information requires knowledge about its functional presynaptic connections. Here the authors report a new method using non-negative matrix factorization to identify the layout of presynaptic bipolar cell inputs onto retinal ganglion cells and predict their responses to natural stimuli.

Multidimensional receptive field processing by cat primary auditory cortical neurons

The receptive fields of many auditory cortical neurons are multidimensional and are best represented by more than one stimulus feature. The number of these dimensions, their characteristics, and how they differ with stimulus context have been relatively unexplored. Standard methods that are often used to characterize multidimensional stimulus selectivity, such as spike-triggered covariance (STC) or maximally informative dimensions (MIDs), are either limited to Gaussian stimuli or are only able to recover a small number of stimulus features due to data limitations. An information theoretic extension of STC, the maximum noise entropy (MNE) model, can be used with non-Gaussian stimulus distributions to find an arbitrary number of stimulus dimensions. When we applied the MNE model to auditory cortical neurons, we often found more than two stimulus features that influenced neuronal firing. Excitatory and suppressive features coded different acoustic contexts: excitatory features encoded higher temporal and spectral modulations, while suppressive features had lower modulation frequency preferences. We found that the excitatory and suppressive features themselves were sensitive to stimulus context when we employed two stimuli that differed only in their short-term correlation structure: while the linear features were similar, the secondary features were strongly affected by stimulus statistics. These results show that multidimensional receptive field processing is influenced by feature type and stimulus context.

Accuracy Maximization Analysis for Sensory-Perceptual Tasks: Computational Improvements, Filter Robustness, and Coding Advantages for Scaled Additive Noise

Accuracy Maximization Analysis (AMA) is a recently developed Bayesian ideal observer method for task-specific dimensionality reduction. Given a training set of proximal stimuli (e.g. retinal images), a response noise model, and a cost function, AMA returns the filters (i.e. receptive fields) that extract the most useful stimulus features for estimating a user-specified latent variable from those stimuli. Here, we first contribute two technical advances that significantly reduce AMA’s compute time: we derive gradients of cost functions for which two popular estimators are appropriate, and we implement a stochastic gradient descent (AMA-SGD) routine for filter learning. Next, we show how the method can be used to simultaneously probe the impact on neural encoding of natural stimulus variability, the prior over the latent variable, noise power, and the choice of cost function. Then, we examine the geometry of AMA’s unique combination of properties that distinguish it from better-known statistical methods. Using binocular disparity estimation as a concrete test case, we develop insights that have general implications for understanding neural encoding and decoding in a broad class of fundamental sensory-perceptual tasks connected to the energy model. Specifically, we find that non-orthogonal (partially redundant) filters with scaled additive noise tend to outperform orthogonal filters with constant additive noise; non-orthogonal filters and scaled additive noise can interact to sculpt noise-induced stimulus encoding uncertainty to match task-irrelevant stimulus variability. Thus, we show that some properties of neural response thought to be biophysical nuisances can confer coding advantages to neural systems. Finally, we speculate that, if repurposed for the problem of neural systems identification, AMA may be able to overcome a fundamental limitation of standard subunit model estimation. As natural stimuli become more widely used in the study of psychophysical and neurophysiological performance, we expect that task-specific methods for feature learning like AMA will become increasingly important.

In psychophysics and neurophysiology, the stimulus features that are manipulated in experiments are often selected based on intuition, trial-and-error, and historical precedence. Accuracy Maximization Analysis (AMA) is a Bayesian ideal observer method for determining the task-relevant features (i.e. filters) from natural stimuli that nervous systems should select for. In other words, AMA is a method for finding optimal receptive fields for specific tasks. Early results suggest that this method has the potential to be of fundamental importance to neuroscience and perception science. First, we develop AMA-SGD, a new version of AMA that significantly reduces filter-learning time, and use it to learn optimal filters for the classic task of binocular disparity estimation. Then, we find that measureable, task-relevant properties of natural stimuli are the most important determinants of the optimal filters; changes to the prior, cost function, and internal noise have little effect on the filters. Last, we demonstrate that some ubiquitous properties of neural systems, generally thought to be biophysical nuisances, can actually improve the fidelity of neural codes. In particular, we show for the first time that scaled additive noise and redundant (non-orthogonal) filters can interact to sculpt uncertainty due to internal noise to match task-irrelevant natural stimulus variability.

Graphics Processing Unit-Accelerated Code for Computing Second-Order Wiener Kernels and Spike-Triggered Covariance

Sensory neuroscience seeks to understand and predict how sensory neurons respond to stimuli. Nonlinear components of neural responses are frequently characterized by the second-order Wiener kernel and the closely-related spike-triggered covariance (STC). Recent advances in data acquisition have made it increasingly common and computationally intensive to compute second-order Wiener kernels/STC matrices. In order to speed up this sort of analysis, we developed a graphics processing unit (GPU)-accelerated module that computes the second-order Wiener kernel of a system's response to a stimulus. The generated kernel can be easily transformed for use in standard STC analyses. Our code speeds up such analyses by factors of over 100 relative to current methods that utilize central processing units (CPUs). It works on any modern GPU and may be integrated into many data analysis workflows. This module accelerates data analysis so that more time can be spent exploring parameter space and interpreting data.

Divisive suppression explains high-precision firing and contrast adaptation in retinal ganglion cells

Visual processing depends on specific computations implemented by complex neural circuits. Here, we present a circuit-inspired model of retinal ganglion cell computation, targeted to explain their temporal dynamics and adaptation to contrast. To localize the sources of such processing, we used recordings at the levels of synaptic input and spiking output in the in vitro mouse retina. We found that an ON-Alpha ganglion cell's excitatory synaptic inputs were described by a divisive interaction between excitation and delayed suppression, which explained nonlinear processing that was already present in ganglion cell inputs. Ganglion cell output was further shaped by spike generation mechanisms. The full model accurately predicted spike responses with unprecedented millisecond precision, and accurately described contrast adaptation of the spike train. These results demonstrate how circuit and cell-intrinsic mechanisms interact for ganglion cell function and, more generally, illustrate the power of circuit-inspired modeling of sensory processing.

How Invariant Feature Selectivity Is Achieved in Cortex

Parsing the visual scene into objects is paramount to survival. Yet, how this is accomplished by the nervous system remains largely unknown, even in the comparatively well understood visual system. It is especially unclear how detailed peripheral signal representations are transformed into the object-oriented representations that are independent of object position and are provided by the final stages of visual processing. This perspective discusses advances in computational algorithms for fitting large-scale models that make it possible to reconstruct the intermediate steps of visual processing based on neural responses to natural stimuli. In particular, it is now possible to characterize how different types of position invariance, such as local (also known as phase invariance) and more global, are interleaved with nonlinear operations to allow for coding of curved contours. Neurons in the mid-level visual area V4 exhibit selectivity to pairs of even- and odd-symmetric profiles along curved contours. Such pairing is reminiscent of the response properties of complex cells in the primary visual cortex (V1) and suggests specific ways in which V1 signals are transformed within subsequent visual cortical areas. These examples illustrate that large-scale models fitted to neural responses to natural stimuli can provide generative models of successive stages of sensory processing.

Analysis of Neuronal Spike Trains, Deconstructed

As information flows through the brain, neuronal firing progresses from encoding the world as sensed by the animal to driving the motor output of subsequent behavior. One of the more tractable goals of quantitative neuroscience is to develop predictive models that relate the sensory or motor streams with neuronal firing. Here we review and contrast analytical tools used to accomplish this task. We focus on classes of models in which the external variable is compared with one or more feature vectors to extract a low-dimensional representation, the history of spiking and other variables are potentially incorporated, and these factors are nonlinearly transformed to predict the occurrences of spikes. We illustrate these techniques in application to datasets of different degrees of complexity. In particular, we address the fitting of models in the presence of strong correlations in the external variable, as occurs in natural sensory stimuli and in movement. Spectral correlation between predicted and measured spike trains is introduced to contrast the relative success of different methods.

Nonlinear computations shaping temporal processing of precortical vision

Computations performed by the visual pathway are constructed by neural circuits distributed over multiple stages of processing, and thus it is challenging to determine how different stages contribute on the basis of recordings from single areas. In the current article, we address this problem in the lateral geniculate nucleus (LGN), using experiments combined with nonlinear modeling capable of isolating various circuit contributions. We recorded cat LGN neurons presented with temporally modulated spots of various sizes, which drove temporally precise LGN responses. We utilized simultaneously recorded S-potentials, corresponding to the primary retinal ganglion cell (RGC) input to each LGN cell, to distinguish the computations underlying temporal precision in the retina from those in the LGN. Nonlinear models with excitatory and delayed suppressive terms were sufficient to explain temporal precision in the LGN, and we found that models of the S-potentials were nearly identical, although with a lower threshold. To determine whether additional influences shaped the response at the level of the LGN, we extended this model to use the S-potential input in combination with stimulus-driven terms to predict the LGN response. We found that the S-potential input "explained away" the major excitatory and delayed suppressive terms responsible for temporal patterning of LGN spike trains but revealed additional contributions, largely PULL suppression, to the LGN response. Using this novel combination of recordings and modeling, we were thus able to dissect multiple circuit contributions to LGN temporal responses across retina and LGN, and set the foundation for targeted study of each stage.

A Simple and Accurate Model to Predict Responses to Multi-electrode Stimulation in the Retina

Implantable electrode arrays are widely used in therapeutic stimulation of the nervous system (e.g. cochlear, retinal, and cortical implants). Currently, most neural prostheses use serial stimulation (i.e. one electrode at a time) despite this severely limiting the repertoire of stimuli that can be applied. Methods to reliably predict the outcome of multi-electrode stimulation have not been available. Here, we demonstrate that a linear-nonlinear model accurately predicts neural responses to arbitrary patterns of stimulation using in vitro recordings from single retinal ganglion cells (RGCs) stimulated with a subretinal multi-electrode array. In the model, the stimulus is projected onto a low-dimensional subspace and then undergoes a nonlinear transformation to produce an estimate of spiking probability. The low-dimensional subspace is estimated using principal components analysis, which gives the neuron’s electrical receptive field (ERF), i.e. the electrodes to which the neuron is most sensitive. Our model suggests that stimulation proportional to the ERF yields a higher efficacy given a fixed amount of power when compared to equal amplitude stimulation on up to three electrodes. We find that the model captures the responses of all the cells recorded in the study, suggesting that it will generalize to most cell types in the retina. The model is computationally efficient to evaluate and, therefore, appropriate for future real-time applications including stimulation strategies that make use of recorded neural activity to improve the stimulation strategy.

Implantable multi-electrode arrays (MEAs) are used to record neurological signals and stimulate the nervous system to restore lost function (e.g. cochlear implants). MEAs that can combine both sensing and stimulation will revolutionize the development of the next generation of devices. Simple models that can accurately characterize neural responses to electrical stimulation are necessary for the development of future neuroprostheses controlled by neural feedback. We demonstrate a model that accurately predicts neural responses to concurrent stimulation across multiple electrodes. The model is simple to evaluate, making it an appropriate model for use with neural feedback. The methods described are applicable to a wide range of neural prostheses, thus greatly assisting future device development.

Understanding spike-triggered covariance using Wiener theory for receptive field identification

Receptive field identification is a vital problem in sensory neurophysiology and vision. Much research has been done in identifying the receptive fields of nonlinear neurons whose firing rate is determined by the nonlinear interactions of a small number of linear filters. Despite more advanced methods that have been proposed, spike-triggered covariance (STC) continues to be the most widely used method in such situations due to its simplicity and intuitiveness. Although the connection between STC and Wiener/Volterra kernels has often been mentioned in the literature, this relationship has never been explicitly derived. Here we derive this relationship and show that the STC matrix is actually a modified version of the second-order Wiener kernel, which incorporates the input autocorrelation and mixes first- and second-order dynamics. It is then shown how, with little modification of the STC method, the Wiener kernels may be obtained and, from them, the principal dynamic modes, a set of compact and efficient linear filters that essentially combine the spike-triggered average and STC matrix and generalize to systems with both continuous and point-process outputs. Finally, using Wiener theory, we show how these obtained filters may be corrected when they were estimated using correlated inputs. Our correction technique is shown to be superior to those commonly used in the literature for both correlated Gaussian images and natural images.

Thalamic neuron models encode stimulus information by burst-size modulation

Thalamic neurons have been long assumed to fire in tonic mode during perceptive states, and in burst mode during sleep and unconsciousness. However, recent evidence suggests that bursts may also be relevant in the encoding of sensory information. Here, we explore the neural code of such thalamic bursts. In order to assess whether the burst code is generic or whether it depends on the detailed properties of each bursting neuron, we analyzed two neuron models incorporating different levels of biological detail. One of the models contained no information of the biophysical processes entailed in spike generation, and described neuron activity at a phenomenological level. The second model represented the evolution of the individual ionic conductances involved in spiking and bursting, and required a large number of parameters. We analyzed the models' input selectivity using reverse correlation methods and information theory. We found that n-spike bursts from both models transmit information by modulating their spike count in response to changes to instantaneous input features, such as slope, phase, amplitude, etc. The stimulus feature that is most efficiently encoded by bursts, however, need not coincide with one of such classical features. We therefore searched for the optimal feature among all those that could be expressed as a linear transformation of the time-dependent input current. We found that bursting neurons transmitted 6 times more information about such more general features. The relevant events in the stimulus were located in a time window spanning ~100 ms before and ~20 ms after burst onset. Most importantly, the neural code employed by the simple and the biologically realistic models was largely the same, implying that the simple thalamic neuron model contains the essential ingredients that account for the computational properties of the thalamic burst code. Thus, our results suggest the n-spike burst code is a general property of thalamic neurons.

Spike-Triggered Covariance Analysis Reveals Phenomenological Diversity of Contrast Adaptation in the Retina

When visual contrast changes, retinal ganglion cells adapt by adjusting their sensitivity as well as their temporal filtering characteristics. The latter has classically been described by contrast-induced gain changes that depend on temporal frequency. Here, we explored a new perspective on contrast-induced changes in temporal filtering by using spike-triggered covariance analysis to extract multiple parallel temporal filters for individual ganglion cells. Based on multielectrode-array recordings from ganglion cells in the isolated salamander retina, we found that contrast adaptation of temporal filtering can largely be captured by contrast-invariant sets of filters with contrast-dependent weights. Moreover, differences among the ganglion cells in the filter sets and their contrast-dependent contributions allowed us to phenomenologically distinguish three types of filter changes. The first type is characterized by newly emerging features at higher contrast, which can be reproduced by computational models that contain response-triggered gain-control mechanisms. The second type follows from stronger adaptation in the Off pathway as compared to the On pathway in On-Off-type ganglion cells. Finally, we found that, in a subset of neurons, contrast-induced filter changes are governed by particularly strong spike-timing dynamics, in particular by pronounced stimulus-dependent latency shifts that can be observed in these cells. Together, our results show that the contrast dependence of temporal filtering in retinal ganglion cells has a multifaceted phenomenology and that a multi-filter analysis can provide a useful basis for capturing the underlying signal-processing dynamics.

Our sensory systems have to process stimuli under a wide range of environmental conditions. To cope with this challenge, the involved neurons adapt by adjusting their signal processing to the recently encountered intensity range. In the visual system, one finds, for example, that higher visual contrast leads to changes in how visual signals are temporally filtered, making signal processing faster and more band-pass-like at higher contrast. By analyzing signals from neurons in the retina of salamanders, we here found that these adaptation effects can be described by a fixed set of filters, independent of contrast, whose relative contributions change with contrast. Also, we found that different phenomena contribute to this adaptation. In particular, some cells change their relative sensitivity to light increments and light decrements, whereas other cells are influenced by a strong contrast-dependence of the exact timing of their responses. Our results show that contrast adaptation in the retina is not an entirely homogeneous phenomenon, and that models with multiple filters can help in characterizing sensory adaptation.

Effects of spike-triggered negative feedback on receptive-field properties

Sensory neurons are often described in terms of a receptive field, that is, a linear kernel through which stimuli are filtered before they are further processed. If information transmission is assumed to proceed in a feedforward cascade, the receptive field may be interpreted as the external stimulus' profile maximizing neuronal output. The nervous system, however, contains many feedback loops, and sensory neurons filter more currents than the ones representing the transduced external stimulus. Some of the additional currents are generated by the output activity of the neuron itself, and therefore constitute feedback signals. By means of a time-frequency analysis of the input/output transformation, here we show how feedback modifies the receptive field. The model is applicable to various types of feedback processes, from spike-triggered intrinsic conductances to inhibitory synaptic inputs from nearby neurons. We distinguish between the intrinsic receptive field (filtering all input currents) and the effective receptive field (filtering only external stimuli). Whereas the intrinsic receptive field summarizes the biophysical properties of the neuron associated to subthreshold integration and spike generation, only the effective receptive field can be interpreted as the external stimulus' profile maximizing neuronal output. We demonstrate that spike-triggered feedback shifts low-pass filtering towards band-pass processing, transforming integrator neurons into resonators. For strong feedback, a sharp resonance in the spectral neuronal selectivity may appear. Our results provide a unified framework to interpret a collection of previous experimental studies where specific feedback mechanisms were shown to modify the filtering properties of neurons.

Computational identification of receptive fields

Natural stimuli elicit robust responses of neurons throughout sensory pathways, and therefore their use provides unique opportunities for understanding sensory coding. This review describes statistical methods that can be used to characterize neural feature selectivity, focusing on the case of natural stimuli. First, we discuss how such classic methods as reverse correlation/spike-triggered average and spike-triggered covariance can be generalized for use with natural stimuli to find the multiple relevant stimulus features that affect the responses of a given neuron. Second, ways to characterize neural feature selectivity while assuming that the neural responses exhibit a certain type of invariance, such as position invariance for visual neurons, are discussed. Finally, we discuss methods that do not require one to make an assumption of invariance and instead can determine the type of invariance by analyzing relationships between the multiple stimulus features that affect the neural responses.

Spike triggered covariance in strongly correlated gaussian stimuli

Many biological systems perform computations on inputs that have very large dimensionality. Determining the relevant input combinations for a particular computation is often key to understanding its function. A common way to find the relevant input dimensions is to examine the difference in variance between the input distribution and the distribution of inputs associated with certain outputs. In systems neuroscience, the corresponding method is known as spike-triggered covariance (STC). This method has been highly successful in characterizing relevant input dimensions for neurons in a variety of sensory systems. So far, most studies used the STC method with weakly correlated Gaussian inputs. However, it is also important to use this method with inputs that have long range correlations typical of the natural sensory environment. In such cases, the stimulus covariance matrix has one (or more) outstanding eigenvalues that cannot be easily equalized because of sampling variability. Such outstanding modes interfere with analyses of statistical significance of candidate input dimensions that modulate neuronal outputs. In many cases, these modes obscure the significant dimensions. We show that the sensitivity of the STC method in the regime of strongly correlated inputs can be improved by an order of magnitude or more. This can be done by evaluating the significance of dimensions in the subspace orthogonal to the outstanding mode(s). Analyzing the responses of retinal ganglion cells probed with Gaussian noise, we find that taking into account outstanding modes is crucial for recovering relevant input dimensions for these neurons.

In many areas of computational biology, including the analyses of genetic mutations, protein stability and neural coding, as well as in economics, one of the most basic and important steps of data analysis is to find the relevant input dimensions for a particular task. In neural coding problems, the spike-triggered covariance (STC) method identifies relevant input dimensions by comparing the variance of the input distribution along different dimensions to the variance of inputs that elicited a neural response. While in theory the method can be applied to Gaussian stimuli with or without correlations, it has so far been used in studies with only weakly correlated stimuli. Here we show that to use STC with strongly correlated, -type inputs, one has to take into account that the covariance matrix of random samples from this distribution has a complex structure, with one or more outstanding modes. We use simulations on model neurons as well as an analysis of the responses of retinal neurons to demonstrate that taking the presence of these outstanding modes into account improves the sensitivity of the STC method by more than an order of magnitude.

Local and global contrast adaptation in retinal ganglion cells

Retinal ganglion cells react to changes in visual contrast by adjusting their sensitivity and temporal filtering characteristics. This contrast adaptation has primarily been studied under spatially homogeneous stimulation. Yet, ganglion cell receptive fields are often characterized by spatial subfields, providing a substrate for nonlinear spatial processing. This raises the question whether contrast adaptation follows a similar subfield structure or whether it occurs globally over the receptive field even for local stimulation. We therefore recorded ganglion cell activity in isolated salamander retinas while locally changing visual contrast. Ganglion cells showed primarily global adaptation characteristics, with notable exceptions in certain aspects of temporal filtering. Surprisingly, some changes in filtering were most pronounced for locations where contrast did not change. This seemingly paradoxical effect can be explained by a simple computational model, which emphasizes the importance of local nonlinearities in the retina and suggests a reevaluation of previously reported local contrast adaptation.

Identifying functional bases for multidimensional neural computations

Current dimensionality-reduction methods can identify relevant subspaces for neural computations but do not favor one basis over the other within the relevant subspace. Finding the appropriate basis can simplify the description of the nonlinear computation with respect to the relevant variables, making it easier to elucidate the underlying neural computation and make hypotheses about the neural circuitry, giving rise to the observed responses. Part of the problem is that although some of the dimensionality reduction methods can identify many of the relevant dimensions, it is usually difficult to map out or interpret the nonlinear transformation with respect to more than a few relevant dimensions simultaneously without some simplifying assumptions. While recent approaches make it possible to create predictive models based on many relevant dimensions simultaneously, there still remains the need to relate such predictive models to the mechanistic descriptions of the operation of underlying neural circuitry. Here we demonstrate that transforming to a basis within the relevant subspace where the neural computation is best described by a given nonlinear function often makes it easier to interpret the computation and describe it with a small number of parameters. We refer to the corresponding basis as the functional basis, and illustrate the utility of such transformation in the context of logical OR and logical AND functions. We show that although dimensionality-reduction methods such as spike-triggered covariance are able to find a relevant subspace, they often produce dimensions that are difficult to interpret and do not correspond to a functional basis. The functional features can be found using a maximum likelihood approach. The results are illustrated using simulated neurons and recordings from retinal ganglion cells. The resulting features are uniquely defined and nonorthogonal, and they make it easier to relate computational and mechanistic models to each other.

Linking dynamical and functional properties of intrinsically bursting neurons

Several studies have shown that bursting neurons can encode information in the number of spikes per burst: As the stimulus varies, so does the length of individual bursts. There presented stimuli, however, vary substantially among different sensory modalities and different neurons.The goal of this paper is to determine which kind of stimulus features can be encoded in burst length, and how those features depend on the mathematical properties of the underlying dynamical system.We show that the initiation and termination of each burst is triggered by specific stimulus features whose temporal characteristsics are determined by the types of bifurcations that initiate and terminate firing in each burst. As only a few bifurcations are possible, only a restricted number of encoded features exists. Here we focus specifically on describing parabolic, square-wave and elliptic bursters. We find that parabolic bursters, whose firing is initiated and terminated by saddle-node bifurcations, behave as prototypical integrators: Firing is triggered by depolarizing stimuli, and lasts for as long as excitation is prolonged. Elliptic bursters, contrastingly, constitute prototypical resonators, since both the initiating and terminating bifurcations possess well-defined oscillation time scales. Firing is therefore triggered by stimulus stretches of matching frequency and terminated by a phase-inversion in the oscillation. The behavior of square-wave bursters is somewhat intermediate, since they are triggered by a fold bifurcation of cycles of well-defined frequency but are terminated by a homoclinic bifurcation lacking an oscillating time scale. These correspondences show that stimulus selectivity is determined by the type of bifurcations. By testing several neuron models, we also demonstrate that additional biological properties that do not modify the bifurcation structure play a minor role in stimulus encoding. Moreover, we show that burst-length variability (and thereby, the capacity to transmit information) depends on a trade-off between the variance of the external signal driving the cell and the strength of the slow internal currents modulating bursts. Thus, our work explicitly links the computational properties of bursting neurons to the mathematical properties of the underlying dynamical systems.

Features and functions of nonlinear spatial integration by retinal ganglion cells

Ganglion cells in the vertebrate retina integrate visual information over their receptive fields. They do so by pooling presynaptic excitatory inputs from typically many bipolar cells, which themselves collect inputs from several photoreceptors. In addition, inhibitory interactions mediated by horizontal cells and amacrine cells modulate the structure of the receptive field. In many models, this spatial integration is assumed to occur in a linear fashion. Yet, it has long been known that spatial integration by retinal ganglion cells also incurs nonlinear phenomena. Moreover, several recent examples have shown that nonlinear spatial integration is tightly connected to specific visual functions performed by different types of retinal ganglion cells. This work discusses these advances in understanding the role of nonlinear spatial integration and reviews recent efforts to quantitatively study the nature and mechanisms underlying spatial nonlinearities. These new insights point towards a critical role of nonlinearities within ganglion cell receptive fields for capturing responses of the cells to natural and behaviorally relevant visual stimuli. In the long run, nonlinear phenomena of spatial integration may also prove important for implementing the actual neural code of retinal neurons when designing visual prostheses for the eye.

The iso-response method: measuring neuronal stimulus integration with closed-loop experiments

Throughout the nervous system, neurons integrate high-dimensional input streams and transform them into an output of their own. This integration of incoming signals involves filtering processes and complex non-linear operations. The shapes of these filters and non-linearities determine the computational features of single neurons and their functional roles within larger networks. A detailed characterization of signal integration is thus a central ingredient to understanding information processing in neural circuits. Conventional methods for measuring single-neuron response properties, such as reverse correlation, however, are often limited by the implicit assumption that stimulus integration occurs in a linear fashion. Here, we review a conceptual and experimental alternative that is based on exploring the space of those sensory stimuli that result in the same neural output. As demonstrated by recent results in the auditory and visual system, such iso-response stimuli can be used to identify the non-linearities relevant for stimulus integration, disentangle consecutive neural processing steps, and determine their characteristics with unprecedented precision. Automated closed-loop experiments are crucial for this advance, allowing rapid search strategies for identifying iso-response stimuli during experiments. Prime targets for the method are feed-forward neural signaling chains in sensory systems, but the method has also been successfully applied to feedback systems. Depending on the specific question, “iso-response” may refer to a predefined firing rate, single-spike probability, first-spike latency, or other output measures. Examples from different studies show that substantial progress in understanding neural dynamics and coding can be achieved once rapid online data analysis and stimulus generation, adaptive sampling, and computational modeling are tightly integrated into experiments.