Structures of Neural Correlation and How They Favor Coding

Information transfer and recovery for the sense of touch

Transformation of postsynaptic potentials into action potentials is the rate-limiting step of communication in neural networks. The efficiency of this intracellular information transfer also powerfully shapes stimulus representations in sensory cortices. Using whole-cell recordings and information-theoretic measures, we show herein that somatic postsynaptic potentials accurately represent stimulus location on a trial-by-trial basis in single neurons, even 4 synapses away from the sensory periphery in the whisker system. This information is largely lost during action potential generation but can be rapidly (<20 ms) recovered using complementary information in local populations in a cell-type-specific manner. These results show that as sensory information is transferred from one neural locus to another, the circuits reconstruct the stimulus with high fidelity so that sensory representations of single neurons faithfully represent the stimulus in the periphery, but only in their postsynaptic potentials, resulting in lossless information processing for the sense of touch in the primary somatosensory cortex.

Nonresponsive Neurons Improve Population Coding of Object Location

Understanding how heterogeneous neural populations represent sensory input to give rise to behavior remains a central problem in systems neuroscience. Here we investigated how midbrain neurons within the electrosensory system of Apteronotus leptorhynchus code for object location in space. In vivo simultaneous recordings were achieved via Neuropixels probes, high-density electrode arrays, with the stimulus positioned at different locations relative to the animal. Midbrain neurons exhibited heterogeneous response profiles, with a significant proportion (65%) seemingly nonresponsive to moving stimuli. Remarkably, we found that nonresponsive neurons increased population coding of object location through synergistic interactions with responsive neurons by effectively reducing noise. Mathematical modeling demonstrated that increased response heterogeneity together with the experimentally observed correlations was sufficient to give rise to independent encoding by responsive neurons. Furthermore, the addition of nonresponsive neurons in the model gave rise to synergistic population coding. Taken together, our findings reveal that nonresponsive neurons, which are frequently excluded from analysis, can significantly improve population coding of object location through synergistic interactions with responsive neurons. Combinations of responsive and nonresponsive neurons have been observed in sensory systems across taxa; it is likely that similar synergistic interactions improve population coding across modalities and behavioral tasks.

The geometry of correlated variability leads to highly suboptimal discriminative sensory coding

The brain represents the world through the activity of neural populations; however, whether the computational goal of sensory coding is to support discrimination of sensory stimuli or to generate an internal model of the sensory world is unclear. Correlated variability across a neural population (noise correlations) is commonly observed experimentally, and many studies demonstrate that correlated variability improves discriminative sensory coding compared to a null model with no correlations. However, such results do not address whether correlated variability is optimal for discriminative sensory coding. If the computational goal of sensory coding is discriminative, than correlated variability should be optimized to support that goal. We assessed optimality of noise correlations for discriminative sensory coding in diverse datasets by developing two novel null models, each with a biological interpretation. Across datasets, we found that correlated variability in neural populations leads to highly suboptimal discriminative sensory coding according to both null models. Furthermore, biological constraints prevent many subsets of the neural populations from achieving optimality, and subselecting based on biological criteria leaves red discriminative coding performance suboptimal. Finally, we show that optimal subpopulations are exponentially small as the population size grows. Together, these results demonstrate that the geometry of correlated variability leads to highly suboptimal discriminative sensory coding.NEW & NOTEWORTHY The brain represents the world through the activity of neural populations that exhibit correlated variability. We assessed optimality of correlated variability for discriminative sensory coding in diverse datasets by developing two novel null models. Across datasets, correlated variability in neural populations leads to highly suboptimal discriminative sensory coding according to both null models. Biological constraints prevent the neural populations from achieving optimality. Together, these results demonstrate that the geometry of correlated variability leads to highly suboptimal discriminative sensory coding.

Disentangling signal and noise in neural responses through generative modeling

Measurements of neural responses to identically repeated experimental events often exhibit large amounts of variability. This noise is distinct from signal, operationally defined as the average expected response across repeated trials for each given event. Accurately distinguishing signal from noise is important, as each is a target that is worthy of study (many believe noise reflects important aspects of brain function) and it is important not to confuse one for the other. Here, we describe a principled modeling approach in which response measurements are explicitly modeled as the sum of samples from multivariate signal and noise distributions. In our proposed method-termed Generative Modeling of Signal and Noise (GSN)-the signal distribution is estimated by subtracting the estimated noise distribution from the estimated data distribution. Importantly, GSN improves estimates of the signal distribution, but does not provide improved estimates of responses to individual events. We validate GSN using ground-truth simulations and show that it compares favorably with related methods. We also demonstrate the application of GSN to empirical fMRI data to illustrate a simple consequence of GSN: by disentangling signal and noise components in neural responses, GSN denoises principal components analysis and improves estimates of dimensionality. We end by discussing other situations that may benefit from GSN's characterization of signal and noise, such as estimation of noise ceilings for computational models of neural activity. A code toolbox for GSN is provided with both MATLAB and Python implementations.

Stimulus type shapes the topology of cellular functional networks in mouse visual cortex

On the timescale of sensory processing, neuronal networks have relatively fixed anatomical connectivity, while functional interactions between neurons can vary depending on the ongoing activity of the neurons within the network. We thus hypothesized that different types of stimuli could lead those networks to display stimulus-dependent functional connectivity patterns. To test this hypothesis, we analyzed single-cell resolution electrophysiological data from the Allen Institute, with simultaneous recordings of stimulus-evoked activity from neurons across 6 different regions of mouse visual cortex. Comparing the functional connectivity patterns during different stimulus types, we made several nontrivial observations: (1) while the frequencies of different functional motifs were preserved across stimuli, the identities of the neurons within those motifs changed; (2) the degree to which functional modules are contained within a single brain region increases with stimulus complexity. Altogether, our work reveals unexpected stimulus-dependence to the way groups of neurons interact to process incoming sensory information.

Tracking Dynamic Conditional Neural Correlation during Task Learning

Single neuron modulates the external stimuli, and neural population coordinates to encode information. An alternate method for examining the coordinated populational activity in neural encoding is conditional neural correlation (CNC). However, such correlations are not static during a new task learning process as neurons adapt their tunings over time for better performance. To investigate how neurons adjust their firing patterns during learning, it's essential to track the time-variant correlation. In this paper, we propose to mathematically model the dynamical CNC by implementing the integrated point process filter which incorporates neural correlation and single neural tuning into decoding. Specifically, we generate synthetic M1 neurons' firing data to simulate the dynamic change of the conditional neural correlation over days, while a rat learns a two-lever discrimination task. By comparing the tracked CNC with the designed CNC, our results show that the CNC can be better tracked over time by CIPPF than that of decoder assuming conditional independence among neurons, which indicates the possibility to better understand the brain dynamics during task learning.

Neural populations within macaque early vestibular pathways are adapted to encode natural self-motion

How the activities of large neural populations are integrated in the brain to ensure accurate perception and behavior remains a central problem in systems neuroscience. Here, we investigated population coding of naturalistic self-motion by neurons within early vestibular pathways in rhesus macaques (Macacca mulatta). While vestibular neurons displayed similar dynamic tuning to self-motion, inspection of their spike trains revealed significant heterogeneity. Further analysis revealed that, during natural but not artificial stimulation, heterogeneity resulted primarily from variability across neurons as opposed to trial-to-trial variability. Interestingly, vestibular neurons displayed different correlation structures during naturalistic and artificial self-motion. Specifically, while correlations due to the stimulus (i.e., signal correlations) did not differ, correlations between the trial-to-trial variabilities of neural responses (i.e., noise correlations) were instead significantly positive during naturalistic but not artificial stimulation. Using computational modeling, we show that positive noise correlations during naturalistic stimulation benefits information transmission by heterogeneous vestibular neural populations. Taken together, our results provide evidence that neurons within early vestibular pathways are adapted to the statistics of natural self-motion stimuli at the population level. We suggest that similar adaptations will be found in other systems and species.

Small-correlation expansion to quantify information in noisy sensory systems

Neural networks encode information through their collective spiking activity in response to external stimuli. This population response is noisy and strongly correlated, with a complex interplay between correlations induced by the stimulus, and correlations caused by shared noise. Understanding how these correlations affect information transmission has so far been limited to pairs or small groups of neurons, because the curse of dimensionality impedes the evaluation of mutual information in larger populations. Here, we develop a small-correlation expansion to compute the stimulus information carried by a large population of neurons, yielding interpretable analytical expressions in terms of the neurons' firing rates and pairwise correlations. We validate the approximation on synthetic data and demonstrate its applicability to electrophysiological recordings in the vertebrate retina, allowing us to quantify the effects of noise correlations between neurons and of memory in single neurons.

Are single-peaked tuning curves tuned for speed rather than accuracy?

According to the efficient coding hypothesis, sensory neurons are adapted to provide maximal information about the environment, given some biophysical constraints. In early visual areas, stimulus-induced modulations of neural activity (or tunings) are predominantly single-peaked. However, periodic tuning, as exhibited by grid cells, has been linked to a significant increase in decoding performance. Does this imply that the tuning curves in early visual areas are sub-optimal? We argue that the time scale at which neurons encode information is imperative to understand the advantages of single-peaked and periodic tuning curves, respectively. Here, we show that the possibility of catastrophic (large) errors creates a trade-off between decoding time and decoding ability. We investigate how decoding time and stimulus dimensionality affect the optimal shape of tuning curves for removing catastrophic errors. In particular, we focus on the spatial periods of the tuning curves for a class of circular tuning curves. We show an overall trend for minimal decoding time to increase with increasing Fisher information, implying a trade-off between accuracy and speed. This trade-off is reinforced whenever the stimulus dimensionality is high, or there is ongoing activity. Thus, given constraints on processing speed, we present normative arguments for the existence of the single-peaked tuning organization observed in early visual areas.

Coding of object location by heterogeneous neural populations with spatially dependent correlations in weakly electric fish

Understanding how neural populations encode sensory stimuli remains a central problem in neuroscience. Here we performed multi-unit recordings from sensory neural populations in the electrosensory system of the weakly electric fish Apteronotus leptorhynchus in response to stimuli located at different positions along the rostro-caudal axis. Our results reveal that the spatial dependence of correlated activity along receptive fields can help mitigate the deleterious effects that these correlations would otherwise have if they were spatially independent. Moreover, using mathematical modeling, we show that experimentally observed heterogeneities in the receptive fields of neurons help optimize information transmission as to object location. Taken together, our results have important implications for understanding how sensory neurons whose receptive fields display antagonistic center-surround organization encode location. Important similarities between the electrosensory system and other sensory systems suggest that our results will be applicable elsewhere.

Computational methods to study information processing in neural circuits

The brain is an information processing machine and thus naturally lends itself to be studied using computational tools based on the principles of information theory. For this reason, computational methods based on or inspired by information theory have been a cornerstone of practical and conceptual progress in neuroscience. In this Review, we address how concepts and computational tools related to information theory are spurring the development of principled theories of information processing in neural circuits and the development of influential mathematical methods for the analyses of neural population recordings. We review how these computational approaches reveal mechanisms of essential functions performed by neural circuits. These functions include efficiently encoding sensory information and facilitating the transmission of information to downstream brain areas to inform and guide behavior. Finally, we discuss how further progress and insights can be achieved, in particular by studying how competing requirements of neural encoding and readout may be optimally traded off to optimize neural information processing.

Visual and motor signatures of locomotion dynamically shape a population code for feature detection in Drosophila

Natural vision is dynamic: as an animal moves, its visual input changes dramatically. How can the visual system reliably extract local features from an input dominated by self-generated signals? In Drosophila, diverse local visual features are represented by a group of projection neurons with distinct tuning properties. Here, we describe a connectome-based volumetric imaging strategy to measure visually evoked neural activity across this population. We show that local visual features are jointly represented across the population, and a shared gain factor improves trial-to-trial coding fidelity. A subset of these neurons, tuned to small objects, is modulated by two independent signals associated with self-movement, a motor-related signal, and a visual motion signal associated with rotation of the animal. These two inputs adjust the sensitivity of these feature detectors across the locomotor cycle, selectively reducing their gain during saccades and restoring it during intersaccadic intervals. This work reveals a strategy for reliable feature detection during locomotion.

Theoretical foundations of studying criticality in the brain

Criticality is hypothesized as a physical mechanism underlying efficient transitions between cortical states and remarkable information-processing capacities in the brain. While considerable evidence generally supports this hypothesis, nonnegligible controversies persist regarding the ubiquity of criticality in neural dynamics and its role in information processing. Validity issues frequently arise during identifying potential brain criticality from empirical data. Moreover, the functional benefits implied by brain criticality are frequently misconceived or unduly generalized. These problems stem from the nontriviality and immaturity of the physical theories that analytically derive brain criticality and the statistic techniques that estimate brain criticality from empirical data. To help solve these problems, we present a systematic review and reformulate the foundations of studying brain criticality, that is, ordinary criticality (OC), quasi-criticality (qC), self-organized criticality (SOC), and self-organized quasi-criticality (SOqC), using the terminology of neuroscience. We offer accessible explanations of the physical theories and statistical techniques of brain criticality, providing step-by-step derivations to characterize neural dynamics as a physical system with avalanches. We summarize error-prone details and existing limitations in brain criticality analysis and suggest possible solutions. Moreover, we present a forward-looking perspective on how optimizing the foundations of studying brain criticality can deepen our understanding of various neuroscience questions.

Dynamic Hippocampal CA2 Responses to Contextual Spatial Novelty

Hippocampal place cells are functional units of spatial navigation and are present in all subregions: CA1, CA2, CA3, and CA4. Recent studies on CA2 have indicated its role in social and contextual memories, but its contribution to spatial novelty detection and encoding remains largely unknown. The current study aims to uncover how CA2 processes spatial novelty and to distinguish its functional role towards the same from CA1. Accordingly, a novel 3-day paradigm was designed where animals were introduced to a completely new environment on the first day, and on subsequent days, novel segments were inserted into the existing spatial environment while the other segments remained the same, allowing us to compare novel and familiar parts of the same closed-loop track on multiple days. We found that spatial novelty leads to dynamic and complex hippocampal place cell firings at both individual neuron and population levels. Place cells in both CA1 and CA2 had strong responses to novel segments, leading to higher average firing rates and increased pairwise cross correlations across all days. However, CA2 place cells that fired for novel areas had lower spatial information scores than CA1 place cells active in the same areas. At the ensemble level, CA1 only responded to spatial novelty on day 1, when the environment was completely novel, whereas CA2 responded to it on all days, each time novelty was introduced. Therefore, CA2 was more sensitive and responsive to novel spatial features even when introduced in a familiar environment, unlike CA1.

Targeted dimensionality reduction enables reliable estimation of neural population coding accuracy from trial-limited data

Rapidly developing technology for large scale neural recordings has allowed researchers to measure the activity of hundreds to thousands of neurons at single cell resolution in vivo. Neural decoding analyses are a widely used tool used for investigating what information is represented in this complex, high-dimensional neural population activity. Most population decoding methods assume that correlated activity between neurons has been estimated accurately. In practice, this requires large amounts of data, both across observations and across neurons. Unfortunately, most experiments are fundamentally constrained by practical variables that limit the number of times the neural population can be observed under a single stimulus and/or behavior condition. Therefore, new analytical tools are required to study neural population coding while taking into account these limitations. Here, we present a simple and interpretable method for dimensionality reduction that allows neural decoding metrics to be calculated reliably, even when experimental trial numbers are limited. We illustrate the method using simulations and compare its performance to standard approaches for dimensionality reduction and decoding by applying it to single-unit electrophysiological data collected from auditory cortex.

The structures and functions of correlations in neural population codes

The collective activity of a population of neurons, beyond the properties of individual cells, is crucial for many brain functions. A fundamental question is how activity correlations between neurons affect how neural populations process information. Over the past 30 years, major progress has been made on how the levels and structures of correlations shape the encoding of information in population codes. Correlations influence population coding through the organization of pairwise-activity correlations with respect to the similarity of tuning of individual neurons, by their stimulus modulation and by the presence of higher-order correlations. Recent work has shown that correlations also profoundly shape other important functions performed by neural populations, including generating codes across multiple timescales and facilitating information transmission to, and readout by, downstream brain areas to guide behaviour. Here, we review this recent work and discuss how the structures of correlations can have opposite effects on the different functions of neural populations, thus creating trade-offs and constraints for the structure-function relationships of population codes. Further, we present ideas on how to combine large-scale simultaneous recordings of neural populations, computational models, analyses of behaviour, optogenetics and anatomy to unravel how the structures of correlations might be optimized to serve multiple functions.

Internally generated population activity in cortical networks hinders information transmission

How neuronal variability affects sensory coding is a central question in systems neuroscience, often with complex and model-dependent answers. Many studies explore population models with a parametric structure for response tuning and variability, preventing an analysis of how synaptic circuitry establishes neural codes. We study stimulus coding in networks of spiking neuron models with spatially ordered excitatory and inhibitory connectivity. The wiring structure is capable of producing rich population-wide shared neuronal variability that agrees with many features of recorded cortical activity. While both the spatial scales of feedforward and recurrent projections strongly affect noise correlations, only recurrent projections, and in particular inhibitory projections, can introduce correlations that limit the stimulus information available to a decoder. Using a spatial neural field model, we relate the recurrent circuit conditions for information limiting noise correlations to how recurrent excitation and inhibition can form spatiotemporal patterns of population-wide activity.

Shallow neural networks trained to detect collisions recover features of visual loom-selective neurons

Animals have evolved sophisticated visual circuits to solve a vital inference problem: detecting whether or not a visual signal corresponds to an object on a collision course. Such events are detected by specific circuits sensitive to visual looming, or objects increasing in size. Various computational models have been developed for these circuits, but how the collision-detection inference problem itself shapes the computational structures of these circuits remains unknown. Here, inspired by the distinctive structures of LPLC2 neurons in the visual system of Drosophila, we build anatomically-constrained shallow neural network models and train them to identify visual signals that correspond to impending collisions. Surprisingly, the optimization arrives at two distinct, opposing solutions, only one of which matches the actual dendritic weighting of LPLC2 neurons. Both solutions can solve the inference problem with high accuracy when the population size is large enough. The LPLC2-like solutions reproduces experimentally observed LPLC2 neuron responses for many stimuli, and reproduces canonical tuning of loom sensitive neurons, even though the models are never trained on neural data. Thus, LPLC2 neuron properties and tuning are predicted by optimizing an anatomically-constrained neural network to detect impending collisions. More generally, these results illustrate how optimizing inference tasks that are important for an animal's perceptual goals can reveal and explain computational properties of specific sensory neurons.

Coordinated multivoxel coding beyond univariate effects is not likely to be observable in fMRI data

Simultaneous recording of activity across brain regions can contain additional information compared to regional recordings done in isolation. In particular, multivariate pattern analysis (MVPA) across voxels has been interpreted as evidence for distributed coding of cognitive or sensorimotor processes beyond what can be gleaned from a collection of univariate effects (UVE) using functional magnetic resonance imaging (fMRI). Here, we argue that regardless of patterns revealed, conventional MVPA is merely a decoding tool with increased sensitivity arising from considering a large number of 'weak classifiers' (i.e., single voxels) in higher dimensions. We propose instead that 'real' multivoxel coding should result in changes in higher-order statistics across voxels between conditions such as second-order multivariate effects (sMVE). Surprisingly, analysis of conditions with robust multivariate effects (MVE) revealed by MVPA failed to show significant sMVE in two species (humans and macaques). Further analysis showed that while both MVE and sMVE can be readily observed in the spiking activity of neuronal populations, the slow and nonlinear hemodynamic coupling and low spatial resolution of fMRI activations make the observation of higher-order statistics between voxels highly unlikely. These results reveal inherent limitations of fMRI signals for studying coordinated coding across voxels. Together, these findings suggest that care should be taken in interpreting significant MVPA results as representing anything beyond a collection of univariate effects.

Stochastic Spiking Behavior in Neuromorphic Networks Enables True Random Number Generation

There is currently a great deal of interest in the use of nanoscale devices to emulate the behaviors of neurons and synapses and to facilitate brain-inspired computation. Here, it is shown that percolating networks of nanoparticles exhibit stochastic spiking behavior that is strikingly similar to that observed in biological neurons. The spiking rate can be controlled by the input stimulus, similar to "rate coding" in biology, and the distributions of times between events are log-normal, providing insights into the atomic-scale spiking mechanism. The stochasticity of the spiking behavior is then used for true random number generation, and the high quality of the generated random bit-streams is demonstrated, opening up promising routes toward integration of neuromorphic computing with secure information processing.

Building population models for large-scale neural recordings: Opportunities and pitfalls

Modern recording technologies now enable simultaneous recording from large numbers of neurons. This has driven the development of new statistical models for analyzing and interpreting neural population activity. Here, we provide a broad overview of recent developments in this area. We compare and contrast different approaches, highlight strengths and limitations, and discuss biological and mechanistic insights that these methods provide.

Synergistic population coding of natural communication stimuli by hindbrain electrosensory neurons

Understanding how neural populations encode natural stimuli with complex spatiotemporal structure to give rise to perception remains a central problem in neuroscience. Here we investigated population coding of natural communication stimuli by hindbrain neurons within the electrosensory system of weakly electric fish Apteronotus leptorhynchus. Overall, we found that simultaneously recorded neural activities were correlated: signal but not noise correlations were variable depending on the stimulus waveform as well as the distance between neurons. Combining the neural activities using an equal-weight sum gave rise to discrimination performance between different stimulus waveforms that was limited by redundancy introduced by noise correlations. However, using an evolutionary algorithm to assign different weights to individual neurons before combining their activities (i.e., a weighted sum) gave rise to increased discrimination performance by revealing synergistic interactions between neural activities. Our results thus demonstrate that correlations between the neural activities of hindbrain electrosensory neurons can enhance information about the structure of natural communication stimuli that allow for reliable discrimination between different waveforms by downstream brain areas.

Population Coding of Natural Electrosensory Stimuli by Midbrain Neurons

Natural stimuli display spatiotemporal characteristics that typically vary over orders of magnitude, and their encoding by sensory neurons remains poorly understood. We investigated population coding of highly heterogeneous natural electrocommunication stimuli in Apteronotus leptorhynchus of either sex. Neuronal activities were positively correlated with one another in the absence of stimulation, and correlation magnitude decayed with increasing distance between recording sites. Under stimulation, we found that correlations between trial-averaged neuronal responses (i.e., signal correlations) were positive and higher in magnitude for neurons located close to another, but that correlations between the trial-to-trial variability (i.e., noise correlations) were independent of physical distance. Overall, signal and noise correlations were independent of stimulus waveform as well as of one another. To investigate how neuronal populations encoded natural electrocommunication stimuli, we considered a nonlinear decoder for which the activities were combined. Decoding performance was best for a timescale of 6 ms, indicating that midbrain neurons transmit information via precise spike timing. A simple summation of neuronal activities (equally weighted sum) revealed that noise correlations limited decoding performance by introducing redundancy. Using an evolution algorithm to optimize performance when considering instead unequally weighted sums of neuronal activities revealed much greater performance values, indicating that midbrain neuron populations transmit information that reliably enable discrimination between different stimulus waveforms. Interestingly, we found that different weight combinations gave rise to similar discriminability, suggesting robustness. Our results have important implications for understanding how natural stimuli are integrated by downstream brain areas to give rise to behavioral responses.SIGNIFICANCE STATEMENT We show that midbrain electrosensory neurons display correlations between their activities and that these can significantly impact performance of decoders. While noise correlations limited discrimination performance by introducing redundancy, considering unequally weighted sums of neuronal activities gave rise to much improved performance and mitigated the deleterious effects of noise correlations. Further analysis revealed that increased discriminability was achieved by making trial-averaged responses more separable, as well as by reducing trial-to-trial variability by eliminating noise correlations. We further found that multiple combinations of weights could give rise to similar discrimination performances, which suggests that such combinatorial codes could be achieved in the brain. We conclude that the activities of midbrain neuronal populations can be used to reliably discriminate between highly heterogeneous stimulus waveforms.

The Geometry of Information Coding in Correlated Neural Populations

Neurons in the brain represent information in their collective activity. The fidelity of this neural population code depends on whether and how variability in the response of one neuron is shared with other neurons. Two decades of studies have investigated the influence of these noise correlations on the properties of neural coding. We provide an overview of the theoretical developments on the topic. Using simple, qualitative, and general arguments, we discuss, categorize, and relate the various published results. We emphasize the relevance of the fine structure of noise correlation, and we present a new approach to the issue. Throughout this review, we emphasize a geometrical picture of how noise correlations impact the neural code.

Efficient and robust coding in heterogeneous recurrent networks

Cortical networks show a large heterogeneity of neuronal properties. However, traditional coding models have focused on homogeneous populations of excitatory and inhibitory neurons. Here, we analytically derive a class of recurrent networks of spiking neurons that close to optimally track a continuously varying input online, based on two assumptions: 1) every spike is decoded linearly and 2) the network aims to reduce the mean-squared error between the input and the estimate. From this we derive a class of predictive coding networks, that unifies encoding and decoding and in which we can investigate the difference between homogeneous networks and heterogeneous networks, in which each neurons represents different features and has different spike-generating properties. We find that in this framework, 'type 1' and 'type 2' neurons arise naturally and networks consisting of a heterogeneous population of different neuron types are both more efficient and more robust against correlated noise. We make two experimental predictions: 1) we predict that integrators show strong correlations with other integrators and resonators are correlated with resonators, whereas the correlations are much weaker between neurons with different coding properties and 2) that 'type 2' neurons are more coherent with the overall network activity than 'type 1' neurons.

Weakly correlated synapses promote dimension reduction in deep neural networks

By controlling synaptic and neural correlations, deep learning has achieved empirical successes in improving classification performances. How synaptic correlations affect neural correlations to produce disentangled hidden representations remains elusive. Here we propose a simplified model of dimension reduction, taking into account pairwise correlations among synapses, to reveal the mechanism underlying how the synaptic correlations affect dimension reduction. Our theory determines the scaling of synaptic correlations requiring only mathematical self-consistency for both binary and continuous synapses. The theory also predicts that weakly correlated synapses encourage dimension reduction compared to their orthogonal counterparts. In addition, these synapses attenuate the decorrelation process along the network depth. These two computational roles are explained by a proposed mean-field equation. The theoretical predictions are in excellent agreement with numerical simulations, and the key features are also captured by deep learning with Hebbian rules.

Electric Stimulation Elicits Heterogeneous Responses in ON but Not OFF Retinal Ganglion Cells to Transmit Rich Neural Information

Retinal implants electrically stimulate surviving retinal neurons to restore vision in people blinded by outer retinal degeneration. Although the healthy retina is known to transmit a vast amount of visual information to the brain, it has not been studied whether prosthetic vision contains a similar amount of information. Here, we assessed the neural information transmitted by population responses arising in brisk transient (BT) and brisk sustained (BS) subtypes of ON and OFF retinal ganglion cells (RGCs) in the rabbit retina. To correlate the response heterogeneity and the information transmission, we first quantified the cell-to-cell heterogeneity by calculating the spike time tiling coefficient (STTC) across spiking patterns of RGCs in each type. Then, we computed the neural information encoded by the RGC population in a given type. In responses to light stimulation, spiking activities were more heterogeneous in OFF than ON RGCs (STTC_AVG = 0.36, 0.45, 0.77 and 0.55 for OFF BT, OFF BS, ON BT, and ON BS, respectively). Interestingly, however, in responses to electric stimulation, both BT and BS subtypes of OFF RGCs showed remarkably homogeneous spiking patterns across cells (STTC_AVG = 0.93 and 0.82 for BT and BS, respectively), whereas the two subtypes of ON RGCs showed slightly increased populational heterogeneity compared to light-evoked responses (STTC_AVG = 0.71 and 0.63 for BT and BS, respectively). Consequently, the neural information encoded by the electrically-evoked responses of a population of 15 RGCs was substantially lower in the OFF than the ON pathway: OFF BT and BS cells transmit only ~23% and ~53% of the neural information transmitted by their ON counterparts. Together with previously-reported natural spiking activities in ON RGCs, the higher neural information may make ON responses more recognizable, eliciting the biased percepts of bright phosphenes.

Predicting synchronous firing of large neural populations from sequential recordings

A major goal in neuroscience is to understand how populations of neurons code for stimuli or actions. While the number of neurons that can be recorded simultaneously is increasing at a fast pace, in most cases these recordings cannot access a complete population: some neurons that carry relevant information remain unrecorded. In particular, it is hard to simultaneously record all the neurons of the same type in a given area. Recent progress have made possible to profile each recorded neuron in a given area thanks to genetic and physiological tools, and to pool together recordings from neurons of the same type across different experimental sessions. However, it is unclear how to infer the activity of a full population of neurons of the same type from these sequential recordings. Neural networks exhibit collective behaviour, e.g. noise correlations and synchronous activity, that are not directly captured by a conditionally-independent model that would just put together the spike trains from sequential recordings. Here we show that we can infer the activity of a full population of retina ganglion cells from sequential recordings, using a novel method based on copula distributions and maximum entropy modeling. From just the spiking response of each ganglion cell to a repeated stimulus, and a few pairwise recordings, we could predict the noise correlations using copulas, and then the full activity of a large population of ganglion cells of the same type using maximum entropy modeling. Remarkably, we could generalize to predict the population responses to different stimuli with similar light conditions and even to different experiments. We could therefore use our method to construct a very large population merging cells' responses from different experiments. We predicted that synchronous activity in ganglion cell populations saturates only for patches larger than 1.5mm in radius, beyond what is today experimentally accessible.

Global Motion Processing by Populations of Direction-Selective Retinal Ganglion Cells

Simple stimuli have been critical to understanding neural population codes in sensory systems. Yet it remains necessary to determine the extent to which this understanding generalizes to more complex conditions. To examine this problem, we measured how populations of direction-selective ganglion cells (DSGCs) from the retinas of male and female mice respond to a global motion stimulus with its direction and speed changing dynamically. We then examined the encoding and decoding of motion direction in both individual and populations of DSGCs. Individual cells integrated global motion over ∼200 ms, and responses were tuned to direction. However, responses were sparse and broadly tuned, which severely limited decoding performance from small DSGC populations. In contrast, larger populations compensated for response sparsity, enabling decoding with high temporal precision (<100 ms). At these timescales, correlated spiking was minimal and had little impact on decoding performance, unlike results obtained using simpler local motion stimuli decoded over longer timescales. We use these data to define different DSGC population decoding regimes that use or mitigate correlated spiking to achieve high-spatial versus high-temporal resolution.SIGNIFICANCE STATEMENT ON-OFF direction-selective ganglion cells (ooDSGCs) in the mammalian retina are typically thought to signal local motion to the brain. However, several recent studies suggest they may signal global motion. Here we analyze the fidelity of encoding and decoding global motion in a natural scene across large populations of ooDSGCs. We show that large populations of DSGCs are capable of signaling rapid changes in global motion.

Early Visual Motion Experience Improves Retinal Encoding of Motion Directions

Altered sensory experience in early life often leads to altered response properties of the sensory neurons. This process is mostly thought to happen in the brain, not in the sensory organs. We show that in the mouse retina of both sexes, exposed to a motion-dominated visual environment from eye-opening, the ON-OFF direction selective ganglion cells (ooDSGCs) develop significantly stronger direction encoding ability for motion in all directions. This improvement occurs independent of the motion direction used for training. We demonstrated that this enhanced ability to encode motion direction is mainly attributed to increased response reliability of ooDSGCs. Closer examination revealed that the excitatory inputs from the ON bipolar pathway showed enhanced response reliability after the motion experience training, while other synaptic inputs remain relatively unchanged. Our results demonstrate that retina adapts to the visual environment during neonatal development.SIGNIFICANCE STATEMENT We found that retina, as the first stage of visual sensation, can also be affected by experience dependent plasticity during development. Exposure to a motion enriched visual environment immediately after eye-opening greatly improves motion direction encoding by direction selective retinal ganglion cells (RGCs). These results motivate future studies aimed at understanding how visual experience shapes the retinal circuits and the response properties of retinal neurons.

Neuronal On- and Off-type heterogeneities improve population coding of envelope signals in the presence of stimulus-induced noise

Understanding the mechanisms by which neuronal population activity gives rise to perception and behavior remains a central question in systems neuroscience. Such understanding is complicated by the fact that natural stimuli often have complex structure. Here we investigated how heterogeneities within a sensory neuron population influence the coding of a noisy stimulus waveform (i.e., the noise) and its behaviorally relevant envelope signal (i.e., the signal). We found that On- and Off-type neurons displayed more heterogeneities in their responses to the noise than in their responses to the signal. These differences in heterogeneities had important consequences when quantifying response similarity between pairs of neurons. Indeed, the larger response heterogeneity displayed by On- and Off-type neurons made their pairwise responses to the noise on average more independent than when instead considering pairs of On-type or Off-type neurons. Such relative independence allowed for better averaging out of the noise response when pooling neural activities in a mixed-type (i.e., On- and Off-type) than for same-type (i.e., only On-type or only Off-type), thereby leading to greater information transmission about the signal. Our results thus reveal a function for the combined activities of On- and Off-type neurons towards improving information transmission of envelope stimuli at the population level. Our results will likely generalize because natural stimuli across modalities are characterized by a stimulus waveform whose envelope varies independently as well as because On- and Off-type neurons are observed across systems and species.

Heterogeneous Synaptic Weighting Improves Neural Coding in the Presence of Common Noise

Simultaneous recordings from the cortex have revealed that neural activity is highly variable and that some variability is shared across neurons in a population. Further experimental work has demonstrated that the shared component of a neuronal population's variability is typically comparable to or larger than its private component. Meanwhile, an abundance of theoretical work has assessed the impact that shared variability has on a population code. For example, shared input noise is understood to have a detrimental impact on a neural population's coding fidelity. However, other contributions to variability, such as common noise, can also play a role in shaping correlated variability. We present a network of linear-nonlinear neurons in which we introduce a common noise input to model-for instance, variability resulting from upstream action potentials that are irrelevant to the task at hand. We show that by applying a heterogeneous set of synaptic weights to the neural inputs carrying the common noise, the network can improve its coding ability as measured by both Fisher information and Shannon mutual information, even in cases where this results in amplification of the common noise. With a broad and heterogeneous distribution of synaptic weights, a population of neurons can remove the harmful effects imposed by afferents that are uninformative about a stimulus. We demonstrate that some nonlinear networks benefit from weight diversification up to a certain population size, above which the drawbacks from amplified noise dominate over the benefits of diversification. We further characterize these benefits in terms of the relative strength of shared and private variability sources. Finally, we studied the asymptotic behavior of the mutual information and Fisher information analytically in our various networks as a function of population size. We find some surprising qualitative changes in the asymptotic behavior as we make seemingly minor changes in the synaptic weight distributions.

Long-range temporal correlations in scale-free neuromorphic networks

Biological neuronal networks are the computing engines of the mammalian brain. These networks exhibit structural characteristics such as hierarchical architectures, small-world attributes, and scale-free topologies, providing the basis for the emergence of rich temporal characteristics such as scale-free dynamics and long-range temporal correlations. Devices that have both the topological and the temporal features of a neuronal network would be a significant step toward constructing a neuromorphic system that can emulate the computational ability and energy efficiency of the human brain. Here we use numerical simulations to show that percolating networks of nanoparticles exhibit structural properties that are reminiscent of biological neuronal networks, and then show experimentally that stimulation of percolating networks by an external voltage stimulus produces temporal dynamics that are self-similar, follow power-law scaling, and exhibit long-range temporal correlations. These results are expected to have important implications for the development of neuromorphic devices, especially for those based on the concept of reservoir computing.

Biological neuronal networks exhibit well-defined properties such as hierarchical structures and scale-free topologies, as well as a high degree of local clustering and short path lengths between nodes. These structural properties are intimately connected to the observed long-range temporal correlations in the network dynamics. Fabrication of artificial networks with similar structural properties would facilitate brain-like (“neuromorphic”) computing. Here we show experimentally that percolating networks of nanoparticles exhibit similar long-range temporal correlations to those of biological neuronal networks and use simulations to demonstrate that the dynamics arise from an underlying scale-free network architecture. We discuss similarities between the biological and percolating systems and highlight the potential for the percolating networks to be used in neuromorphic computing applications.

Clustering of Neural Activity: A Design Principle for Population Codes

We propose that correlations among neurons are generically strong enough to organize neural activity patterns into a discrete set of clusters, which can each be viewed as a population codeword. Our reasoning starts with the analysis of retinal ganglion cell data using maximum entropy models, showing that the population is robustly in a frustrated, marginally sub-critical, or glassy, state. This leads to an argument that neural populations in many other brain areas might share this structure. Next, we use latent variable models to show that this glassy state possesses well-defined clusters of neural activity. Clusters have three appealing properties: (i) clusters exhibit error correction, i.e., they are reproducibly elicited by the same stimulus despite variability at the level of constituent neurons; (ii) clusters encode qualitatively different visual features than their constituent neurons; and (iii) clusters can be learned by downstream neural circuits in an unsupervised fashion. We hypothesize that these properties give rise to a "learnable" neural code which the cortical hierarchy uses to extract increasingly complex features without supervision or reinforcement.

Neural Synchrony Gives Rise to Amplitude- and Duration-Invariant Encoding Consistent With Perception of Natural Communication Stimuli

When confronted with a highly variable environment, it remains poorly understood how neural populations encode and classify natural stimuli to give rise to appropriate and consistent behavioral responses. Here we investigated population coding of natural communication signals with different attributes (i.e., amplitude and duration) in the electrosensory system of the weakly electric fish Apteronotus leptorhynchus. Our results show that, while single peripheral neurons encode the detailed timecourse of different stimulus waveforms, measures of population synchrony are effectively unchanged because of coordinated increases and decreases in activity. A phenomenological mathematical model reproduced this invariance and shows that this can be explained by considering homogeneous populations whose responses are solely determined by single neuron firing properties. Moreover, recordings from downstream central neurons reveal that synchronous afferent activity is actually decoded and thus most likely transmitted to higher brain areas. Finally, we demonstrate that the associated behavioral responses at the organism level are invariant. Our results provide a mechanism by which amplitude- and duration-invariant coding of behaviorally relevant sensory input emerges across successive brain areas thereby presumably giving rise to invariant behavioral responses. Such mechanisms are likely to be found in other systems that share anatomical and functional features with the electrosensory system (e.g., auditory, visual, vestibular).

Information-Limiting Correlations in Large Neural Populations

Understanding the neural code requires understanding how populations of neurons code information. Theoretical models predict that information may be limited by correlated noise in large neural populations. Nevertheless, analyses based on tens of neurons have failed to find evidence of saturation. Moreover, some studies have shown that noise correlations can be very small, and therefore may not affect information coding. To determine whether information-limiting correlations exist, we implanted eight Utah arrays in prefrontal cortex (PFC; area 46) of two male macaque monkeys, recording >500 neurons simultaneously. We estimated information in PFC about saccades as a function of ensemble size. Noise correlations were, on average, small (∼10^-3). However, information scaled strongly sublinearly with ensemble size. After shuffling trials, destroying noise correlations, information was a linear function of ensemble size. Thus, we provide evidence for the existence of information-limiting noise correlations in large populations of PFC neurons.SIGNIFICANCE STATEMENT Recent theoretical work has shown that even small correlations can limit information if they are "differential correlations," which are difficult to measure directly. However, they can be detected through decoding analyses on recordings from a large number of neurons over a large number of trials. We have achieved both by collecting neural activity in dorsal-lateral prefrontal cortex of macaques using eight microelectrode arrays (768 electrodes), from which we were able to compute accurate information estimates. We show, for the first time, strong evidence for information-limiting correlations. Despite pairwise correlations being small (on the order of 10^-3), they affect information coding in populations on the order of 100 s of neurons.

Locating the engram: Should we look for plastic synapses or information-storing molecules?

Karl Lashley began the search for the engram nearly seventy years ago. In the time since, much has been learned but divisions remain. In the contemporary neurobiology of learning and memory, two profoundly different conceptions contend: the associative/connectionist (A/C) conception and the computational/representational (C/R) conception. Both theories ground themselves in the belief that the mind is emergent from the properties and processes of a material brain. Where these theories differ is in their description of what the neurobiological substrate of memory is and where it resides in the brain. The A/C theory of memory emphasizes the need to distinguish memory cognition from the memory engram and postulates that memory cognition is an emergent property of patterned neural activity routed through engram circuits. In this model, learning re-organizes synapse association strengths to guide future neural activity. Importantly, the version of the A/C theory advocated for here contends that synaptic change is not symbolic and, despite normally being necessary, is not sufficient for memory cognition. Instead, synaptic change provides the capacity and a blueprint for reinstating symbolic patterns of neural activity. Unlike the A/C theory, which posits that memory emerges at the circuit level, the C/R conception suggests that memory manifests at the level of intracellular molecular structures. In C/R theory, these intracellular structures are information-conveying and have properties compatible with the view that brain computation utilizes a read/write memory, functionally similar to that in a computer. New research has energized both sides and highlighted the need for new discussion. Both theories, the key questions each theory has yet to resolve and several potential paths forward are presented here.

A simple linear readout of MT supports motion direction-discrimination performance

Motion discrimination is a well-established model system for investigating how sensory signals are used to form perceptual decisions. Classic studies relating single-neuron activity in the middle temporal area (MT) to perceptual decisions have suggested that a simple linear readout could underlie motion discrimination behavior. A theoretically optimal readout, in contrast, would take into account the correlations between neurons and the sensitivity of individual neurons at each time point. However, it remains unknown how sophisticated the readout needs to be to support actual motion-discrimination behavior or to approach optimal performance. In this study, we evaluated the performance of various neurally plausible decoders, trained to discriminate motion direction from small ensembles of simultaneously recorded MT neurons. We found that decoding the stimulus without knowledge of the interneuronal correlations was sufficient to match an optimal (correlation aware) decoder. Additionally, a decoder could match the psychophysical performance of the animals with flat integration of up to half the stimulus and inherited temporal dynamics from the time-varying MT responses. These results demonstrate that simple, linear decoders operating on small ensembles of neurons can match both psychophysical performance and optimal sensitivity without taking correlations into account and that such simple read-out mechanisms can exhibit complex temporal properties inherited from the sensory dynamics themselves.NEW & NOTEWORTHY Motion perception depends on the ability to decode the activity of neurons in the middle temporal area. Theoretically optimal decoding requires knowledge of the sensitivity of neurons and interneuronal correlations. We report that a simple correlation-blind decoder performs as well as the optimal decoder for coarse motion discrimination. Additionally, the decoder could match the psychophysical performance with moderate temporal integration and dynamics inherited from sensory responses.

Modeling stimulus-dependent variability improves decoding of population neural responses

OBJECTIVE

Neural responses to repeated presentations of an identical stimulus often show substantial trial-to-trial variability. How the mean firing rate varies in response to different stimuli or during different movements (tuning curves) has been extensively modeled in a wide variety of neural systems. However, the variability of neural responses can also have clear tuning independent of the tuning in the mean firing rate. This suggests that the variability could contain information regarding the stimulus/movement beyond what is encoded in the mean firing rate. Here we demonstrate how taking variability into account can improve neural decoding.

APPROACH

In a typical neural coding model spike counts are assumed to be Poisson with the mean response depending on an external variable, such as a stimulus or movement. Bayesian decoding methods then use the probabilities under these Poisson tuning models (the likelihood) to estimate the probability of each stimulus given the spikes on a given trial (the posterior). However, under the Poisson model, spike count variability is always exactly equal to the mean (Fano factor  =  1). Here we use two alternative models-the Conway-Maxwell-Poisson (CMP) model and negative binomial (NB) model-to more flexibly characterize how neural variability depends on external stimuli. These models both contain the Poisson distribution as a special case but have an additional parameter that allows the variance to be greater than the mean (Fano factor  >  1) or, for the CMP model, less than the mean (Fano factor  <  1).

MAIN RESULTS

We find that neural responses in primary motor (M1), visual (V1), and auditory (A1) cortices have diverse tuning in both their mean firing rates and response variability. Across cortical areas, we find that Bayesian decoders using the CMP or NB models improve stimulus/movement estimation accuracy by 4%-12% compared to the Poisson model.

SIGNIFICANCE

Moreover, the uncertainty of the non-Poisson decoders more accurately reflects the magnitude of estimation errors. In addition to tuning curves that reflect average neural responses, stimulus-dependent response variability may be an important aspect of the neural code. Modeling this structure could, potentially, lead to improvements in brain machine interfaces.

Selective Changes in Noise Correlations Contribute to an Enhanced Representation of Saccadic Targets in Prefrontal Neuronal Ensembles

An ensemble of neurons can provide a dynamic representation of external stimuli, ongoing processes, or upcoming actions. This dynamic representation could be achieved by changes in the activity of individual neurons and/or their interactions. To investigate these possibilities, we simultaneously recorded from ensembles of prefrontal neurons in non-human primates during a memory-guided saccade task. Using both decoding and encoding methods, we examined changes in the information content of individual neurons and that of ensembles between visual encoding and saccadic target selection. We found that individual neurons maintained their limited spatial sensitivity between these cognitive states, whereas the ensemble selectively improved its encoding of spatial locations far from the neurons’ preferred locations. This population-level “encoding expansion” was not due to the ceiling effect at the preferred locations and was accompanied by selective changes in noise correlations for non-preferred locations. Moreover, the encoding expansion was observed for ensembles of different types of neurons and could not be explained by shifts in the preferred location of individual neurons. Our results demonstrate that the representation of space by neuronal ensembles is dynamically enhanced prior to saccades, and this enhancement occurs alongside changes in noise correlations more than changes in the activity of individual neurons.

Dendritic Spikes Expand the Range of Well Tolerated Population Noise Structures

The brain operates surprisingly well despite the noisy nature of individual neurons. The central mechanism for noise mitigation in the nervous system is thought to involve averaging over multiple noise-corrupted inputs. Subsequently, there has been considerable interest in identifying noise structures that can be integrated linearly in a way that preserves reliable signal encoding. By analyzing realistic synaptic integration in biophysically accurate neuronal models, I report a complementary denoising approach that is mediated by focal dendritic spikes. Dendritic spikes might seem to be unlikely candidates for noise reduction due to their miniscule integration compartments and poor averaging abilities. Nonetheless, the extra thresholding step introduced by dendritic spike generation increases neuronal tolerance for a broad category of noise structures, some of which cannot be resolved well with averaging. This property of active dendrites compensates for compartment size constraints and expands the repertoire of conditions that can be processed by neuronal populations.SIGNIFICANCE STATEMENT Noise, or random variability, is a prominent feature of the neuronal code and poses a fundamental challenge for information processing. To reconcile the surprisingly accurate output of the brain with the inherent noisiness of biological systems, previous work examined signal integration in idealized neurons. The notion that emerged from this body of work is that accurate signal representation relies largely on input averaging in neuronal dendrites. In contrast to the prevailing view, I show that denoising in simulated neurons with realistic morphology and biophysical properties follows a different strategy: dendritic spikes act as classifiers that assist in extracting information from a variety of noise structures that have been considered before to be particularly disruptive for reliable brain function.

Large-Scale, High-Resolution Microelectrode Arrays for Interrogation of Neurons and Networks

High-density microelectrode arrays (HD-MEAs) are increasingly being used for the observation and manipulation of neurons and networks in vitro. Large-scale electrode arrays allow for long-term extracellular recording of the electrical activity from thousands of neurons simultaneously. Beyond population activity, it has also become possible to extract information of single neurons at subcellular level (e.g., the propagation of action potentials along axons). In effect, HD-MEAs have become an electrical imaging platform for label-free extraction of the structure and activation of cells in cultures and tissues. The quality of HD-MEA data depends on the resolution of the electrode array and the signal-to-noise ratio. In this chapter, we begin with an introduction to HD-MEA signals. We provide an overview of the developments on complementary metal-oxide-semiconductor or CMOS-based HD-MEA technology. We also discuss the factors affecting the performance of HD-MEAs and the trending application requirements that drive the efforts for future devices. We conclude with an outlook on the potential of HD-MEAs for advancing basic neuroscience and drug discovery.

Synergistic Coding of Visual Information in Columnar Networks

Incoming stimuli are encoded collectively by populations of cortical neurons, which transmit information by using a neural code thought to be predominantly redundant. Redundant coding is widely believed to reflect a design choice whereby neurons with overlapping receptive fields sample environmental stimuli to convey similar information. Here, we performed multi-electrode laminar recordings in awake monkey V1 to report significant synergistic interactions between nearby neurons within a cortical column. These interactions are clustered non-randomly across cortical layers to form synergy and redundancy hubs. Homogeneous sub-populations comprising synergy hubs decode stimulus information significantly better compared to redundancy hubs or heterogeneous sub-populations. Mechanistically, synergistic interactions emerge from the stimulus dependence of correlated activity between neurons. Our findings suggest a refinement of the prevailing ideas regarding coding schemes in sensory cortex: columnar populations can efficiently encode information due to synergistic interactions even when receptive fields overlap and shared noise between cells is high.

Weighting neurons by selectivity produces near-optimal population codes

Perception is produced by "reading out" the representation of a sensory stimulus contained in the activity of a population of neurons. To examine experimentally how populations code information, a common approach is to decode a linearly weighted sum of the neurons' spike counts. This approach is popular because of the biological plausibility of weighted, nonlinear integration. For neurons recorded in vivo, weights are highly variable when derived through optimization methods, but it is unclear how the variability affects decoding performance in practice. To address this, we recorded from neurons in the middle temporal area (MT) of anesthetized marmosets (Callithrix jacchus) viewing stimuli comprising a sheet of dots that moved coherently in 1 of 12 different directions. We found that high peak response and direction selectivity both predicted that a neuron would be weighted more highly in an optimized decoding model. Although learned weights differed markedly from weights chosen according to a priori rules based on a neuron's tuning profile, decoding performance was only marginally better for the learned weights. In the models with a priori rules, selectivity is the best predictor of weighting, and defining weights according to a neuron's preferred direction and selectivity improves decoding performance to very near the maximum level possible, as defined by the learned weights. NEW & NOTEWORTHY We examined which aspects of a neuron's tuning account for its contribution to sensory coding. Strongly direction-selective neurons are weighted most highly by optimal decoders trained to discriminate motion direction. Models with predefined decoding weights demonstrate that this weighting scheme causally improved direction representation by a neuronal population. Optimizing decoders (using a generalized linear model or Fisher's linear discriminant) led to only marginally better performance than decoders based purely on a neuron's preferred direction and selectivity.

Training and Spontaneous Reinforcement of Neuronal Assemblies by Spike Timing Plasticity

The synaptic connectivity of cortex is plastic, with experience shaping the ongoing interactions between neurons. Theoretical studies of spike timing-dependent plasticity (STDP) have focused on either just pairs of neurons or large-scale simulations. A simple analytic account for how fast spike time correlations affect both microscopic and macroscopic network structure is lacking. We develop a low-dimensional mean field theory for STDP in recurrent networks and show the emergence of assemblies of strongly coupled neurons with shared stimulus preferences. After training, this connectivity is actively reinforced by spike train correlations during the spontaneous dynamics. Furthermore, the stimulus coding by cell assemblies is actively maintained by these internally generated spiking correlations, suggesting a new role for noise correlations in neural coding. Assembly formation has often been associated with firing rate-based plasticity schemes; our theory provides an alternative and complementary framework, where fine temporal correlations and STDP form and actively maintain learned structure in cortical networks.

Activity Correlations between Direction-Selective Retinal Ganglion Cells Synergistically Enhance Motion Decoding from Complex Visual Scenes

Neurons in sensory systems are often tuned to particular stimulus features. During complex naturalistic stimulation, however, multiple features may simultaneously affect neuronal responses, which complicates the readout of individual features. To investigate feature representation under complex stimulation, we studied how direction-selective ganglion cells in salamander retina respond to texture motion where direction, velocity, and spatial pattern inside the receptive field continuously change. We found that the cells preserve their direction preference under this stimulation, yet their direction encoding becomes ambiguous due to simultaneous activation by luminance changes. The ambiguities can be resolved by considering populations of direction-selective cells with different preferred directions. This gives rise to synergistic motion decoding, yielding more information from the population than the summed information from single-cell responses. Strong positive response correlations between cells with different preferred directions amplify this synergy. Our results show how correlated population activity can enhance feature extraction in complex visual scenes.

•

Direction-selective ganglion cells respond to motion as well as luminance changes

•

This obscures the readout of direction from single cells under complex texture motion

•

Population decoding improves direction readout supralinearly over individual cells

•

Strong spike correlations further enhance readout through increased synergy

Stimulus- and goal-oriented frameworks for understanding natural vision

Our knowledge of sensory processing has advanced dramatically in the last few decades, but this understanding remains far from complete, especially for stimuli with the large dynamic range and strong temporal and spatial correlations characteristic of natural visual inputs. Here we describe some of the issues that make understanding the encoding of natural images a challenge. We highlight two broad strategies for approaching this problem: a stimulus-oriented framework and a goal-oriented one. Different contexts can call for one framework or the other. Looking forward, recent advances, particularly those based in machine learning, show promise in borrowing key strengths of both frameworks and by doing so illuminating a path to a more comprehensive understanding of the encoding of natural stimuli.