Compact deep neural network models of visual cortex

A powerful approach to understanding the computations carried out in visual cortex is to develop models that predict neural responses to arbitrary images. Deep neural network (DNN) models have worked remarkably well at predicting neural responses [1, 2, 3], yet their underlying computations remain buried in millions of parameters. Have we simply replaced one complicated system in vivo with another in silico ? Here, we train a data-driven deep ensemble model that predicts macaque V4 responses ∼50% more accurately than currently-used task-driven DNN models. We then compress this deep ensemble to identify compact models that have 5,000x fewer parameters yet equivalent accuracy as the deep ensemble. We verified that the stimulus preferences of the compact models matched those of the real V4 neurons by measuring V4 responses to both 'maximizing' and adversarial images generated using compact models. We then analyzed the inner workings of the compact models and discovered a common circuit motif: Compact models share a similar set of filters in early stages of processing but then specialize by heavily consolidating this shared representation with a precise readout. This suggests that a V4 neuron's stimulus preference is determined entirely by its consolidation step. To demonstrate this, we investigated the compression step of a dot-detecting compact model and found a set of simple computations that may be carried out by dot-selective V4 neurons. Overall, our work demonstrates that the DNN models currently used in computational neuroscience are needlessly large; our approach provides a new way forward for obtaining explainable, high-accuracy models of visual cortical neurons.

Expectation reshapes V4 neuronal activity and improves perceptual performance

Recent visual experience creates expectations that heavily influence our visual perception. How does expectation shape the activity of cortical neurons to allow for improved perceptual discrimination of visual inputs? We recorded from populations of neurons in visual cortical area V4 while monkeys performed a natural image change detection task under different expectation conditions. We found that higher expectation led to an improvement in the ability to detect a change in an image. This improvement was associated with decreased neural responses to the image, providing evidence that a reduction in activity can improve stimulus encoding. The decrease in response could not be fully explained by short-timescale adaptation, suggesting partially separate mechanisms of adaptation and expectation. Additionally, higher expectation was associated with decreased trial-to-trial shared variability, indicating that a reduction in variability is a key means by which expectation influences perception. Taken together, the results of our study contribute to an understanding of how visual experience and expectation can shape our perception and behavior through modulating activity patterns across the visual cortex.

Development of Natural Scene Representation in Primary Visual Cortex Requires Early Postnatal Experience

The development of the visual system is known to be shaped by early-life experience. To identify response properties that contribute to enhanced natural scene representation, we performed calcium imaging of excitatory neurons in the primary visual cortex (V1) of awake mice raised in three different conditions (standard-reared, dark-reared, and delayed-visual experience) and compared neuronal responses to natural scene features in relation to simpler grating stimuli that varied in orientation and spatial frequency. We assessed population selectivity in the V1 by using decoding methods and found that natural scene discriminability increased by 75% between the ages of 4 and 6 weeks. Both natural scene and grating discriminability were higher in standard-reared animals than in those raised in the dark. This increase in discriminability was accompanied by a reduction in the number of neurons that responded to low-spatial-frequency gratings. At the same time, there was an increase in neuronal preference for natural scenes. Light exposure restricted to a 2- to 4-week window during adulthood did not induce improvements in natural scene or in grating stimulus discriminability. Our results demonstrate that experience reduces the number of neurons needed to effectively encode grating stimuli and that early visual experience enhances natural scene discriminability by directly increasing responsiveness to natural scene features.