Spontaneous Retinal Waves Can Generate Long-Range Horizontal Connectivity in Visual Cortex

Pre-training artificial neural networks with spontaneous retinal activity improves motion prediction in natural scenes

The ability to process visual stimuli rich with motion represents an essential skill for animal survival and is largely already present at the onset of vision. Although the exact mechanisms underlying its maturation remain elusive, spontaneous activity patterns in the retina, known as retinal waves, have been shown to contribute to this developmental process. Retinal waves exhibit complex spatio-temporal statistics and contribute to the establishment of circuit connectivity and function in the visual system, including the formation of retinotopic maps and the refinement of receptive fields in downstream areas such as the thalamus and visual cortex. Recent work in mice has shown that retinal waves have statistical features matching those of natural visual stimuli, such as optic flow, suggesting that they could prime the visual system for motion processing upon vision onset. Motivated by these findings, we examined whether artificial neural network (ANN) models trained on natural movies show improved performance if pre-trained with retinal waves. We employed the spatio-temporally complex task of next-frame prediction, in which the ANN was trained to predict the next frame based on preceding input frames of a movie. We found that pre-training ANNs with retinal waves enhances the processing of real-world visual stimuli and accelerates learning. Strikingly, when we merely replaced the initial training epochs on naturalistic stimuli with retinal waves, keeping the total training time the same, we still found that an ANN trained on retinal waves temporarily outperforms one trained solely on natural movies. Similar to observations made in biological systems, we also found that pre-training with spontaneous activity refines the receptive field of ANN neurons. Overall, our work sheds light on the functional role of spatio-temporally patterned spontaneous activity in the processing of motion in natural scenes, suggesting it acts as a training signal to prepare the developing visual system for adult visual processing.

Modeling the Emergence of Circuit Organization and Function during Development

Developing neural circuits show unique patterns of spontaneous activity and structured network connectivity shaped by diverse activity-dependent plasticity mechanisms. Based on extensive experimental work characterizing patterns of spontaneous activity in different brain regions over development, theoretical and computational models have played an important role in delineating the generation and function of individual features of spontaneous activity and their role in the plasticity-driven formation of circuit connectivity. Here, we review recent modeling efforts that explore how the developing cortex and hippocampus generate spontaneous activity, focusing on specific connectivity profiles and the gradual strengthening of inhibition as the key drivers behind the observed developmental changes in spontaneous activity. We then discuss computational models that mechanistically explore how different plasticity mechanisms use this spontaneous activity to instruct the formation and refinement of circuit connectivity, from the formation of single neuron receptive fields to sensory feature maps and recurrent architectures. We end by highlighting several open challenges regarding the functional implications of the discussed circuit changes, wherein models could provide the missing step linking immature developmental and mature adult information processing capabilities.

Relationship between functional structures and horizontal connections in macaque inferior temporal cortex

Horizontal connections in anterior inferior temporal cortex (ITC) are thought to play an important role in object recognition by integrating information across spatially separated functional columns, but their functional organization remains unclear. Using a combination of optical imaging, electrophysiological recording, and anatomical tracing, we investigated the relationship between stimulus-response maps and patterns of horizontal axon terminals in the macaque ITC. In contrast to the "like-to-like" connectivity observed in the early visual cortex, we found that horizontal axons in ITC do not preferentially connect sites with similar object selectivity. While some axon terminal patches shared responsiveness to specific visual features with the injection site, many connected to regions with different selectivity. Our results suggest that horizontal connections in anterior ITC exhibit diverse functional connectivity, potentially supporting flexible integration of visual information for advanced object recognition processes.

A unifying framework for functional organization in early and higher ventral visual cortex

A key feature of cortical systems is functional organization: the arrangement of functionally distinct neurons in characteristic spatial patterns. However, the principles underlying the emergence of functional organization in the cortex are poorly understood. Here, we develop the topographic deep artificial neural network (TDANN), the first model to predict several aspects of the functional organization of multiple cortical areas in the primate visual system. We analyze the factors driving the TDANN's success and find that it balances two objectives: learning a task-general sensory representation and maximizing the spatial smoothness of responses according to a metric that scales with cortical surface area. In turn, the representations learned by the TDANN are more brain-like than in spatially unconstrained models. Finally, we provide evidence that the TDANN's functional organization balances performance with between-area connection length. Our results offer a unified principle for understanding the functional organization of the primate ventral visual system.

Emergence of input selective recurrent dynamics via information transfer maximization

Network structures of the brain have wiring patterns specialized for specific functions. These patterns are partially determined genetically or evolutionarily based on the type of task or stimulus. These wiring patterns are important in information processing; however, their organizational principles are not fully understood. This study frames the maximization of information transmission alongside the reduction of maintenance costs as a multi-objective optimization challenge, utilizing information theory and evolutionary computing algorithms with an emphasis on the visual system. The goal is to understand the underlying principles of circuit formation by exploring the patterns of wiring and information processing. The study demonstrates that efficient information transmission necessitates sparse circuits with internal modular structures featuring distinct wiring patterns. Significant trade-offs underscore the necessity of balance in wiring pattern development. The dynamics of effective circuits exhibit moderate flexibility in response to stimuli, in line with observations from prior visual system studies. Maximizing information transfer may allow for the self-organization of information processing functions similar to actual biological circuits, without being limited by modality. This study offers insights into neuroscience and the potential to improve reservoir computing performance.

Self-organization of modular activity in immature cortical networks

During development, cortical activity is organized into distributed modular patterns that are a precursor of the mature columnar functional architecture. Theoretically, such structured neural activity can emerge dynamically from local synaptic interactions through a recurrent network with effective local excitation with lateral inhibition (LE/LI) connectivity. Utilizing simultaneous widefield calcium imaging and optogenetics in juvenile ferret cortex prior to eye opening, we directly test several critical predictions of an LE/LI mechanism. We show that cortical networks transform uniform stimulations into diverse modular patterns exhibiting a characteristic spatial wavelength. Moreover, patterned optogenetic stimulation matching this wavelength selectively biases evoked activity patterns, while stimulation with varying wavelengths transforms activity towards this characteristic wavelength, revealing a dynamic compromise between input drive and the network's intrinsic tendency to organize activity. Furthermore, the structure of early spontaneous cortical activity - which is reflected in the developing representations of visual orientation - strongly overlaps that of uniform opto-evoked activity, suggesting a common underlying mechanism as a basis for the formation of orderly columnar maps underlying sensory representations in the brain.

Self-organization of modular activity in immature cortical networks

During development, cortical activity is organized into distributed modular patterns that are a precursor of the mature columnar functional architecture. Theoretically, such structured neural activity can emerge dynamically from local synaptic interactions through a recurrent network with effective local excitation with lateral inhibition (LE/LI) connectivity. Utilizing simultaneous widefield calcium imaging and optogenetics in juvenile ferret cortex prior to eye opening, we directly test several critical predictions of an LE/LI mechanism. We show that cortical networks transform uniform stimulations into diverse modular patterns exhibiting a characteristic spatial wavelength. Moreover, patterned optogenetic stimulation matching this wavelength selectively biases evoked activity patterns, while stimulation with varying wavelengths transforms activity towards this characteristic wavelength, revealing a dynamic compromise between input drive and the network's intrinsic tendency to organize activity. Furthermore, the structure of early spontaneous cortical activity - which is reflected in the developing representations of visual orientation - strongly overlaps that of uniform opto-evoked activity, suggesting a common underlying mechanism as a basis for the formation of orderly columnar maps underlying sensory representations in the brain.

Comparison of visual quantities in untrained neural networks

The ability to compare quantities of visual objects with two distinct measures, proportion and difference, is observed even in newborn animals. However, how this function originates in the brain, even before visual experience, remains unknown. Here, we propose a model in which neuronal tuning for quantity comparisons can arise spontaneously in completely untrained neural circuits. Using a biologically inspired model neural network, we find that single units selective to proportions and differences between visual quantities emerge in randomly initialized feedforward wirings and that they enable the network to perform quantity comparison tasks. Notably, we find that two distinct tunings to proportion and difference originate from a random summation of monotonic, nonlinear neural activities and that a slight difference in the nonlinear response function determines the type of measure. Our results suggest that visual quantity comparisons are primitive types of functions that can emerge spontaneously before learning in young brains.

Species-specific wiring of cortical circuits for small-world networks in the primary visual cortex

Long-range horizontal connections (LRCs) are conspicuous anatomical structures in the primary visual cortex (V1) of mammals, yet their detailed functions in relation to visual processing are not fully understood. Here, we show that LRCs are key components to organize a "small-world network" optimized for each size of the visual cortex, enabling the cost-efficient integration of visual information. Using computational simulations of a biologically inspired model neural network, we found that sparse LRCs added to networks, combined with dense local connections, compose a small-world network and significantly enhance image classification performance. We confirmed that the performance of the network appeared to be strongly correlated with the small-world coefficient of the model network under various conditions. Our theoretical model demonstrates that the amount of LRCs to build a small-world network depends on each size of cortex and that LRCs are beneficial only when the size of the network exceeds a certain threshold. Our model simulation of various sizes of cortices validates this prediction and provides an explanation of the species-specific existence of LRCs in animal data. Our results provide insight into a biological strategy of the brain to balance functional performance and resource cost.

Cortical topographic motifs emerge in a self-organized map of object space

The human ventral visual stream has a highly systematic organization of object information, but the causal pressures driving these topographic motifs are highly debated. Here, we use self-organizing principles to learn a topographic representation of the data manifold of a deep neural network representational space. We find that a smooth mapping of this representational space showed many brain-like motifs, with a large-scale organization by animacy and real-world object size, supported by mid-level feature tuning, with naturally emerging face- and scene-selective regions. While some theories of the object-selective cortex posit that these differently tuned regions of the brain reflect a collection of distinctly specified functional modules, the present work provides computational support for an alternate hypothesis that the tuning and topography of the object-selective cortex reflect a smooth mapping of a unified representational space.

A Unifying Principle for the Functional Organization of Visual Cortex

A key feature of many cortical systems is functional organization: the arrangement of neurons with specific functional properties in characteristic spatial patterns across the cortical surface. However, the principles underlying the emergence and utility of functional organization are poorly understood. Here we develop the Topographic Deep Artificial Neural Network (TDANN), the first unified model to accurately predict the functional organization of multiple cortical areas in the primate visual system. We analyze the key factors responsible for the TDANN's success and find that it strikes a balance between two specific objectives: achieving a task-general sensory representation that is self-supervised, and maximizing the smoothness of responses across the cortical sheet according to a metric that scales relative to cortical surface area. In turn, the representations learned by the TDANN are lower dimensional and more brain-like than those in models that lack a spatial smoothness constraint. Finally, we provide evidence that the TDANN's functional organization balances performance with inter-area connection length, and use the resulting models for a proof-of-principle optimization of cortical prosthetic design. Our results thus offer a unified principle for understanding functional organization and a novel view of the functional role of the visual system in particular.

Invariance of object detection in untrained deep neural networks

The ability to perceive visual objects with various types of transformations, such as rotation, translation, and scaling, is crucial for consistent object recognition. In machine learning, invariant object detection for a network is often implemented by augmentation with a massive number of training images, but the mechanism of invariant object detection in biological brains—how invariance arises initially and whether it requires visual experience—remains elusive. Here, using a model neural network of the hierarchical visual pathway of the brain, we show that invariance of object detection can emerge spontaneously in the complete absence of learning. First, we found that units selective to a particular object class arise in randomly initialized networks even before visual training. Intriguingly, these units show robust tuning to images of each object class under a wide range of image transformation types, such as viewpoint rotation. We confirmed that this “innate” invariance of object selectivity enables untrained networks to perform an object-detection task robustly, even with images that have been significantly modulated. Our computational model predicts that invariant object tuning originates from combinations of non-invariant units via random feedforward projections, and we confirmed that the predicted profile of feedforward projections is observed in untrained networks. Our results suggest that invariance of object detection is an innate characteristic that can emerge spontaneously in random feedforward networks.

Unification of free energy minimization, spatiotemporal energy, and dimension reduction models of V1 organization: Postnatal learning on an antenatal scaffold

Developmental selection of neurons and synapses so as to maximize pulse synchrony has recently been used to explain antenatal cortical development. Consequences of the same selection process—an application of the Free Energy Principle—are here followed into the postnatal phase in V1, and the implications for cognitive function are considered. Structured inputs transformed via lag relay in superficial patch connections lead to the generation of circumferential synaptic connectivity superimposed upon the antenatal, radial, “like-to-like” connectivity surrounding each singularity. The spatiotemporal energy and dimension reduction models of cortical feature preferences are accounted for and unified within the expanded model, and relationships of orientation preference (OP), space frequency preference (SFP), and temporal frequency preference (TFP) are resolved. The emergent anatomy provides a basis for “active inference” that includes interpolative modification of synapses so as to anticipate future inputs, as well as learn directly from present stimuli. Neurodynamic properties are those of heteroclinic networks with coupled spatial eigenmodes.

Visuospatial coding as ubiquitous scaffolding for human cognition

For more than 100 years we have known that the visual field is mapped onto the surface of visual cortex, imposing an inherently spatial reference frame on visual information processing. Recent studies highlight visuospatial coding not only throughout visual cortex, but also brain areas not typically considered visual. Such widespread access to visuospatial coding raises important questions about its role in wider cognitive functioning. Here, we synthesise these recent developments and propose that visuospatial coding scaffolds human cognition by providing a reference frame through which neural computations interface with environmental statistics and task demands via perception-action loops.

Projection of Orthogonal Tiling from the Retina to the Visual Cortex

In higher mammals, the primary visual cortex (V1) is organized into diverse tuning maps of visual features. The topography of these maps intersects orthogonally, but it remains unclear how such a systematic relationship can develop. Here, we show that the orthogonal organization already exists in retinal ganglion cell (RGC) mosaics, providing a blueprint of the organization in V1. From analysis of the RGC mosaics data in monkeys and cats, we find that the ON-OFF RGC distance and ON-OFF angle of neighboring RGCs are organized into a topographic tiling across mosaics, analogous to the orthogonal intersection of cortical tuning maps. Our model simulation shows that the ON-OFF distance and angle in RGC mosaics correspondingly initiate ocular dominance/spatial frequency tuning and orientation tuning, resulting in the orthogonal intersection of cortical tuning maps. These findings suggest that the regularly structured ON-OFF patterns mirrored from the retina initiate the uniform representation of combinations of map features over the visual space.

Visual number sense in untrained deep neural networks

Number sense, the ability to estimate numerosity, is observed in naïve animals, but how this cognitive function emerges in the brain remains unclear. Here, using an artificial deep neural network that models the ventral visual stream of the brain, we show that number-selective neurons can arise spontaneously, even in the complete absence of learning. We also show that the responses of these neurons can induce the abstract number sense, the ability to discriminate numerosity independent of low-level visual cues. We found number tuning in a randomly initialized network originating from a combination of monotonically decreasing and increasing neuronal activities, which emerges spontaneously from the statistical properties of bottom-up projections. We confirmed that the responses of these number-selective neurons show the single- and multineuron characteristics observed in the brain and enable the network to perform number comparison tasks. These findings provide insight into the origin of innate cognitive functions.