Synaptic convergence regulates synchronization-dependent spike transfer in feedforward neural networks

Comparison of visual quantities in untrained neural networks

The ability to compare quantities of visual objects with two distinct measures, proportion and difference, is observed even in newborn animals. However, how this function originates in the brain, even before visual experience, remains unknown. Here, we propose a model in which neuronal tuning for quantity comparisons can arise spontaneously in completely untrained neural circuits. Using a biologically inspired model neural network, we find that single units selective to proportions and differences between visual quantities emerge in randomly initialized feedforward wirings and that they enable the network to perform quantity comparison tasks. Notably, we find that two distinct tunings to proportion and difference originate from a random summation of monotonic, nonlinear neural activities and that a slight difference in the nonlinear response function determines the type of measure. Our results suggest that visual quantity comparisons are primitive types of functions that can emerge spontaneously before learning in young brains.

Species-specific wiring of cortical circuits for small-world networks in the primary visual cortex

Long-range horizontal connections (LRCs) are conspicuous anatomical structures in the primary visual cortex (V1) of mammals, yet their detailed functions in relation to visual processing are not fully understood. Here, we show that LRCs are key components to organize a "small-world network" optimized for each size of the visual cortex, enabling the cost-efficient integration of visual information. Using computational simulations of a biologically inspired model neural network, we found that sparse LRCs added to networks, combined with dense local connections, compose a small-world network and significantly enhance image classification performance. We confirmed that the performance of the network appeared to be strongly correlated with the small-world coefficient of the model network under various conditions. Our theoretical model demonstrates that the amount of LRCs to build a small-world network depends on each size of cortex and that LRCs are beneficial only when the size of the network exceeds a certain threshold. Our model simulation of various sizes of cortices validates this prediction and provides an explanation of the species-specific existence of LRCs in animal data. Our results provide insight into a biological strategy of the brain to balance functional performance and resource cost.

Synaptic and circuit mechanisms prevent detrimentally precise correlation in the developing mammalian visual system

The developing visual thalamus and cortex extract positional information encoded in the correlated activity of retinal ganglion cells by synaptic plasticity, allowing for the refinement of connectivity. Here, we use a biophysical model of the visual thalamus during the initial visual circuit refinement period to explore the role of synaptic and circuit properties in the regulation of such neural correlations. We find that the NMDA receptor dominance, combined with weak recurrent excitation and inhibition characteristic of this age, prevents the emergence of spike-correlations between thalamocortical neurons on the millisecond timescale. Such precise correlations, which would emerge due to the broad, unrefined connections from the retina to the thalamus, reduce the spatial information contained by thalamic spikes, and therefore we term them 'parasitic' correlations. Our results suggest that developing synapses and circuits evolved mechanisms to compensate for such detrimental parasitic correlations arising from the unrefined and immature circuit.

Invariance of object detection in untrained deep neural networks

The ability to perceive visual objects with various types of transformations, such as rotation, translation, and scaling, is crucial for consistent object recognition. In machine learning, invariant object detection for a network is often implemented by augmentation with a massive number of training images, but the mechanism of invariant object detection in biological brains—how invariance arises initially and whether it requires visual experience—remains elusive. Here, using a model neural network of the hierarchical visual pathway of the brain, we show that invariance of object detection can emerge spontaneously in the complete absence of learning. First, we found that units selective to a particular object class arise in randomly initialized networks even before visual training. Intriguingly, these units show robust tuning to images of each object class under a wide range of image transformation types, such as viewpoint rotation. We confirmed that this “innate” invariance of object selectivity enables untrained networks to perform an object-detection task robustly, even with images that have been significantly modulated. Our computational model predicts that invariant object tuning originates from combinations of non-invariant units via random feedforward projections, and we confirmed that the predicted profile of feedforward projections is observed in untrained networks. Our results suggest that invariance of object detection is an innate characteristic that can emerge spontaneously in random feedforward networks.

Face detection in untrained deep neural networks

Face-selective neurons are observed in the primate visual pathway and are considered as the basis of face detection in the brain. However, it has been debated as to whether this neuronal selectivity can arise innately or whether it requires training from visual experience. Here, using a hierarchical deep neural network model of the ventral visual stream, we suggest a mechanism in which face-selectivity arises in the complete absence of training. We found that units selective to faces emerge robustly in randomly initialized networks and that these units reproduce many characteristics observed in monkeys. This innate selectivity also enables the untrained network to perform face-detection tasks. Intriguingly, we observed that units selective to various non-face objects can also arise innately in untrained networks. Our results imply that the random feedforward connections in early, untrained deep neural networks may be sufficient for initializing primitive visual selectivity.

Periodic clustering of simple and complex cells in visual cortex

Neurons in the primary visual cortex (V1) are often classified as simple or complex cells, but it is debated whether they are discrete hierarchical classes of neurons or if they represent a continuum of variation within a single class of cells. Herein, we show that simple and complex cells may arise commonly from the feedforward projections from the retina. From analysis of the cortical receptive fields in cats, we show evidence that simple and complex cells originate from the periodic variation of ON-OFF segregation in the feedforward projection of retinal mosaics, by which they organize into periodic clusters in V1. From data in cats, we observed that clusters of simple and complex receptive fields correlate topographically with orientation maps, which supports our model prediction. Our results suggest that simple and complex cells are not two distinct neural populations but arise from common retinal afferents, simultaneous with orientation tuning.

Precise Mapping of Single Neurons by Calibrated 3D Reconstruction of Brain Slices Reveals Topographic Projection in Mouse Visual Cortex

Recent breakthroughs in neuroanatomical tracing methods have helped unravel complicated neural connectivity in whole-brain tissue at single-cell resolution. However, in most cases, analysis of brain images remains dependent on highly subjective and sample-specific manual processing, preventing precise comparison across sample animals. In the present study, we introduce AMaSiNe, software for automated mapping of single neurons in the standard mouse brain atlas with annotated regions. AMaSiNe automatically calibrates misaligned and deformed slice samples to locate labeled neuronal positions from multiple brain samples into the standardized 3D Allen Mouse Brain Reference Atlas. We exploit the high fidelity and reliability of AMaSiNe to investigate the topographic structures of feedforward projections from the lateral geniculate nucleus to the primary visual area by reconstructing rabies-virus-injected brain slices in 3D space. Our results demonstrate that distinct organization of neural projections can be precisely mapped using AMaSiNe.

Retino-Cortical Mapping Ratio Predicts Columnar and Salt-and-Pepper Organization in Mammalian Visual Cortex

In the mammalian primary visual cortex, neural tuning to stimulus orientation is organized in either columnar or salt-and-pepper patterns across species. For decades, this sharp contrast has spawned fundamental questions about the origin of functional architectures in visual cortex. However, it is unknown whether these patterns reflect disparate developmental mechanisms across mammalian taxa or simply originate from variation of biological parameters under a universal development process. In this work, after the analysis of data from eight mammalian species, we show that cortical organization is predictable by a single factor, the retino-cortical mapping ratio. Groups of species with or without columnar clustering are distinguished by the feedforward sampling ratio, and model simulations with controlled mapping conditions reproduce both types of organization. Prediction from the Nyquist theorem explains this parametric division of the patterns with high accuracy. Our results imply that evolutionary variation of physical parameters may induce development of distinct functional circuitry.

Projection of Orthogonal Tiling from the Retina to the Visual Cortex

In higher mammals, the primary visual cortex (V1) is organized into diverse tuning maps of visual features. The topography of these maps intersects orthogonally, but it remains unclear how such a systematic relationship can develop. Here, we show that the orthogonal organization already exists in retinal ganglion cell (RGC) mosaics, providing a blueprint of the organization in V1. From analysis of the RGC mosaics data in monkeys and cats, we find that the ON-OFF RGC distance and ON-OFF angle of neighboring RGCs are organized into a topographic tiling across mosaics, analogous to the orthogonal intersection of cortical tuning maps. Our model simulation shows that the ON-OFF distance and angle in RGC mosaics correspondingly initiate ocular dominance/spatial frequency tuning and orientation tuning, resulting in the orthogonal intersection of cortical tuning maps. These findings suggest that the regularly structured ON-OFF patterns mirrored from the retina initiate the uniform representation of combinations of map features over the visual space.

A brain-inspired network architecture for cost-efficient object recognition in shallow hierarchical neural networks

The brain successfully performs visual object recognition with a limited number of hierarchical networks that are much shallower than artificial deep neural networks (DNNs) that perform similar tasks. Here, we show that long-range horizontal connections (LRCs), often observed in the visual cortex of mammalian species, enable such a cost-efficient visual object recognition in shallow neural networks. Using simulations of a model hierarchical network with convergent feedforward connections and LRCs, we found that the addition of LRCs to the shallow feedforward network significantly enhances the performance of networks for image classification, to a degree that is comparable to much deeper networks. We found that a combination of sparse LRCs and dense local connections dramatically increases performance per wiring cost. From network pruning with gradient-based optimization, we also confirmed that LRCs could emerge spontaneously by minimizing the total connection length while maintaining performance. Ablation of emerged LRCs led to a significant reduction of classification performance, which implies these LRCs are crucial for performing image classification. Taken together, our findings suggest a brain-inspired strategy for constructing a cost-efficient network architecture to implement parsimonious object recognition under physical constraints such as shallow hierarchical depth.

Spontaneous Retinal Waves Can Generate Long-Range Horizontal Connectivity in Visual Cortex

In the primary visual cortex (V1) of higher mammals, long-range horizontal connections (LHCs) are observed to develop, linking iso-orientation domains of cortical tuning. It is unknown how this feature-specific wiring of circuitry develops before eye-opening. Here, we suggest that LHCs in V1 may originate from spatiotemporally structured feedforward activities generated from spontaneous retinal waves. Using model simulations based on the anatomy and observed activity patterns of the retina, we show that waves propagating in retinal mosaics can initialize the wiring of LHCs by coactivating neurons of similar tuning, whereas equivalent random activities cannot induce such organizations. Simulations showed that emerged LHCs can produce the patterned activities observed in V1, matching the topography of the underlying orientation map. The model can also reproduce feature-specific microcircuits in the salt-and-pepper organizations found in rodents. Our results imply that early peripheral activities contribute significantly to cortical development of functional circuits.SIGNIFICANCE STATEMENT Long-range horizontal connections (LHCs) in the primary visual cortex (V1) are observed to emerge before the onset of visual experience, thereby selectively connecting iso-domains of orientation map. However, it is unknown how such feature-specific wirings develop before eye-opening. Here, we show that LHCs in V1 may originate from the feature-specific activation of cortical neurons by spontaneous retinal waves during early developmental stages. Our simulations of a visual cortex model show that feedforward activities from the retina initialize the spatial organization of activity patterns in V1, which induces visual feature-specific wirings in the V1 neurons. Our model also explains the origin of cortical microcircuits observed in rodents, suggesting that the proposed developmental mechanism is universally applicable to circuits of various mammalian species.

Distinct subtypes of inhibitory interneurons differentially promote the propagation of rate and temporal codes in the feedforward neural network

Sensory information is believed to be encoded in neuronal spikes using two different neural codes, the rate code (spike firing rate) and the temporal code (precisely-timed spikes). Since the sensory cortex has a highly hierarchical feedforward structure, sensory information-carrying neural codes should reliably propagate across the feedforward network (FFN) of the cortex. Experimental evidence suggests that inhibitory interneurons, such as the parvalbumin-positive (PV) and somatostatin-positive (SST) interneurons, that have distinctively different electrophysiological and synaptic properties, modulate the neural codes during sensory information processing in the cortex. However, how PV and SST interneurons impact on the neural code propagation in the cortical FFN is unknown. We address this question by building a five-layer FFN model consisting of a physiologically realistic Hodgkin-Huxley-type models of excitatory neurons and PV/SST interneurons at different ratios. In response to different firing rate inputs (20-80 Hz), a higher ratio of PV over SST interneurons promoted a reliable propagation of all ranges of firing rate inputs. In contrast, in response to a range of precisely-timed spikes in the form of pulse-packets [with a different number of spikes (α, 40-400 spikes) and degree of dispersion (σ, 0-20 ms)], a higher ratio of SST over PV interneurons promoted a reliable propagation of pulse-packets. Our simulation results show that PV and SST interneurons differentially promote a reliable propagation of the rate and temporal codes, respectively, indicating that the dynamic recruitment of PV and SST interneurons may play critical roles in a reliable propagation of sensory information-carrying neural codes in the cortical FFN.