Sparse synaptic connectivity is required for decorrelation and pattern separation in feedforward networks

Regulation of dentate gyrus pattern separation by hilus ectopic granule cells

The dentate gyrus (DG) in hippocampus is reported to perform pattern separation, converting similar inputs into different outputs and thus avoiding memory interference. Previous studies have found that human and mice with epilepsy have significant pattern separation defects and a portion of adult-born granule cells (abGCs) migrate abnormally into the hilus, forming hilus ectopic granule cells (HEGCs). For the lack of relevant pathophysiological experiments, how HEGCs affect pattern separation remains unclear. Therefore, in this paper, we will construct the DG neuronal circuit and focus on discussing effects of HEGCs on pattern separation numerically. The obtained results showed that HEGCs impaired pattern separation efficiency since the sparse firing of granule cells (GCs) was destroyed. We provided new insights into the underlining mechanisms of HEGCs impairing pattern separation through analyzing two excitatory circuits: GC-HEGC-GC and GC-Mossy cell (MC)-GC, both of which involve the participation of HEGCs within the DG. It is revealed that the recurrent excitatory circuit GC-HEGC-GC formed by HEGCs mossy fiber sprouting significantly enhanced GCs activity, consequently disrupted pattern separation. However, another excitatory circuit had negligible effects on pattern separation due to the direct and indirect influences of MCs on GCs, which in turn led to the GCs sparse firing. Thus, HEGCs impair DG pattern separation mainly through the GC-HEGC-GC circuit and therefore ablating HEGCs may be one of the effective ways to improve pattern separation in patients with epilepsy.

Linking cellular-level phenomena to brain architecture: the case of spiking cerebellar controllers

Linking cellular-level phenomena to brain architecture and behavior is a holy grail for theoretical and computational neuroscience. Advances in neuroinformatics have recently allowed scientists to embed spiking neural networks of the cerebellum with realistic neuron models and multiple synaptic plasticity rules into sensorimotor controllers. By minimizing the distance (error) between the desired and the actual sensory state, and exploiting the sensory prediction, the cerebellar network acquires knowledge about the body-environment interaction and generates corrective signals. In doing so, the cerebellum implements a generalized computational algorithm, allowing it "to learn to predict the timing between correlated events" in a rich set of behavioral contexts. Plastic changes evolve trial by trial and are distributed over multiple synapses, regulating the timing of neuronal discharge and fine-tuning high-speed movements on the millisecond timescale. Thus, spiking cerebellar built-in controllers, among various computational approaches to studying cerebellar function, are helping to reveal the cellular-level substrates of network learning and signal coding, opening new frontiers for predictive computing and autonomous learning in robots.

Fast implicit and slow explicit learning of temporal context

One is seldom aware of the anticipatory and preemptive feats that the eyeblink system achieves in daily life but it frequently protects the eye from projectiles gone awry and insects on apparent collision courses. This poor awareness is why predictive eyeblinks are considered a form of implicit learning. In motor neuroscience, implicit learning is considered to be slow and, eyeblink conditioning, in particular, is believed to be a rigid and inflexible cerebellar-dependent behavior. In cognitive neuroscience, however, implicit and automatic processes are thought to be rapidly acquired. Here we show that the eyeblink system is, in fact, capable of remarkable cognitive flexibility and can learn on more rapid timescales than previously expected. In a task where we yoked contextual learning of predictive eyeblinks and manual responses in humans, well-timed eyeblink responses flexibly adjusted to external context on each trial. The temporal precision of the predictive eyeblinks exceeded that of manual response times. Learning of the well-timed eyeblink responses was also more rapid than that for the manual response times. This pattern persevered with the use of a cognitive strategy, which seemed to accelerate both types of learning. These results suggest that behaviors associated with the cerebellar cortex that were previously believed to be inflexible and largely implicit, can demonstrate rapid and precise context-dependent temporal control.

Assistive sensory-motor perturbations influence learned neural representations

Task errors are used to learn and refine motor skills. We investigated how task assistance influences learned neural representations using Brain-Computer Interfaces (BCIs), which map neural activity into movement via a decoder. We analyzed motor cortex activity as monkeys practiced BCI with a decoder that adapted to improve or maintain performance over days. Over time, task-relevant information became concentrated in fewer neurons, unlike with fixed decoders. At the population level, task information also became largely confined to a few neural modes that accounted for an unexpectedly small fraction of the population variance. A neural network model suggests the adaptive decoders directly contribute to forming these more compact neural representations. Our findings show that assistive decoders manipulate error information used for long-term learning computations like credit assignment, which informs our understanding of motor learning and has implications for designing real-world BCIs.

The NeuroML ecosystem for standardized multi-scale modeling in neuroscience

Data-driven models of neurons and circuits are important for understanding how the properties of membrane conductances, synapses, dendrites, and the anatomical connectivity between neurons generate the complex dynamical behaviors of brain circuits in health and disease. However, the inherent complexity of these biological processes makes the construction and reuse of biologically detailed models challenging. A wide range of tools have been developed to aid their construction and simulation, but differences in design and internal representation act as technical barriers to those who wish to use data-driven models in their research workflows. NeuroML, a model description language for computational neuroscience, was developed to address this fragmentation in modeling tools. Since its inception, NeuroML has evolved into a mature community standard that encompasses a wide range of model types and approaches in computational neuroscience. It has enabled the development of a large ecosystem of interoperable open-source software tools for the creation, visualization, validation, and simulation of data-driven models. Here, we describe how the NeuroML ecosystem can be incorporated into research workflows to simplify the construction, testing, and analysis of standardized models of neural systems, and supports the FAIR (Findability, Accessibility, Interoperability, and Reusability) principles, thus promoting open, transparent and reproducible science.

Cerebellar-driven cortical dynamics can enable task acquisition, switching and consolidation

The brain must maintain a stable world model while rapidly adapting to the environment, but the underlying mechanisms are not known. Here, we posit that cortico-cerebellar loops play a key role in this process. We introduce a computational model of cerebellar networks that learn to drive cortical networks with task-outcome predictions. First, using sensorimotor tasks, we show that cerebellar feedback in the presence of stable cortical networks is sufficient for rapid task acquisition and switching. Next, we demonstrate that, when trained in working memory tasks, the cerebellum can also underlie the maintenance of cognitive-specific dynamics in the cortex, explaining a range of optogenetic and behavioural observations. Finally, using our model, we introduce a systems consolidation theory in which task information is gradually transferred from the cerebellum to the cortex. In summary, our findings suggest that cortico-cerebellar loops are an important component of task acquisition, switching, and consolidation in the brain.

Recurrent Connectivity Shapes Spatial Coding in Hippocampal CA3 Subregions

Stable and flexible neural representations of space in the hippocampus are crucial for navigating complex environments. However, how these distinct representations emerge from the underlying local circuit architecture remains unknown. Using two-photon imaging of CA3 subareas during active behavior, we reveal opposing coding strategies within specific CA3 subregions, with proximal neurons demonstrating stable and generalized representations and distal neurons showing dynamic and context-specific activity. We show in artificial neural network models that varying the recurrence level causes these differences in coding properties to emerge. We confirmed the contribution of recurrent connectivity to functional heterogeneity by characterizing the representational geometry of neural recordings and comparing it with theoretical predictions of neural manifold dimensionality. Our results indicate that local circuit organization, particularly recurrent connectivity among excitatory neurons, plays a key role in shaping complementary spatial representations within the hippocampus.

Increased understanding of complex neuronal circuits in the cerebellar cortex

The prevailing belief has been that the fundamental structures of cerebellar neuronal circuits, consisting of a few major neuron types, are simple and well understood. Given that the cerebellum has long been known to be crucial for motor behaviors, these simple yet organized circuit structures seemed beneficial for theoretical studies proposing neural mechanisms underlying cerebellar motor functions and learning. On the other hand, experimental studies using advanced techniques have revealed numerous structural properties that were not traditionally defined. These include subdivided neuronal types and their circuit structures, feedback pathways from output Purkinje cells, and the multidimensional organization of neuronal interactions. With the recent recognition of the cerebellar involvement in non-motor functions, it is possible that these newly identified structural properties, which are potentially capable of generating greater complexity than previously recognized, are associated with increased information capacity. This, in turn, could contribute to the wide range of cerebellar functions. However, it remains largely unknown how such structural properties contribute to cerebellar neural computations through the regulation of neuronal activity or synaptic transmissions. To promote further research into cerebellar circuit structures and their functional significance, we aim to summarize the newly identified structural properties of the cerebellar cortex and discuss future research directions concerning cerebellar circuit structures and their potential functions.

Gap Junctions May Have A Computational Function In The Cerebellum: A Hypothesis

In the cerebellum, granule cells make parallel fibre contact on (and excite) Golgi cells and Golgi cells inhibit granule cells, forming an open feedback loop. Parallel fibres excite Golgi cells synaptically, each making a single contact. Golgi cells inhibit granule cells in a structure called a glomerulus almost exclusively by GABA spillover acting through extrasynaptic GABAA receptors. Golgi cells are connected dendritically by gap junctions. It has long been suspected that feedback contributes to homeostatic regulation of parallel fibre signals activity, causing the fraction of the population that are active to be maintained at a low level. We present a detailed neurophysiological and computationally-rendered model of functionally grouped Golgi cells which can infer the density of parallel fibre signals activity and convert it into proportional modulation of inhibition of granule cells. The conversion is unlearned and not actively computed; rather, output is simply the computational effect of cell morphology and network architecture. Unexpectedly, the conversion becomes more precise at low density, suggesting that self-regulation is attracted to sparse code, because it is stable. A computational function of gap junctions may not be confined to the cerebellum.

Effect of adult hippocampal neurogenesis on pattern separation and its applications

Adult hippocampal neurogenesis (AHN) is considered essential in memory formation. The dentate gyrus neural network containing newborn dentate gyrus granule cells at the critical period (4-6 weeks) have been widely discussed in neurophysiological and behavioral experiments. However, how newborn dentate gyrus granule cells at this critical period influence pattern separation of dentate gyrus remains unclear. To address this issue, we propose a biologically related dentate gyrus neural network model with AHN. By Leveraging this model, we find pattern separation is enhanced at the medium level of neurogenesis (5% of mature granule cells). This is because the sparse firing of mature granule cells is increased. We can understand this change from the following two aspects. On one hand, newborn granule cells compete with mature granule cells for inputs from the entorhinal cortex, thereby weakening the firing of mature granule cells. On the other hand, newborn granule cells effectively enhance the feedback inhibition level of the network by promoting the firing of interneurons (Mossy cells and Basket cells) and then indirectly regulating the sparse firing of mature granule cells. To verify the validity of the model for pattern separation, we apply the proposed model to a similar concept separation task and reveal that our model outperforms the original model counterparts in this task.

Ion-channel degeneracy and heterogeneities in the emergence of signature physiological characteristics of dentate gyrus granule cells

Complex systems are neither fully determined nor completely random. Biological complex systems, including single neurons, manifest intermediate regimes of randomness that recruit integration of specific combinations of functionally specialized subsystems. Such emergence of biological function provides the substrate for the expression of degeneracy, the ability of disparate combinations of subsystems to yield similar function. Here, we present evidence for the expression of degeneracy in morphologically realistic models of dentate gyrus granule cells (GCs) through functional integration of disparate ion-channel combinations. We performed a 45-parameter randomized search spanning 16 active and passive ion channels, each biophysically constrained by their gating kinetics and localization profiles, to search for valid GC models. Valid models were those that satisfied 17 sub- and suprathreshold cellular-scale electrophysiological measurements from rat GCs. A vast majority (>99%) of the 15,000 random models were not electrophysiologically valid, demonstrating that arbitrarily random ion-channel combinations would not yield GC functions. The 141 valid models (0.94% of 15,000) manifested heterogeneities in and cross-dependencies across local and propagating electrophysiological measurements, which matched with their respective biological counterparts. Importantly, these valid models were widespread throughout the parametric space and manifested weak cross-dependencies across different parameters. These observations together showed that GC physiology could neither be obtained by entirely random ion-channel combinations nor is there an entirely determined single parametric combination that satisfied all constraints. The complexity, the heterogeneities in measurement and parametric spaces, and degeneracy associated with GC physiology should be rigorously accounted for while assessing GCs and their robustness under physiological and pathological conditions.NEW & NOTEWORTHY A recent study from our laboratory had demonstrated pronounced heterogeneities in a set of 17 electrophysiological measurements obtained from a large population of rat hippocampal granule cells. Here, we demonstrate the manifestation of ion-channel degeneracy in a heterogeneous population of morphologically realistic conductance-based granule cell models that were validated against these measurements and their cross-dependencies. Our analyses show that single neurons are complex entities whose functions emerge through intricate interactions among several functionally specialized subsystems.

A cerebellar granule cell-climbing fiber computation to learn to track long time intervals

In classical cerebellar learning, Purkinje cells (PkCs) associate climbing fiber (CF) error signals with predictive granule cells (GrCs) that were active just prior (∼150 ms). The cerebellum also contributes to behaviors characterized by longer timescales. To investigate how GrC-CF-PkC circuits might learn seconds-long predictions, we imaged simultaneous GrC-CF activity over days of forelimb operant conditioning for delayed water reward. As mice learned reward timing, numerous GrCs developed anticipatory activity ramping at different rates until reward delivery, followed by widespread time-locked CF spiking. Relearning longer delays further lengthened GrC activations. We computed CF-dependent GrC→PkC plasticity rules, demonstrating that reward-evoked CF spikes sufficed to grade many GrC synapses by anticipatory timing. We predicted and confirmed that PkCs could thereby continuously ramp across seconds-long intervals from movement to reward. Learning thus leads to new GrC temporal bases linking predictors to remote CF reward signals-a strategy well suited for learning to track the long intervals common in cognitive domains.

Dynamic gating of perceptual flexibility by non-classically responsive cortical neurons

The ability to flexibly respond to sensory cues in dynamic environments is essential to adaptive auditory-guided behaviors. Cortical spiking responses during behavior are highly diverse, ranging from reliable trial-averaged responses to seemingly random firing patterns. While the reliable responses of 'classically responsive' cells have been extensively studied for decades, the contribution of irregular spiking 'non-classically responsive' cells to behavior has remained underexplored despite their prevalence. Here, we show that flexible auditory behavior results from interactions between local auditory cortical circuits comprised of heterogeneous responses and inputs from secondary motor cortex. Strikingly, non-classically responsive neurons in auditory cortex were preferentially recruited during learning, specifically during rapid learning phases when the greatest gains in behavioral performance occur. Population-level decoding revealed that during rapid learning mixed ensembles comprised of both classically and non-classically responsive cells encode significantly more task information than homogenous ensembles of either type and emerge as a functional unit critical for learning. Optogenetically silencing inputs from secondary motor cortex selectively modulated non-classically responsive cells in the auditory cortex and impaired reversal learning by preventing the remapping of a previously learned stimulus-reward association. Top-down inputs orchestrated highly correlated non-classically responsive ensembles in sensory cortex providing a unique task-relevant manifold for learning. Thus, non-classically responsive cells in sensory cortex are preferentially recruited by top-down inputs to enable neural and behavioral flexibility.

Spatial constraints and cell surface molecule depletion structure a randomly connected learning circuit

The brain can represent almost limitless objects to "categorize an unlabeled world" (Edelman, 1989). This feat is supported by expansion layer circuit architectures, in which neurons carrying information about discrete sensory channels make combinatorial connections onto much larger postsynaptic populations. Combinatorial connections in expansion layers are modeled as randomized sets. The extent to which randomized wiring exists in vivo is debated, and how combinatorial connectivity patterns are generated during development is not understood. Non-deterministic wiring algorithms could program such connectivity using minimal genomic information. Here, we investigate anatomic and transcriptional patterns and perturb partner availability to ask how Kenyon cells, the expansion layer neurons of the insect mushroom body, obtain combinatorial input from olfactory projection neurons. Olfactory projection neurons form their presynaptic outputs in an orderly, predictable, and biased fashion. We find that Kenyon cells accept spatially co-located but molecularly heterogeneous inputs from this orderly map, and ask how Kenyon cell surface molecule expression impacts partner choice. Cell surface immunoglobulins are broadly depleted in Kenyon cells, and we propose that this allows them to form connections with molecularly heterogeneous partners. This model can explain how developmentally identical neurons acquire diverse wiring identities.

Gradient-free training of recurrent neural networks using random perturbations

Recurrent neural networks (RNNs) hold immense potential for computations due to their Turing completeness and sequential processing capabilities, yet existing methods for their training encounter efficiency challenges. Backpropagation through time (BPTT), the prevailing method, extends the backpropagation (BP) algorithm by unrolling the RNN over time. However, this approach suffers from significant drawbacks, including the need to interleave forward and backward phases and store exact gradient information. Furthermore, BPTT has been shown to struggle to propagate gradient information for long sequences, leading to vanishing gradients. An alternative strategy to using gradient-based methods like BPTT involves stochastically approximating gradients through perturbation-based methods. This learning approach is exceptionally simple, necessitating only forward passes in the network and a global reinforcement signal as feedback. Despite its simplicity, the random nature of its updates typically leads to inefficient optimization, limiting its effectiveness in training neural networks. In this study, we present a new approach to perturbation-based learning in RNNs whose performance is competitive with BPTT, while maintaining the inherent advantages over gradient-based learning. To this end, we extend the recently introduced activity-based node perturbation (ANP) method to operate in the time domain, leading to more efficient learning and generalization. We subsequently conduct a range of experiments to validate our approach. Our results show similar performance, convergence time and scalability when compared to BPTT, strongly outperforming standard node perturbation and weight perturbation methods. These findings suggest that perturbation-based learning methods offer a versatile alternative to gradient-based methods for training RNNs which can be ideally suited for neuromorphic computing applications.

Computational role of structure in neural activity and connectivity

One major challenge of neuroscience is identifying structure in seemingly disorganized neural activity. Different types of structure have different computational implications that can help neuroscientists understand the functional role of a particular brain area. Here, we outline a unified approach to characterize structure by inspecting the representational geometry and the modularity properties of the recorded activity and show that a similar approach can also reveal structure in connectivity. We start by setting up a general framework for determining geometry and modularity in activity and connectivity and relating these properties with computations performed by the network. We then use this framework to review the types of structure found in recent studies of model networks performing three classes of computations.

Intrinsic and synaptic determinants of receptive field plasticity in Purkinje cells of the mouse cerebellum

Non-synaptic (intrinsic) plasticity of membrane excitability contributes to aspects of memory formation, but it remains unclear whether it merely facilitates synaptic long-term potentiation or plays a permissive role in determining the impact of synaptic weight increase. We use tactile stimulation and electrical activation of parallel fibers to probe intrinsic and synaptic contributions to receptive field plasticity in awake mice during two-photon calcium imaging of cerebellar Purkinje cells. Repetitive activation of both stimuli induced response potentiation that is impaired in mice with selective deficits in either synaptic or intrinsic plasticity. Spatial analysis of calcium signals demonstrated that intrinsic, but not synaptic plasticity, enhances the spread of dendritic parallel fiber response potentiation. Simultaneous dendrite and axon initial segment recordings confirm these dendritic events affect axonal output. Our findings support the hypothesis that intrinsic plasticity provides an amplification mechanism that exerts a permissive control over the impact of long-term potentiation on neuronal responsiveness.

Intrinsic and Synaptic Contributions to Repetitive Spiking in Dentate Granule Cells

Repetitive firing of granule cells (GCs) in the dentate gyrus (DG) facilitates synaptic transmission to the CA3 region. This facilitation can gate and amplify the flow of information through the hippocampus. High-frequency bursts in the DG are linked to behavior and plasticity, but GCs do not readily burst. Under normal conditions, a single shock to the perforant path in a hippocampal slice typically drives a GC to fire a single spike, and only occasionally more than one spike is seen. Repetitive spiking in GCs is not robust, and the mechanisms are poorly understood. Here, we used a hybrid genetically encoded voltage sensor to image voltage changes evoked by cortical inputs in many mature GCs simultaneously in hippocampal slices from male and female mice. This enabled us to study relatively infrequent double and triple spikes. We found GCs are relatively homogeneous and their double spiking behavior is cell autonomous. Blockade of GABA type A receptors increased multiple spikes and prolonged the interspike interval, indicating inhibitory interneurons limit repetitive spiking and set the time window for successive spikes. Inhibiting synaptic glutamate release showed that recurrent excitation mediated by hilar mossy cells contributes to, but is not necessary for, multiple spiking. Blockade of T-type Ca2+ channels did not reduce multiple spiking but prolonged interspike intervals. Imaging voltage changes in different GC compartments revealed that second spikes can be initiated in either dendrites or somata. Thus, pharmacological and biophysical experiments reveal roles for both synaptic circuitry and intrinsic excitability in GC repetitive spiking.

Local synaptic inhibition mediates cerebellar granule cell pattern separation and enables learned sensorimotor associations

The cerebellar cortex has a key role in generating predictive sensorimotor associations. To do so, the granule cell layer is thought to establish unique sensorimotor representations for learning. However, how this is achieved and how granule cell population responses contribute to behavior have remained unclear. To address these questions, we have used in vivo calcium imaging and granule cell-specific pharmacological manipulation of synaptic inhibition in awake, behaving mice. These experiments indicate that inhibition sparsens and thresholds sensory responses, limiting overlap between sensory ensembles and preventing spiking in many granule cells that receive excitatory input. Moreover, inhibition can be recruited in a stimulus-specific manner to powerfully decorrelate multisensory ensembles. Consistent with these results, granule cell inhibition is required for accurate cerebellum-dependent sensorimotor behavior. These data thus reveal key mechanisms for granule cell layer pattern separation beyond those envisioned by classical models.

Distinguishing examples while building concepts in hippocampal and artificial networks

The hippocampal subfield CA3 is thought to function as an auto-associative network that stores experiences as memories. Information from these experiences arrives directly from the entorhinal cortex as well as indirectly through the dentate gyrus, which performs sparsification and decorrelation. The computational purpose for these dual input pathways has not been firmly established. We model CA3 as a Hopfield-like network that stores both dense, correlated encodings and sparse, decorrelated encodings. As more memories are stored, the former merge along shared features while the latter remain distinct. We verify our model's prediction in rat CA3 place cells, which exhibit more distinct tuning during theta phases with sparser activity. Finally, we find that neural networks trained in multitask learning benefit from a loss term that promotes both correlated and decorrelated representations. Thus, the complementary encodings we have found in CA3 can provide broad computational advantages for solving complex tasks.

The spinal cord facilitates cerebellar upper limb motor learning and control; inputs from neuromusculoskeletal simulation

Complex interactions between brain regions and the spinal cord (SC) govern body motion, which is ultimately driven by muscle activation. Motor planning or learning are mainly conducted at higher brain regions, whilst the SC acts as a brain-muscle gateway and as a motor control centre providing fast reflexes and muscle activity regulation. Thus, higher brain areas need to cope with the SC as an inherent and evolutionary older part of the body dynamics. Here, we address the question of how SC dynamics affects motor learning within the cerebellum; in particular, does the SC facilitate cerebellar motor learning or constitute a biological constraint? We provide an exploratory framework by integrating biologically plausible cerebellar and SC computational models in a musculoskeletal upper limb control loop. The cerebellar model, equipped with the main form of cerebellar plasticity, provides motor adaptation; whilst the SC model implements stretch reflex and reciprocal inhibition between antagonist muscles. The resulting spino-cerebellar model is tested performing a set of upper limb motor tasks, including external perturbation studies. A cerebellar model, lacking the implemented SC model and directly controlling the simulated muscles, was also tested in the same. The performances of the spino-cerebellar and cerebellar models were then compared, thus allowing directly addressing the SC influence on cerebellar motor adaptation and learning, and on handling external motor perturbations. Performance was assessed in both joint and muscle space, and compared with kinematic and EMG recordings from healthy participants. The differences in cerebellar synaptic adaptation between both models were also studied. We conclude that the SC facilitates cerebellar motor learning; when the SC circuits are in the loop, faster convergence in motor learning is achieved with simpler cerebellar synaptic weight distributions. The SC is also found to improve robustness against external perturbations, by better reproducing and modulating muscle cocontraction patterns.

From lazy to rich to exclusive task representations in neural networks and neural codes

Neural circuits-both in the brain and in "artificial" neural network models-learn to solve a remarkable variety of tasks, and there is a great current opportunity to use neural networks as models for brain function. Key to this endeavor is the ability to characterize the representations formed by both artificial and biological brains. Here, we investigate this potential through the lens of recently developing theory that characterizes neural networks as "lazy" or "rich" depending on the approach they use to solve tasks: lazy networks solve tasks by making small changes in connectivity, while rich networks solve tasks by significantly modifying weights throughout the network (including "hidden layers"). We further elucidate rich networks through the lens of compression and "neural collapse", ideas that have recently been of significant interest to neuroscience and machine learning. We then show how these ideas apply to a domain of increasing importance to both fields: extracting latent structures through self-supervised learning.

Hopfield-like network with complementary encodings of memories

We present a Hopfield-like autoassociative network for memories representing examples of concepts. Each memory is encoded by two activity patterns with complementary properties. The first is dense and correlated across examples within concepts, and the second is sparse and exhibits no correlation among examples. The network stores each memory as a linear combination of its encodings. During retrieval, the network recovers sparse or dense patterns with a high or low activity threshold, respectively. As more memories are stored, the dense representation at low threshold shifts from examples to concepts, which are learned from accumulating common example features. Meanwhile, the sparse representation at high threshold maintains distinctions between examples due to the high capacity of sparse, decorrelated patterns. Thus, a single network can retrieve memories at both example and concept scales and perform heteroassociation between them. We obtain our results by deriving macroscopic mean-field equations that yield capacity formulas for sparse examples, dense examples, and dense concepts. We also perform simulations that verify our theoretical results and explicitly demonstrate the capabilities of the network.

Recent data on the cerebellum require new models and theories

The cerebellum has been a popular topic for theoretical studies because its structure was thought to be simple. Since David Marr and James Albus related its function to motor skill learning and proposed the Marr-Albus cerebellar learning model, this theory has guided and inspired cerebellar research. In this review, we summarize the theoretical progress that has been made within this framework of error-based supervised learning. We discuss the experimental progress that demonstrates more complicated molecular and cellular mechanisms in the cerebellum as well as new cell types and recurrent connections. We also cover its involvement in diverse non-motor functions and evidence of other forms of learning. Finally, we highlight the need to explain these new experimental findings into an integrated cerebellar model that can unify its diverse computational functions.

Phase information is conserved in sparse, synchronous population-rate-codes via phase-to-rate recoding

Neural computation is often traced in terms of either rate- or phase-codes. However, most circuit operations will simultaneously affect information across both coding schemes. It remains unclear how phase and rate coded information is transmitted, in the face of continuous modification at consecutive processing stages. Here, we study this question in the entorhinal cortex (EC)- dentate gyrus (DG)- CA3 system using three distinct computational models. We demonstrate that DG feedback inhibition leverages EC phase information to improve rate-coding, a computation we term phase-to-rate recoding. Our results suggest that it i) supports the conservation of phase information within sparse rate-codes and ii) enhances the efficiency of plasticity in downstream CA3 via increased synchrony. Given the ubiquity of both phase-coding and feedback circuits, our results raise the question whether phase-to-rate recoding is a recurring computational motif, which supports the generation of sparse, synchronous population-rate-codes in areas beyond the DG.

Task-dependent optimal representations for cerebellar learning

The cerebellar granule cell layer has inspired numerous theoretical models of neural representations that support learned behaviors, beginning with the work of Marr and Albus. In these models, granule cells form a sparse, combinatorial encoding of diverse sensorimotor inputs. Such sparse representations are optimal for learning to discriminate random stimuli. However, recent observations of dense, low-dimensional activity across granule cells have called into question the role of sparse coding in these neurons. Here, we generalize theories of cerebellar learning to determine the optimal granule cell representation for tasks beyond random stimulus discrimination, including continuous input-output transformations as required for smooth motor control. We show that for such tasks, the optimal granule cell representation is substantially denser than predicted by classical theories. Our results provide a general theory of learning in cerebellum-like systems and suggest that optimal cerebellar representations are task-dependent.

Distinctive properties of biological neural networks and recent advances in bottom-up approaches toward a better biologically plausible neural network

Although it may appear infeasible and impractical, building artificial intelligence (AI) using a bottom-up approach based on the understanding of neuroscience is straightforward. The lack of a generalized governing principle for biological neural networks (BNNs) forces us to address this problem by converting piecemeal information on the diverse features of neurons, synapses, and neural circuits into AI. In this review, we described recent attempts to build a biologically plausible neural network by following neuroscientifically similar strategies of neural network optimization or by implanting the outcome of the optimization, such as the properties of single computational units and the characteristics of the network architecture. In addition, we proposed a formalism of the relationship between the set of objectives that neural networks attempt to achieve, and neural network classes categorized by how closely their architectural features resemble those of BNN. This formalism is expected to define the potential roles of top-down and bottom-up approaches for building a biologically plausible neural network and offer a map helping the navigation of the gap between neuroscience and AI engineering.

Generalized extinction of fear memory depends on co-allocation of synaptic plasticity in dendrites

Memories can be modified by new experience in a specific or generalized manner. Changes in synaptic connections are crucial for memory storage, but it remains unknown how synaptic changes associated with different memories are distributed within neuronal circuits and how such distributions affect specific or generalized modification by novel experience. Here we show that fear conditioning with two different auditory stimuli (CS) and footshocks (US) induces dendritic spine elimination mainly on different dendritic branches of layer 5 pyramidal neurons in the mouse motor cortex. Subsequent fear extinction causes CS-specific spine formation and extinction of freezing behavior. In contrast, spine elimination induced by fear conditioning with >2 different CS-USs often co-exists on the same dendritic branches. Fear extinction induces CS-nonspecific spine formation and generalized fear extinction. Moreover, activation of somatostatin-expressing interneurons increases the occurrence of spine elimination induced by different CS-USs on the same dendritic branches and facilitates the generalization of fear extinction. These findings suggest that specific or generalized modification of existing memories by new experience depends on whether synaptic changes induced by previous experiences are segregated or co-exist at the level of individual dendritic branches.

Structured cerebellar connectivity supports resilient pattern separation

The cerebellum is thought to help detect and correct errors between intended and executed commands1,2 and is critical for social behaviours, cognition and emotion3-6. Computations for motor control must be performed quickly to correct errors in real time and should be sensitive to small differences between patterns for fine error correction while being resilient to noise7. Influential theories of cerebellar information processing have largely assumed random network connectivity, which increases the encoding capacity of the network's first layer8-13. However, maximizing encoding capacity reduces the resilience to noise7. To understand how neuronal circuits address this fundamental trade-off, we mapped the feedforward connectivity in the mouse cerebellar cortex using automated large-scale transmission electron microscopy and convolutional neural network-based image segmentation. We found that both the input and output layers of the circuit exhibit redundant and selective connectivity motifs, which contrast with prevailing models. Numerical simulations suggest that these redundant, non-random connectivity motifs increase the resilience to noise at a negligible cost to the overall encoding capacity. This work reveals how neuronal network structure can support a trade-off between encoding capacity and redundancy, unveiling principles of biological network architecture with implications for the design of artificial neural networks.

An emergent temporal basis set robustly supports cerebellar time-series learning

The cerebellum is considered a "learning machine" essential for time interval estimation underlying motor coordination and other behaviors. Theoretical work has proposed that the cerebellum's input recipient structure, the granule cell layer (GCL), performs pattern separation of inputs that facilitates learning in Purkinje cells (P-cells). However, the relationship between input reformatting and learning has remained debated, with roles emphasized for pattern separation features from sparsification to decorrelation. We took a novel approach by training a minimalist model of the cerebellar cortex to learn complex time-series data from time-varying inputs, typical during movements. The model robustly produced temporal basis sets from these inputs, and the resultant GCL output supported better learning of temporally complex target functions than mossy fibers alone. Learning was optimized at intermediate threshold levels, supporting relatively dense granule cell activity, yet the key statistical features in GCL population activity that drove learning differed from those seen previously for classification tasks. These findings advance testable hypotheses for mechanisms of temporal basis set formation and predict that moderately dense population activity optimizes learning.NEW & NOTEWORTHY During movement, mossy fiber inputs to the cerebellum relay time-varying information with strong intrinsic relationships to ongoing movement. Are such mossy fibers signals sufficient to support Purkinje signals and learning? In a model, we show how the GCL greatly improves Purkinje learning of complex, temporally dynamic signals relative to mossy fibers alone. Learning-optimized GCL population activity was moderately dense, which retained intrinsic input variance while also performing pattern separation.

Synaptic basis of a sub-second representation of time in a neural circuit model

Temporal sequences of neural activity are essential for driving well-timed behaviors, but the underlying cellular and circuit mechanisms remain elusive. We leveraged the well-defined architecture of the cerebellum, a brain region known to support temporally precise actions, to explore theoretically whether the experimentally observed diversity of short-term synaptic plasticity (STP) at the input layer could generate neural dynamics sufficient for sub-second temporal learning. A cerebellar circuit model equipped with dynamic synapses produced a diverse set of transient granule cell firing patterns that provided a temporal basis set for learning precisely timed pauses in Purkinje cell activity during simulated delay eyelid conditioning and Bayesian interval estimation. The learning performance across time intervals was influenced by the temporal bandwidth of the temporal basis, which was determined by the input layer synaptic properties. The ubiquity of STP throughout the brain positions it as a general, tunable cellular mechanism for sculpting neural dynamics and fine-tuning behavior.

Cerebellum as a kernel machine: A novel perspective on expansion recoding in granule cell layer

Sensorimotor information provided by mossy fibers (MF) is mapped to high-dimensional space by a huge number of granule cells (GrC) in the cerebellar cortex's input layer. Significant studies have demonstrated the computational advantages and primary contributor of this expansion recoding. Here, we propose a novel perspective on the expansion recoding where each GrC serve as a kernel basis function, thereby the cerebellum can operate like a kernel machine that implicitly use high dimensional (even infinite) feature spaces. We highlight that the generation of kernel basis function is indeed biologically plausible scenario, considering that the key idea of kernel machine is to memorize important input patterns. We present potential regimes for developing kernels under constrained resources and discuss the advantages and disadvantages of each regime using various simulation settings.

Mesodiencephalic junction GABAergic inputs are processed separately from motor cortical inputs in the basilar pons

The basilar pontine nuclei (bPN) are known to receive excitatory input from the entire neocortex and constitute the main source of mossy fibers to the cerebellum. Various potential inhibitory afferents have been described, but their origin, synaptic plasticity, and network function have remained elusive. Here we identify the mesodiencephalic junction (MDJ) as a prominent source of monosynaptic GABAergic inputs to the bPN. We found no evidence that these inputs converge with motor cortex (M1) inputs at the single neuron or at the local network level. Tracing the inputs to GABAergic MDJ neurons revealed inputs to these neurons from neocortical areas. Additionally, we observed little short-term synaptic facilitation or depression in afferents from the MDJ, enabling MDJ inputs to carry sign-inversed neocortical inputs. Thus, our results show a prominent source of GABAergic inhibition to the bPN that could enrich input to the cerebellar granule cell layer.

Higher-dimensional neural representations predict better episodic memory

Episodic memory enables humans to encode and later vividly retrieve information about our rich experiences, yet the neural representations that support this mental capacity are poorly understood. Using a large fMRI dataset (n = 468) of face-name associative memory tasks and principal component analysis to examine neural representational dimensionality (RD), we found that the human brain maintained a high-dimensional representation of faces through hierarchical representation within and beyond the face-selective regions. Critically, greater RD was associated with better subsequent memory performance both within and across participants, and this association was specific to episodic memory but not general cognitive abilities. Furthermore, the frontoparietal activities could suppress the shared low-dimensional fluctuations and reduce the correlations of local neural responses, resulting in greater RD. RD was not associated with the degree of item-specific pattern similarity, and it made complementary contributions to episodic memory. These results provide a mechanistic understanding of the role of RD in supporting accurate episodic memory.

A Nonlinear Hidden Layer Enables Actor-Critic Agents to Learn Multiple Paired Association Navigation

Navigation to multiple cued reward locations has been increasingly used to study rodent learning. Though deep reinforcement learning agents have been shown to be able to learn the task, they are not biologically plausible. Biologically plausible classic actor-critic agents have been shown to learn to navigate to single reward locations, but which biologically plausible agents are able to learn multiple cue-reward location tasks has remained unclear. In this computational study, we show versions of classic agents that learn to navigate to a single reward location, and adapt to reward location displacement, but are not able to learn multiple paired association navigation. The limitation is overcome by an agent in which place cell and cue information are first processed by a feedforward nonlinear hidden layer with synapses to the actor and critic subject to temporal difference error-modulated plasticity. Faster learning is obtained when the feedforward layer is replaced by a recurrent reservoir network.

The anterior paired lateral neuron normalizes odour-evoked activity in the Drosophila mushroom body calyx

To identify and memorize discrete but similar environmental inputs, the brain needs to distinguish between subtle differences of activity patterns in defined neuronal populations. The Kenyon cells (KCs) of the Drosophila adult mushroom body (MB) respond sparsely to complex olfactory input, a property that is thought to support stimuli discrimination in the MB. To understand how this property emerges, we investigated the role of the inhibitory anterior paired lateral (APL) neuron in the input circuit of the MB, the calyx. Within the calyx, presynaptic boutons of projection neurons (PNs) form large synaptic microglomeruli (MGs) with dendrites of postsynaptic KCs. Combining electron microscopy (EM) data analysis and in vivo calcium imaging, we show that APL, via inhibitory and reciprocal synapses targeting both PN boutons and KC dendrites, normalizes odour-evoked representations in MGs of the calyx. APL response scales with the PN input strength and is regionalized around PN input distribution. Our data indicate that the formation of a sparse code by the KCs requires APL-driven normalization of their MG postsynaptic responses. This work provides experimental insights on how inhibition shapes sensory information representation in a higher brain centre, thereby supporting stimuli discrimination and allowing for efficient associative memory formation.

The Shape of Data: a Theory of the Representation of Information in the Cerebellar Cortex

This paper presents a model of rate coding in the cerebellar cortex. The pathway of input to output of the cerebellum forms an anatomically repeating, functionally modular network, whose basic wiring is preserved across vertebrate taxa. Each network is bisected centrally by a functionally defined cell group, a microzone, which forms part of the cerebellar circuit. Input to a network may be from tens of thousands of concurrently active mossy fibres. The model claims to quantify the conversion of input rates into the code received by a microzone. Recoding on entry converts input rates into an internal code which is homogenised in the functional equivalent of an imaginary plane, occupied by the centrally positioned microzone. Homogenised means the code exists in any random sample of parallel fibre signals over a minimum number. The nature of the code and the regimented architecture of the cerebellar cortex mean that the threshold can be represented by space so that the threshold can be met by the physical dimensions of the Purkinje cell dendritic arbour and planar interneuron networks. As a result, the whole population of a microzone receives the same code. This is part of a mechanism which orchestrates functionally indivisible behaviour of the cerebellar circuit and is necessary for coordinated control of the output cells of the circuit. In this model, fine control of Purkinje cells is by input rates to the system and not by learning so that it is in conflict with the for-years-dominant supervised learning model.

How connectivity rules and synaptic properties shape the efficacy of pattern separation in the entorhinal cortex-dentate gyrus-CA3 network

Pattern separation is a fundamental brain computation that converts small differences in input patterns into large differences in output patterns. Several synaptic mechanisms of pattern separation have been proposed, including code expansion, inhibition and plasticity; however, which of these mechanisms play a role in the entorhinal cortex (EC)-dentate gyrus (DG)-CA3 circuit, a classical pattern separation circuit, remains unclear. Here we show that a biologically realistic, full-scale EC-DG-CA3 circuit model, including granule cells (GCs) and parvalbumin-positive inhibitory interneurons (PV+-INs) in the DG, is an efficient pattern separator. Both external gamma-modulated inhibition and internal lateral inhibition mediated by PV+-INs substantially contributed to pattern separation. Both local connectivity and fast signaling at GC-PV+-IN synapses were important for maximum effectiveness. Similarly, mossy fiber synapses with conditional detonator properties contributed to pattern separation. By contrast, perforant path synapses with Hebbian synaptic plasticity and direct EC-CA3 connection shifted the network towards pattern completion. Our results demonstrate that the specific properties of cells and synapses optimize higher-order computations in biological networks and might be useful to improve the deep learning capabilities of technical networks.

Gating by Memory: a Theory of Learning in the Cerebellum

This paper presents a model of learning by the cerebellar circuit. In the traditional and dominant learning model, training teaches finely graded parallel fibre synaptic weights which modify transmission to Purkinje cells and to interneurons that inhibit Purkinje cells. Following training, input in a learned pattern drives a training-modified response. The function is that the naive response to input rates is displaced by a learned one, trained under external supervision. In the proposed model, there is no weight-controlled graduated balance of excitation and inhibition of Purkinje cells. Instead, the balance has two functional states-a switch-at synaptic, whole cell and microzone level. The paper is in two parts. The first is a detailed physiological argument for the synaptic learning function. The second uses the function in a computational simulation of pattern memory. Against expectation, this generates a predictable outcome from input chaos (real-world variables). Training always forces synaptic weights away from the middle and towards the limits of the range, causing them to polarise, so that transmission is either robust or blocked. All conditions teach the same outcome, such that all learned patterns receive the same, rather than a bespoke, effect on transmission. In this model, the function of learning is gating-that is, to select patterns that trigger output merely, and not to modify output. The outcome is memory-operated gate activation which operates a two-state balance of weight-controlled transmission. Group activity of parallel fibres also simultaneously contains a second code contained in collective rates, which varies independently of the pattern code. A two-state response to the pattern code allows faithful, and graduated, control of Purkinje cell firing by the rate code, at gated times.

Developmental emergence of two-stage nonlinear synaptic integration in cerebellar interneurons

Synaptic transmission, connectivity, and dendritic morphology mature in parallel during brain development and are often disrupted in neurodevelopmental disorders. Yet how these changes influence the neuronal computations necessary for normal brain function are not well understood. To identify cellular mechanisms underlying the maturation of synaptic integration in interneurons, we combined patch-clamp recordings of excitatory inputs in mouse cerebellar stellate cells (SCs), three-dimensional reconstruction of SC morphology with excitatory synapse location, and biophysical modeling. We found that postnatal maturation of postsynaptic strength was homogeneously reduced along the somatodendritic axis, but dendritic integration was always sublinear. However, dendritic branching increased without changes in synapse density, leading to a substantial gain in distal inputs. Thus, changes in synapse distribution, rather than dendrite cable properties, are the dominant mechanism underlying the maturation of neuronal computation. These mechanisms favor the emergence of a spatially compartmentalized two-stage integration model promoting location-dependent integration within dendritic subunits.

Neural mechanism mimetic selective electronic nose based on programmed M13 bacteriophage

The electronic nose is a reliable practical sensor device that mimics olfactory organs. Although numerous studies have demonstrated excellence in detecting various target substances with the help of ideal models, biomimetic approaches still suffer in practical realization because of the inability to mimic the signal processing performed by olfactory neural systems. Herein, we propose an electronic nose based on the programable surface chemistry of M13 bacteriophage, inspired by the neural mechanism of the mammalian olfactory system. The neural pattern separation (NPS) was devised to apply the pattern separation that operates in the memory and learning process of the brain to the electronic nose. We demonstrate an electronic nose in a portable device form, distinguishing polycyclic aromatic compounds (harmful in living environment) in an atomic-level resolution (97.5% selectivity rate) for the first time. Our results provide practical methodology and inspiration for the second-generation electronic nose development toward the performance of detection dogs (K9).

Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity

The ongoing exponential rise in recording capacity calls for new approaches for analysing and interpreting neural data. Effective dimensionality has emerged as an important property of neural activity across populations of neurons, yet different studies rely on different definitions and interpretations of this quantity. Here, we focus on intrinsic and embedding dimensionality, and discuss how they might reveal computational principles from data. Reviewing recent works, we propose that the intrinsic dimensionality reflects information about the latent variables encoded in collective activity while embedding dimensionality reveals the manner in which this information is processed. We conclude by highlighting the role of network models as an ideal substrate for testing more specifically various hypotheses on the computational principles reflected through intrinsic and embedding dimensionality.

Chronic Ethanol Exposure Enhances Facial Stimulation-Evoked Mossy Fiber-Granule Cell Synaptic Transmission via GluN2A Receptors in the Mouse Cerebellar Cortex

Sensory information is transferred to the cerebellar cortex via the mossy fiber–granule cell (MF–GC) pathway, which participates in motor coordination and motor learning. We previously reported that chronic ethanol exposure from adolescence facilitated the sensory-evoked molecular layer interneuron–Purkinje cell synaptic transmission in adult mice in vivo. Herein, we investigated the effect of chronic ethanol exposure from adolescence on facial stimulation-evoked MF–GC synaptic transmission in the adult mouse cerebellar cortex using electrophysiological recording techniques and pharmacological methods. Chronic ethanol exposure from adolescence induced an enhancement of facial stimulation-evoked MF–GC synaptic transmission in the cerebellar cortex of adult mice. The application of an N-methyl-D-aspartate receptor (NMDAR) antagonist, D-APV (250 μM), induced stronger depression of facial stimulation-evoked MF–GC synaptic transmission in chronic ethanol-exposed mice compared with that in control mice. Chronic ethanol exposure-induced facilitation of facial stimulation evoked by MF–GC synaptic transmission was abolished by a selective GluN2A antagonist, PEAQX (10 μM), but was unaffected by the application of a selective GluN2B antagonist, TCN-237 (10 μM), or a type 1 metabotropic glutamate receptor blocker, JNJ16259685 (10 μM). These results indicate that chronic ethanol exposure from adolescence enhances facial stimulation-evoked MF–GC synaptic transmission via GluN2A, which suggests that chronic ethanol exposure from adolescence impairs the high-fidelity transmission capability of sensory information in the cerebellar cortex by enhancing the NMDAR-mediated components of MF–GC synaptic transmission in adult mice in vivo.

Cerebellar granule cell axons support high-dimensional representations

In classical theories of cerebellar cortex, high-dimensional sensorimotor representations are used to separate neuronal activity patterns, improving associative learning and motor performance. Recent experimental studies suggest that cerebellar granule cell (GrC) population activity is low-dimensional. To examine sensorimotor representations from the point of view of downstream Purkinje cell 'decoders', we used three-dimensional acousto-optic lens two-photon microscopy to record from hundreds of GrC axons. Here we show that GrC axon population activity is high dimensional and distributed with little fine-scale spatial structure during spontaneous behaviors. Moreover, distinct behavioral states are represented along orthogonal dimensions in neuronal activity space. These results suggest that the cerebellar cortex supports high-dimensional representations and segregates behavioral state-dependent computations into orthogonal subspaces, as reported in the neocortex. Our findings match the predictions of cerebellar pattern separation theories and suggest that the cerebellum and neocortex use population codes with common features, despite their vastly different circuit structures.

GluA4 facilitates cerebellar expansion coding and enables associative memory formation

AMPA receptors (AMPARs) mediate excitatory neurotransmission in the central nervous system (CNS) and their subunit composition determines synaptic efficacy. Whereas AMPAR subunits GluA1–GluA3 have been linked to particular forms of synaptic plasticity and learning, the functional role of GluA4 remains elusive. Here, we demonstrate a crucial function of GluA4 for synaptic excitation and associative memory formation in the cerebellum. Notably, GluA4-knockout mice had ~80% reduced mossy fiber to granule cell synaptic transmission. The fidelity of granule cell spike output was markedly decreased despite attenuated tonic inhibition and increased NMDA receptor-mediated transmission. Computational network modeling incorporating these changes revealed that deletion of GluA4 impairs granule cell expansion coding, which is important for pattern separation and associative learning. On a behavioral level, while locomotor coordination was generally spared, GluA4-knockout mice failed to form associative memories during delay eyeblink conditioning. These results demonstrate an essential role for GluA4-containing AMPARs in cerebellar information processing and associative learning.

Multidimensional population activity in an electrically coupled inhibitory circuit in the cerebellar cortex

Inhibitory neurons orchestrate the activity of excitatory neurons and play key roles in circuit function. Although individual interneurons have been studied extensively, little is known about their properties at the population level. Using random-access 3D two-photon microscopy, we imaged local populations of cerebellar Golgi cells (GoCs), which deliver inhibition to granule cells. We show that population activity is organized into multiple modes during spontaneous behaviors. A slow, network-wide common modulation of GoC activity correlates with the level of whisking and locomotion, while faster (<1 s) differential population activity, arising from spatially mixed heterogeneous GoC responses, encodes more precise information. A biologically detailed GoC circuit model reproduced the common population mode and the dimensionality observed experimentally, but these properties disappeared when electrical coupling was removed. Our results establish that local GoC circuits exhibit multidimensional activity patterns that could be used for inhibition-mediated adaptive gain control and spatiotemporal patterning of downstream granule cells.

Autoencoder networks extract latent variables and encode these variables in their connectomes

Advances in electron microscopy and data processing techniques are leading to increasingly large and complete microscale connectomes. At the same time, advances in artificial neural networks have produced model systems that perform comparably rich computations with perfectly specified connectivity. This raises an exciting scientific opportunity for the study of both biological and artificial neural networks: to infer the underlying circuit function from the structure of its connectivity. A potential roadblock, however, is that - even with well constrained neural dynamics - there are in principle many different connectomes that could support a given computation. Here, we define a tractable setting in which the problem of inferring circuit function from circuit connectivity can be analyzed in detail: the function of input compression and reconstruction, in an autoencoder network with a single hidden layer. Here, in general there is substantial ambiguity in the weights that can produce the same circuit function, because largely arbitrary changes to input weights can be undone by applying the inverse modifications to the output weights. However, we use mathematical arguments and simulations to show that adding simple, biologically motivated regularization of connectivity resolves this ambiguity in an interesting way: weights are constrained such that the latent variable structure underlying the inputs can be extracted from the weights by using nonlinear dimensionality reduction methods.

Frozen algorithms: how the brain's wiring facilitates learning

Synapses and neural connectivity are plastic and shaped by experience. But to what extent does connectivity itself influence the ability of a neural circuit to learn? Insights from optimization theory and AI shed light on how learning can be implemented in neural circuits. Though abstract in their nature, learning algorithms provide a principled set of hypotheses on the necessary ingredients for learning in neural circuits. These include the kinds of signals and circuit motifs that enable learning from experience, as well as an appreciation of the constraints that make learning challenging in a biological setting. Remarkably, some simple connectivity patterns can boost the efficiency of relatively crude learning rules, showing how the brain can use anatomy to compensate for the biological constraints of known synaptic plasticity mechanisms. Modern connectomics provides rich data for exploring this principle, and may reveal how brain connectivity is constrained by the requirement to learn efficiently.