Conditional modulation of spike-timing-dependent plasticity for olfactory learning

Eligibility traces as a synaptic substrate for learning

Animals can learn to associate a behavior or a stimulus with a delayed reward, this is essential for survival. A mechanism proposed for bridging this gap are synaptic eligibility traces, which are slowly decaying tags, which can lead to synaptic plasticity if followed by rewards. Recently, experiments have demonstrated the existence of synaptic eligibility traces in diverse neural systems, depending on either neuromodulators or plateau potentials. Evidence for both eligibility trace-dependent potentiation and depression of synaptic efficacies has emerged. We discuss the commonalities and differences of these different results. We show why the existence of both potentiation and depression is important because these opposing forces can lead to a synaptic stopping rule. Without a stopping rule, synapses would saturate at their upper bound thus leading to a loss of selectivity and representational power. We discuss the possible underlying mechanisms of the eligibility traces as well as their functional and theoretical significance.

Astrocyte D1/D5 Dopamine Receptors Govern Non-Hebbian Long-Term Potentiation at Sensory Synapses onto Lamina I Spinoparabrachial Neurons

Recent work demonstrated that activation of spinal D1 and D5 dopamine receptors (D1/D5Rs) facilitates non-Hebbian long-term potentiation (LTP) at primary afferent synapses onto spinal projection neurons. However, the cellular localization of the D1/D5Rs driving non-Hebbian LTP in spinal nociceptive circuits remains unknown, and it is also unclear whether D1/D5R signaling must occur concurrently with sensory input in order to promote non-Hebbian LTP at these synapses. Here we investigate these issues using cell-type-selective knockdown of D1Rs or D5Rs from lamina I spinoparabrachial neurons, dorsal root ganglion (DRG) neurons, or astrocytes in adult mice of either sex using Cre recombinase-based genetic strategies. The LTP evoked by low-frequency stimulation of primary afferents in the presence of the selective D1/D5R agonist SKF82958 persisted following the knockdown of D1R or D5R in spinoparabrachial neurons, suggesting that postsynaptic D1/D5R signaling was dispensable for non-Hebbian plasticity at sensory synapses onto these key output neurons of the superficial dorsal horn (SDH). Similarly, the knockdown of D1Rs or D5Rs in DRG neurons failed to influence SKF82958-enabled LTP in lamina I projection neurons. In contrast, SKF82958-induced LTP was suppressed by the knockdown of D1R or D5R in spinal astrocytes. Furthermore, the data indicate that the activation of D1R/D5Rs in spinal astrocytes can either retroactively or proactively drive non-Hebbian LTP in spinoparabrachial neurons. Collectively, these results suggest that dopaminergic signaling in astrocytes can strongly promote activity-dependent LTP in the SDH, which is predicted to significantly enhance the amplification of ascending nociceptive transmission from the spinal cord to the brain.

Ring-shaped odor coding in the antennal lobe of migratory locusts

The representation of odors in the locust antennal lobe with its >2,000 glomeruli has long remained a perplexing puzzle. We employed the CRISPR-Cas9 system to generate transgenic locusts expressing the genetically encoded calcium indicator GCaMP in olfactory sensory neurons. Using two-photon functional imaging, we mapped the spatial activation patterns representing a wide range of ecologically relevant odors across all six developmental stages. Our findings reveal a functionally ring-shaped organization of the antennal lobe composed of specific glomerular clusters. This configuration establishes an odor-specific chemotopic representation by encoding different chemical classes and ecologically distinct odors in the form of glomerular rings. The ring-shaped glomerular arrangement, which we confirm by selective targeting of OR70a-expressing sensory neurons, occurs throughout development, and the odor-coding pattern within the glomerular population is consistent across developmental stages. Mechanistically, this unconventional spatial olfactory code reflects the locust-specific and multiplexed glomerular innervation pattern of the antennal lobe.

Beyond prediction error: 25 years of modeling the associations formed in the insect mushroom body

The insect mushroom body has gained increasing attention as a system in which the computational basis of neural learning circuits can be unraveled. We now understand in detail the key locations in this circuit where synaptic associations are formed between sensory patterns and values leading to actions. However, the actual learning rule (or rules) implemented by neural activity and leading to synaptic change is still an open question. Here, I survey the diversity of answers that have been offered in computational models of this system over the past decades, including the recurring assumption-in line with top-down theories of associative learning-that the core function is to reduce prediction error. However, I will argue, a more bottom-up approach may ultimately reveal a richer algorithmic capacity in this still enigmatic brain neuropil.

Sensory encoding and memory in the mushroom body: signals, noise, and variability

To survive in changing environments, animals need to learn to associate specific sensory stimuli with positive or negative valence. How do they form stimulus-specific memories to distinguish between positively/negatively associated stimuli and other irrelevant stimuli? Solving this task is one of the functions of the mushroom body, the associative memory center in insect brains. Here we summarize recent work on sensory encoding and memory in the Drosophila mushroom body, highlighting general principles such as pattern separation, sparse coding, noise and variability, coincidence detection, and spatially localized neuromodulation, and placing the mushroom body in comparative perspective with mammalian memory systems.

A biochemical description of postsynaptic plasticity-with timescales ranging from milliseconds to seconds

Synaptic plasticity [long-term potentiation/depression (LTP/D)], is a cellular mechanism underlying learning. Two distinct types of early LTP/D (E-LTP/D), acting on very different time scales, have been observed experimentally-spike timing dependent plasticity (STDP), on time scales of tens of ms; and behavioral time scale synaptic plasticity (BTSP), on time scales of seconds. BTSP is a candidate for a mechanism underlying rapid learning of spatial location by place cells. Here, a computational model of the induction of E-LTP/D at a spine head of a synapse of a hippocampal pyramidal neuron is developed. The single-compartment model represents two interacting biochemical pathways for the activation (phosphorylation) of the kinase (CaMKII) with a phosphatase, with ion inflow through channels (NMDAR, CaV1,Na). The biochemical reactions are represented by a deterministic system of differential equations, with a detailed description of the activation of CaMKII that includes the opening of the compact state of CaMKII. This single model captures realistic responses (temporal profiles with the differing timescales) of STDP and BTSP and their asymmetries. The simulations distinguish several mechanisms underlying STDP vs. BTSP, including i) the flow of [Formula: see text] through NMDAR vs. CaV1 channels, and ii) the origin of several time scales in the activation of CaMKII. The model also realizes a priming mechanism for E-LTP that is induced by [Formula: see text] flow through CaV1.3 channels. Once in the spine head, this small additional [Formula: see text] opens the compact state of CaMKII, placing CaMKII ready for subsequent induction of LTP.

Using rodent data to elucidate dopaminergic mechanisms of ADHD: Implications for human personality

An altered behavioral response to positive reinforcement has been proposed to be a core deficit in attention deficit hyperactivity disorder (ADHD). The spontaneously hypertensive rat (SHR), a congenic animal strain, displays a similarly altered response to reinforcement. The presence of this genetically determined phenotype in a rodent model allows experimental investigation of underlying neural mechanisms. Behaviorally, the SHR displays increased preference for immediate reinforcement, increased sensitivity to individual instances of reinforcement relative to integrated reinforcement history, and a steeper delay of reinforcement gradient compared to other rat strains. The SHR also shows less development of incentive to approach sensory stimuli, or cues, that predict reward after repeated cue-reward pairing. We consider the underlying neural mechanisms for these characteristics. It is well known that midbrain dopamine neurons are initially activated by unexpected reward and gradually transfer their responses to reward-predicting cues. This finding has inspired the dopamine transfer deficit (DTD) hypothesis, which predicts certain behavioral effects that would arise from a deficient transfer of dopamine responses from actual rewards to reward-predicting cues. We argue that the DTD predicts the altered responses to reinforcement seen in the SHR and individuals with ADHD. These altered responses to reinforcement in turn predict core symptoms of ADHD. We also suggest that variations in the degree of dopamine transfer may underlie variations in personality dimensions related to altered reinforcement sensitivity. In doing so, we highlight the value of rodent models to the study of human personality.

Wired together, change together: Spike timing modifies transmission in converging assemblies

The precise timing of neuronal spikes may lead to changes in synaptic connectivity and is thought to be crucial for learning and memory. However, the effect of spike timing on neuronal connectivity in the intact brain remains unknown. Using closed-loop optogenetic stimulation in CA1 of freely moving mice, we generated unique spike patterns between presynaptic pyramidal cells (PYRs) and postsynaptic parvalbumin (PV)-immunoreactive cells. The stimulation led to spike transmission changes that occurred together across all presynaptic PYRs connected to the same postsynaptic PV cell. The precise timing of all presynaptic and postsynaptic cell spikes affected transmission changes. These findings reveal an unexpected plasticity mechanism, in which the spike timing of an entire cell assembly has a more substantial impact on effective connectivity than that of individual cell pairs.

Neural manifolds for odor-driven innate and acquired appetitive preferences

Sensory stimuli evoke spiking neural responses that innately or after learning drive suitable behavioral outputs. How are these spiking activities intrinsically patterned to encode for innate preferences, and could the neural response organization impose constraints on learning? We examined this issue in the locust olfactory system. Using a diverse odor panel, we found that ensemble activities both during ('ON response') and after stimulus presentations ('OFF response') could be linearly mapped onto overall appetitive preference indices. Although diverse, ON and OFF response patterns generated by innately appetitive odorants (higher palp-opening responses) were still limited to a low-dimensional subspace (a 'neural manifold'). Similarly, innately non-appetitive odorants evoked responses that were separable yet confined to another neural manifold. Notably, only odorants that evoked neural response excursions in the appetitive manifold could be associated with gustatory reward. In sum, these results provide insights into how encoding for innate preferences can also impact associative learning.

Harnessing insect olfactory neural circuits for detecting and discriminating human cancers

There is overwhelming evidence that presence of cancer alters cellular metabolic processes, and these changes are manifested in emitted volatile organic compound (VOC) compositions of cancer cells. Here, we take a novel forward engineering approach by developing an insect olfactory neural circuit-based VOC sensor for cancer detection. We obtained oral cancer cell culture VOC-evoked extracellular neural responses from in vivo insect (locust) antennal lobe neurons. We employed biological neural computations of the antennal lobe circuitry for generating spatiotemporal neuronal response templates corresponding to each cell culture VOC mixture, and employed these neuronal templates to distinguish oral cancer cell lines (SAS, Ca9-22, and HSC-3) vs. a non-cancer cell line (HaCaT). Our results demonstrate that three different human oral cancers can be robustly distinguished from each other and from a non-cancer oral cell line. By using high-dimensional population neuronal response analysis and leave-one-trial-out methodology, our approach yielded high classification success for each cell line tested. Our analyses achieved 76-100% success in identifying cell lines by using the population neural response (n = 194) collected for the entire duration of the cell culture study. We also demonstrate this cancer detection technique can distinguish between different types of oral cancers and non-cancer at different time-matched points of growth. This brain-based cancer detection approach is fast as it can differentiate between VOC mixtures within 250 ms of stimulus onset. Our brain-based cancer detection system comprises a novel VOC sensing methodology that incorporates entire biological chemosensory arrays, biological signal transduction, and neuronal computations in a form of a forward-engineered technology for cancer VOC detection.

Sleep prevents catastrophic forgetting in spiking neural networks by forming a joint synaptic weight representation

Artificial neural networks overwrite previously learned tasks when trained sequentially, a phenomenon known as catastrophic forgetting. In contrast, the brain learns continuously, and typically learns best when new training is interleaved with periods of sleep for memory consolidation. Here we used spiking network to study mechanisms behind catastrophic forgetting and the role of sleep in preventing it. The network could be trained to learn a complex foraging task but exhibited catastrophic forgetting when trained sequentially on different tasks. In synaptic weight space, new task training moved the synaptic weight configuration away from the manifold representing old task leading to forgetting. Interleaving new task training with periods of off-line reactivation, mimicking biological sleep, mitigated catastrophic forgetting by constraining the network synaptic weight state to the previously learned manifold, while allowing the weight configuration to converge towards the intersection of the manifolds representing old and new tasks. The study reveals a possible strategy of synaptic weights dynamics the brain applies during sleep to prevent forgetting and optimize learning.

Neurotensin orchestrates valence assignment in the amygdala

The ability to associate temporally segregated information and assign positive or negative valence to environmental cues is paramount for survival. Studies have shown that different projections from the basolateral amygdala (BLA) are potentiated following reward or punishment learning1-7. However, we do not yet understand how valence-specific information is routed to the BLA neurons with the appropriate downstream projections, nor do we understand how to reconcile the sub-second timescales of synaptic plasticity8-11 with the longer timescales separating the predictive cues from their outcomes. Here we demonstrate that neurotensin (NT)-expressing neurons in the paraventricular nucleus of the thalamus (PVT) projecting to the BLA (PVT-BLA:NT) mediate valence assignment by exerting NT concentration-dependent modulation in BLA during associative learning. We found that optogenetic activation of the PVT-BLA:NT projection promotes reward learning, whereas PVT-BLA projection-specific knockout of the NT gene (Nts) augments punishment learning. Using genetically encoded calcium and NT sensors, we further revealed that both calcium dynamics within the PVT-BLA:NT projection and NT concentrations in the BLA are enhanced after reward learning and reduced after punishment learning. Finally, we showed that CRISPR-mediated knockout of the Nts gene in the PVT-BLA pathway blunts BLA neural dynamics and attenuates the preference for active behavioural strategies to reward and punishment predictive cues. In sum, we have identified NT as a neuropeptide that signals valence in the BLA, and showed that NT is a critical neuromodulator that orchestrates positive and negative valence assignment in amygdala neurons by extending valence-specific plasticity to behaviourally relevant timescales.

Norepinephrine potentiates and serotonin depresses visual cortical responses by transforming eligibility traces

Reinforcement allows organisms to learn which stimuli predict subsequent biological relevance. Hebbian mechanisms of synaptic plasticity are insufficient to account for reinforced learning because neuromodulators signaling biological relevance are delayed with respect to the neural activity associated with the stimulus. A theoretical solution is the concept of eligibility traces (eTraces), silent synaptic processes elicited by activity which upon arrival of a neuromodulator are converted into a lasting change in synaptic strength. Previously we demonstrated in visual cortical slices the Hebbian induction of eTraces and their conversion into LTP and LTD by the retroactive action of norepinephrine and serotonin Here we show in vivo in mouse V1 that the induction of eTraces and their conversion to LTP/D by norepinephrine and serotonin respectively potentiates and depresses visual responses. We also show that the integrity of this process is crucial for ocular dominance plasticity, a canonical model of experience-dependent plasticity.

Dopamine and serotonin interplay for valence-based spatial learning

Dopamine (DA) and serotonin (5-HT) are important neuromodulators of synaptic plasticity that have been linked to learning from positive or negative outcomes or valence-based learning. In the hippocampus, both affect long-term plasticity but play different roles in encoding uncertainty or predicted reward. DA has been related to positive valence, from reward consumption or avoidance behavior, and 5-HT to aversive encoding. We propose DA produces overall LTP while 5-HT elicits LTD. Here, we compare two reward-modulated spike timing-dependent plasticity (R-STDP) rules to describe the action of these neuromodulators. We examined their role in cognitive performance and flexibility for computational models of the Morris water maze task and reversal learning. Our results show that the interplay of DA and 5-HT improves learning performance and can explain experimental evidence. This study reinforces the importance of neuromodulation in determining the direction of plasticity.

Fast and slow feedforward inhibitory circuits for cortical odor processing

Feedforward inhibitory circuits are key contributors to the complex interplay between excitation and inhibition in the brain. Little is known about the function of feedforward inhibition in the primary olfactory (piriform) cortex. Using in vivo two-photon-targeted patch clamping and calcium imaging in mice, we find that odors evoke strong excitation in two classes of interneurons – neurogliaform (NG) cells and horizontal (HZ) cells – that provide feedforward inhibition in layer 1 of the piriform cortex. NG cells fire much earlier than HZ cells following odor onset, a difference that can be attributed to the faster odor-driven excitatory synaptic drive that NG cells receive from the olfactory bulb. As a result, NG cells strongly but transiently inhibit odor-evoked excitation in layer 2 principal cells, whereas HZ cells provide more diffuse and prolonged feedforward inhibition. Our findings reveal unexpected complexity in the operation of inhibition in the piriform cortex.

Integrated In-Sensor Computing Optoelectronic Device for Environment-Adaptable Artificial Retina Perception Application

With the development and application of artificial intelligence, there is an appeal to the exploitation of various sensors and memories. As the most important perception of human beings, vision occupies more than 80% of all the received information. Inspired by biological eyes, an artificial retina based on 2D Janus MoSSe was fabricated, which could simulate functions of visual perception with electronic/ion and optical comodulation. Furthermore, inspired by human brain, sensing, memory, and neuromorphic computing functions were integrated on one device for multifunctional intelligent electronics, which was beneficial for scalability and high efficiency. Through the formation of faradic electric double layer (EDL) at the metal-oxide/electrolyte interfaces could realize synaptic weight changes. On the basis of the optoelectronic performances, light adaptation of biological eyes, preprocessing, and recognition of handwritten digits were implemented successfully. This work may provide a strategy for the future integrated sensing-memory-processing device for optoelectronic artificial retina perception application.

Long anisotropic absolute refractory periods with rapid rise times to reliable responsiveness

Refractoriness is a fundamental property of excitable elements, such as neurons, indicating the probability for re-excitation in a given time lag, and is typically linked to the neuronal hyperpolarization following an evoked spike. Here we measured the refractory periods (RPs) in neuronal cultures and observed that an average anisotropic absolute RP could exceed 10 ms and its tail is 20 ms, independent of a large stimulation frequency range. It is an order of magnitude longer than anticipated and comparable with the decaying membrane potential time scale. It is followed by a sharp rise-time (relative RP) of merely ∼1 md to complete responsiveness. Extracellular stimulations result in longer absolute RPs than solely intracellular ones, and a pair of extracellular stimulations from two different routes exhibits distinct absolute RPs, depending on their order. Our results indicate that a neuron is an accurate excitable element, where the diverse RPs cannot be attributed solely to the soma and imply fast mutual interactions between different stimulation routes and dendrites. Further elucidation of neuronal computational capabilities and their interplay with adaptation mechanisms is warranted.

Calcium-Permeable AMPA Receptors Mediate Timing-Dependent LTP Elicited by Low Repeat Coincident Pre- and Postsynaptic Activity at Schaffer Collateral-CA1 Synapses

High-frequency stimulation induced long-term potentiation (LTP) and low-frequency stimulation induced LTD are considered as cellular models of memory formation. Interestingly, spike timing-dependent plasticity (STDP) can induce equally robust timing-dependent LTP (t-LTP) and t-LTD in response to low frequency repeats of coincident action potential (AP) firing in presynaptic and postsynaptic cells. Commonly, STDP paradigms relying on 25-100 repeats of coincident AP firing are used to elicit t-LTP or t-LTD, but the minimum number of repeats required for successful STDP is barely explored. However, systematic investigation of physiologically relevant low repeat STDP paradigms is of utmost importance to explain learning mechanisms in vivo. Here, we examined low repeat STDP at Schaffer collateral-CA1 synapses by pairing one presynaptic AP with either one postsynaptic AP (1:1 t-LTP), or a burst of 4 APs (1:4 t-LTP) and found 3-6 repeats to be sufficient to elicit t-LTP. 6× 1:1 t-LTP required postsynaptic Ca2+ influx via NMDARs and L-type VGCCs and was mediated by increased presynaptic glutamate release. In contrast, 1:4 t-LTP depended on postsynaptic metabotropic GluRs and ryanodine receptor signaling and was mediated by postsynaptic insertion of AMPA receptors. Unexpectedly, both 6× t-LTP variants were strictly dependent on activation of postsynaptic Ca2+-permeable AMPARs but were differentially regulated by dopamine receptor signaling. Our data show that synaptic changes induced by only 3-6 repeats of mild STDP stimulation occurring in ≤10 s can take place on time scales observed also during single trial learning.

Two-Dimensional Metal-Organic Framework Film for Realizing Optoelectronic Synaptic Plasticity

2D metal-organic framework (MOF) film as the active layer show promising application prospects in various fields including sensors, catalysis, and electronic devices. However, exploring the application of 2D MOF film in the field of artificial synapses has not been implemented yet. In this work, we fabricated a novel 2D MOF film (Cu-THPP, THPP=5,10,15,20-Tetrakis(4-hydroxyphenyl)-21H,23H-porphine), and further used it as an active layer to explore the application in the simulation of human brain synapses. It shows excellent light-stimulated synaptic plasticity properties, and exhibits the foundation function of synapses such as long-term plasticity (LTP), short-term plasticity (STP), and the conversion of STP to LTP. Most critically, the MOF based artificial synaptic device exhibits an excellent stability in atmosphere. This work opens the door for the application of 2D MOF film in the simulation of human brain synapses.

What Behavioral Abilities Emerged at Key Milestones in Human Brain Evolution? 13 Hypotheses on the 600-Million-Year Phylogenetic History of Human Intelligence

This paper presents 13 hypotheses regarding the specific behavioral abilities that emerged at key milestones during the 600-million-year phylogenetic history from early bilaterians to extant humans. The behavioral, intellectual, and cognitive faculties of humans are complex and varied: we have abilities as diverse as map-based navigation, theory of mind, counterfactual learning, episodic memory, and language. But these faculties, which emerge from the complex human brain, are likely to have evolved from simpler prototypes in the simpler brains of our ancestors. Understanding the order in which behavioral abilities evolved can shed light on how and why our brains evolved. To propose these hypotheses, I review the available data from comparative psychology and evolutionary neuroscience.

Neuromodulated Dopamine Plastic Networks for Heterogeneous Transfer Learning with Hebbian Principle

The plastic modifications in synaptic connectivity is primarily from changes triggered by neuromodulated dopamine signals. These activities are controlled by neuromodulation, which is itself under the control of the brain. The subjective brain’s self-modifying abilities play an essential role in learning and adaptation. The artificial neural networks with neuromodulated plasticity are used to implement transfer learning in the image classification domain. In particular, this has application in image detection, image segmentation, and transfer of learning parameters with significant results. This paper proposes a novel approach to enhance transfer learning accuracy in a heterogeneous source and target, using the neuromodulation of the Hebbian learning principle, called NDHTL (Neuromodulated Dopamine Hebbian Transfer Learning). Neuromodulation of plasticity offers a powerful new technique with applications in training neural networks implementing asymmetric backpropagation using Hebbian principles in transfer learning motivated CNNs (Convolutional neural networks). Biologically motivated concomitant learning, where connected brain cells activate positively, enhances the synaptic connection strength between the network neurons. Using the NDHTL algorithm, the percentage of change of the plasticity between the neurons of the CNN layer is directly managed by the dopamine signal’s value. The discriminative nature of transfer learning fits well with the technique. The learned model’s connection weights must adapt to unseen target datasets with the least cost and effort in transfer learning. Using distinctive learning principles such as dopamine Hebbian learning in transfer learning for asymmetric gradient weights update is a novel approach. The paper emphasizes the NDHTL algorithmic technique as synaptic plasticity controlled by dopamine signals in transfer learning to classify images using source-target datasets. The standard transfer learning using gradient backpropagation is a symmetric framework. Experimental results using CIFAR-10 and CIFAR-100 datasets show that the proposed NDHTL algorithm can enhance transfer learning efficiency compared to existing methods.

Integration of sleep homeostasis and navigation in Drosophila

During sleep, the brain undergoes dynamic and structural changes. In Drosophila, such changes have been observed in the central complex, a brain area important for sleep control and navigation. The connectivity of the central complex raises the question about how navigation, and specifically the head direction system, can operate in the face of sleep related plasticity. To address this question, we develop a model that integrates sleep homeostasis and head direction. We show that by introducing plasticity, the head direction system can function in a stable way by balancing plasticity in connected circuits that encode sleep pressure. With increasing sleep pressure, the head direction system nevertheless becomes unstable and a sleep phase with a different plasticity mechanism is introduced to reset network connectivity. The proposed integration of sleep homeostasis and head direction circuits captures features of their neural dynamics observed in flies and mice.

Learning with reinforcement prediction errors in a model of the Drosophila mushroom body

Effective decision making in a changing environment demands that accurate predictions are learned about decision outcomes. In Drosophila, such learning is orchestrated in part by the mushroom body, where dopamine neurons signal reinforcing stimuli to modulate plasticity presynaptic to mushroom body output neurons. Building on previous mushroom body models, in which dopamine neurons signal absolute reinforcement, we propose instead that dopamine neurons signal reinforcement prediction errors by utilising feedback reinforcement predictions from output neurons. We formulate plasticity rules that minimise prediction errors, verify that output neurons learn accurate reinforcement predictions in simulations, and postulate connectivity that explains more physiological observations than an experimentally constrained model. The constrained and augmented models reproduce a broad range of conditioning and blocking experiments, and we demonstrate that the absence of blocking does not imply the absence of prediction error dependent learning. Our results provide five predictions that can be tested using established experimental methods.

Predictive olfactory learning in Drosophila

Olfactory learning and conditioning in the fruit fly is typically modelled by correlation-based associative synaptic plasticity. It was shown that the conditioning of an odor-evoked response by a shock depends on the connections from Kenyon cells (KC) to mushroom body output neurons (MBONs). Although on the behavioral level conditioning is recognized to be predictive, it remains unclear how MBONs form predictions of aversive or appetitive values (valences) of odors on the circuit level. We present behavioral experiments that are not well explained by associative plasticity between conditioned and unconditioned stimuli, and we suggest two alternative models for how predictions can be formed. In error-driven predictive plasticity, dopaminergic neurons (DANs) represent the error between the predictive odor value and the shock strength. In target-driven predictive plasticity, the DANs represent the target for the predictive MBON activity. Predictive plasticity in KC-to-MBON synapses can also explain trace-conditioning, the valence-dependent sign switch in plasticity, and the observed novelty-familiarity representation. The model offers a framework to dissect MBON circuits and interpret DAN activity during olfactory learning.

The connectome of the adult Drosophila mushroom body provides insights into function

Making inferences about the computations performed by neuronal circuits from synapse-level connectivity maps is an emerging opportunity in neuroscience. The mushroom body (MB) is well positioned for developing and testing such an approach due to its conserved neuronal architecture, recently completed dense connectome, and extensive prior experimental studies of its roles in learning, memory, and activity regulation. Here, we identify new components of the MB circuit in Drosophila, including extensive visual input and MB output neurons (MBONs) with direct connections to descending neurons. We find unexpected structure in sensory inputs, in the transfer of information about different sensory modalities to MBONs, and in the modulation of that transfer by dopaminergic neurons (DANs). We provide insights into the circuitry used to integrate MB outputs, connectivity between the MB and the central complex and inputs to DANs, including feedback from MBONs. Our results provide a foundation for further theoretical and experimental work.

Plasticity and modulation of olfactory circuits in insects

Olfactory circuits change structurally and physiologically during development and adult life. This allows insects to respond to olfactory cues in an appropriate and adaptive way according to their physiological and behavioral state, and to adapt to their specific abiotic and biotic natural environment. We highlight here findings on olfactory plasticity and modulation in various model and non-model insects with an emphasis on moths and social Hymenoptera. Different categories of plasticity occur in the olfactory systems of insects. One type relates to the reproductive or feeding state, as well as to adult age. Another type of plasticity is context-dependent and includes influences of the immediate sensory and abiotic environment, but also environmental conditions during postembryonic development, periods of adult behavioral maturation, and short- and long-term sensory experience. Finally, plasticity in olfactory circuits is linked to associative learning and memory formation. The vast majority of the available literature summarized here deals with plasticity in primary and secondary olfactory brain centers, but also peripheral modulation is treated. The described molecular, physiological, and structural neuronal changes occur under the influence of neuromodulators such as biogenic amines, neuropeptides, and hormones, but the mechanisms through which they act are only beginning to be analyzed.

Active maintenance of eligibility trace in rodent prefrontal cortex

Even though persistent neural activity has been proposed as a mechanism for maintaining eligibility trace, direct empirical evidence for active maintenance of eligibility trace has been lacking. We recorded neuronal activity in the medial prefrontal cortex (mPFC) in rats performing a dynamic foraging task in which a choice must be remembered until its outcome on the timescale of seconds for correct credit assignment. We found that mPFC neurons maintain significant choice signals during the time period between action selection and choice outcome. We also found that neural signals for choice, outcome, and action value converge in the mPFC when choice outcome was revealed. Our results indicate that the mPFC maintains choice signals necessary for temporal credit assignment in the form of persistent neural activity in our task. They also suggest that the mPFC might update action value by combining actively maintained eligibility trace with action value and outcome signals.

In vivo odorant input induces distinct synaptic plasticity of GABAergic synapses in developing zebrafish olfactory bulb

In the first station of central odor processing, the main olfactory bulb, signal processing is regulated by synaptic interactions between glutamatergic and GABAergic inputs of the mitral cells (MCs), the major projection neurons. Our previous study has found that repetitive postsynaptic spiking within a critical time window after presynaptic activation by natural odorant stimulation results in persistent enhancement of glutamatergic inputs of MCs in larval zebrafish. Here we observed a long-term depression of GABAergic synapses induced by the same protocol. This long-term depression was mediated by presynaptic NMDA receptors (NMDARs). Further dissecting GABAergic neurotransmission revealed that the STDP-induction protocol induced persistent modification in recurrent and lateral inhibition with opposite directions and distinct requirements on NMDARs. Thus, at the plasticity level, different types of GABAergic inhibition may utilize different mechanisms to cooperate or compete with excitatory inputs to optimize patterns of olfactory bulb output.

A solution to the learning dilemma for recurrent networks of spiking neurons

Recurrently connected networks of spiking neurons underlie the astounding information processing capabilities of the brain. Yet in spite of extensive research, how they can learn through synaptic plasticity to carry out complex network computations remains unclear. We argue that two pieces of this puzzle were provided by experimental data from neuroscience. A mathematical result tells us how these pieces need to be combined to enable biologically plausible online network learning through gradient descent, in particular deep reinforcement learning. This learning method-called e-prop-approaches the performance of backpropagation through time (BPTT), the best-known method for training recurrent neural networks in machine learning. In addition, it suggests a method for powerful on-chip learning in energy-efficient spike-based hardware for artificial intelligence.

In vivo odorant input induces spike timing-dependent plasticity of glutamatergic synapses in developing zebrafish olfactory bulb

Early odorant experience and neural activity are essential for refining developing neural connections. Although neural activity-induced synaptic plasticity is one of the most important cellular mechanisms underlying the refinement of neural circuits, whether and how natural odorant experience induces long-term plasticity in the olfactory bulb remains unknown. In vivo perforated whole-cell recording from mitral cells (MCs) in larval zebrafish showed that odorant experience induced persistent modification of developing olfactory bulb circuits via spike timing-dependent plasticity (STDP). Repetitive odorant stimuli paired with postsynaptic spiking in a critical time window (pre-post, positive timing) resulted in persistent enhancement of glutamatergic inputs from olfactory sensory neurons, but long-term depression within the opposite time window (post-pre, negative timing). Furthermore, spike-timing-dependent potentiation (tLTP) in STDP induced by repetitive odorant stimulation had similar cellular processes to those of electrical stimulation-induced tLTP. Finally, odorant input induced STDP required the activation of postsynaptic N-methyl-d-aspartate receptors (NMDARs). Thus, the NMDAR is likely to be a postsynaptic coincidence detector responsible for the sensory experience-dependent refinement of developing connections.

Effect of Circuit Structure on Odor Representation in the Insect Olfactory System

In neuroscience, the structure of a circuit has often been used to intuit function—an inversion of Louis Kahn’s famous dictum, “Form follows function” (Kristan and Katz, 2006). However, different brain networks may use different network architectures to solve the same problem. The olfactory circuits of two insects, the locust, Schistocerca americana, and the fruit fly, Drosophila melanogaster, serve the same function—to identify and discriminate odors. The neural circuitry that achieves this shows marked structural differences. Projection neurons (PNs) in the antennal lobe innervate Kenyon cells (KCs) of the mushroom body. In locust, each KC receives inputs from ∼50% of PNs, a scheme that maximizes the difference between inputs to any two of ∼50,000 KCs. In contrast, in Drosophila, this number is only 5% and appears suboptimal. Using a computational model of the olfactory system, we show that the activity of KCs is sufficiently high-dimensional that it can separate similar odors regardless of the divergence of PN–KC connections. However, when temporal patterning encodes odor attributes, dense connectivity outperforms sparse connections. Increased separability comes at the cost of reliability. The disadvantage of sparse connectivity can be mitigated by incorporating other aspects of circuit architecture seen in Drosophila. Our simulations predict that Drosophila and locust circuits lie at different ends of a continuum where the Drosophila gives up on the ability to resolve similar odors to generalize across varying environments, while the locust separates odor representations but risks misclassifying noisy variants of the same odor.

Autonomous emergence of connectivity assemblies via spike triplet interactions

Non-random connectivity can emerge without structured external input driven by activity-dependent mechanisms of synaptic plasticity based on precise spiking patterns. Here we analyze the emergence of global structures in recurrent networks based on a triplet model of spike timing dependent plasticity (STDP), which depends on the interactions of three precisely-timed spikes, and can describe plasticity experiments with varying spike frequency better than the classical pair-based STDP rule. We derive synaptic changes arising from correlations up to third-order and describe them as the sum of structural motifs, which determine how any spike in the network influences a given synaptic connection through possible connectivity paths. This motif expansion framework reveals novel structural motifs under the triplet STDP rule, which support the formation of bidirectional connections and ultimately the spontaneous emergence of global network structure in the form of self-connected groups of neurons, or assemblies. We propose that under triplet STDP assembly structure can emerge without the need for externally patterned inputs or assuming a symmetric pair-based STDP rule common in previous studies. The emergence of non-random network structure under triplet STDP occurs through internally-generated higher-order correlations, which are ubiquitous in natural stimuli and neuronal spiking activity, and important for coding. We further demonstrate how neuromodulatory mechanisms that modulate the shape of the triplet STDP rule or the synaptic transmission function differentially promote structural motifs underlying the emergence of assemblies, and quantify the differences using graph theoretic measures.

Emergent non-random connectivity structures in different brain regions are tightly related to specific patterns of neural activity and support diverse brain functions. For instance, self-connected groups of neurons, known as assemblies, have been proposed to represent functional units in brain circuits and can emerge even without patterned external instruction. Here we investigate the emergence of non-random connectivity in recurrent networks using a particular plasticity rule, triplet STDP, which relies on the interaction of spike triplets and can capture higher-order statistical dependencies in neural activity. We derive the evolution of the synaptic strengths in the network and explore the conditions for the self-organization of connectivity into assemblies. We demonstrate key differences of the triplet STDP rule compared to the classical pair-based rule in terms of how assemblies are formed, including the realistic asymmetric shape and influence of novel connectivity motifs on network plasticity driven by higher-order correlations. Assembly formation depends on the specific shape of the STDP window and synaptic transmission function, pointing towards an important role of neuromodulatory signals on formation of intrinsically generated assemblies.

Multiple network properties overcome random connectivity to enable stereotypic sensory responses

Connections between neuronal populations may be genetically hardwired or random. In the insect olfactory system, projection neurons of the antennal lobe connect randomly to Kenyon cells of the mushroom body. Consequently, while the odor responses of the projection neurons are stereotyped across individuals, the responses of the Kenyon cells are variable. Surprisingly, downstream of Kenyon cells, mushroom body output neurons show stereotypy in their responses. We found that the stereotypy is enabled by the convergence of inputs from many Kenyon cells onto an output neuron, and does not require learning. The stereotypy emerges in the total response of the Kenyon cell population using multiple odor-specific features of the projection neuron responses, benefits from the nonlinearity in the transfer function, depends on the convergence:randomness ratio, and is constrained by sparseness. Together, our results reveal the fundamental mechanisms and constraints with which convergence enables stereotypy in sensory responses despite random connectivity.

Synaptic Plasticity Forms and Functions

Synaptic plasticity, the activity-dependent change in neuronal connection strength, has long been considered an important component of learning and memory. Computational and engineering work corroborate the power of learning through the directed adjustment of connection weights. Here we review the fundamental elements of four broadly categorized forms of synaptic plasticity and discuss their functional capabilities and limitations. Although standard, correlation-based, Hebbian synaptic plasticity has been the primary focus of neuroscientists for decades, it is inherently limited. Three-factor plasticity rules supplement Hebbian forms with neuromodulation and eligibility traces, while true supervised types go even further by adding objectives and instructive signals. Finally, a recently discovered hippocampal form of synaptic plasticity combines the above elements, while leaving behind the primary Hebbian requirement. We suggest that the effort to determine the neural basis of adaptive behavior could benefit from renewed experimental and theoretical investigation of more powerful directed types of synaptic plasticity.

Optimality of sparse olfactory representations is not affected by network plasticity

The neural representation of a stimulus is repeatedly transformed as it moves from the sensory periphery to deeper layers of the nervous system. Sparsening transformations are thought to increase the separation between similar representations, encode stimuli with great specificity, maximize storage capacity of associative memories, and provide an energy efficient instantiation of information in neural circuits. In the insect olfactory system, odors are initially represented in the periphery as a combinatorial code with relatively simple temporal dynamics. Subsequently, in the antennal lobe this representation is transformed into a dense and complex spatiotemporal activity pattern. Next, in the mushroom body Kenyon cells (KCs), the representation is dramatically sparsened. Finally, in mushroom body output neurons (MBONs), the representation takes on a new dense spatiotemporal format. Here, we develop a computational model to simulate this chain of olfactory processing from the receptor neurons to MBONs. We demonstrate that representations of similar odorants are maximally separated, measured by the distance between the corresponding MBON activity vectors, when KC responses are sparse. Sparseness is maintained across variations in odor concentration by adjusting the feedback inhibition that KCs receive from an inhibitory neuron, the Giant GABAergic neuron. Different odor concentrations require different strength and timing of feedback inhibition for optimal processing. Importantly, as observed in vivo, the KC–MBON synapse is highly plastic, and, therefore, changes in synaptic strength after learning can change the balance of excitation and inhibition, potentially leading to changes in the distance between MBON activity vectors of two odorants for the same level of KC population sparseness. Thus, what is an optimal degree of sparseness before odor learning, could be rendered sub–optimal post learning. Here, we show, however, that synaptic weight changes caused by spike timing dependent plasticity increase the distance between the odor representations from the perspective of MBONs. A level of sparseness that was optimal before learning remains optimal post-learning.

Kenyon cells (KCs) of the mushroom body represent odors as a sparse code. When viewed from the perspective of follower neurons, mushroom body output neurons (MBONs) reveal an optimal level of coding sparseness that maximally separates the representations of odors. However, the KC–MBON synapse is highly plastic and may be potentiated or depressed by odor–driven experience that could, in turn, disrupt the optimality formed by pre–synaptic circuits. Contrary to this expectation, we show that synaptic plasticity based on spike timing of pre- and postsynaptic neurons improves the ability of the system to distinguish between the representations of similar odors while preserving the optimality determined by pre–synaptic circuits.

Opponent processes in visual memories: A model of attraction and repulsion in navigating insects' mushroom bodies

Solitary foraging insects display stunning navigational behaviours in visually complex natural environments. Current literature assumes that these insects are mostly driven by attractive visual memories, which are learnt when the insect's gaze is precisely oriented toward the goal direction, typically along its familiar route or towards its nest. That way, an insect could return home by simply moving in the direction that appears most familiar. Here we show using virtual reconstructions of natural environments that this principle suffers from fundamental drawbacks, notably, a given view of the world does not provide information about whether the agent should turn or not to reach its goal. We propose a simple model where the agent continuously compares its current view with both goal and anti-goal visual memories, which are treated as attractive and repulsive respectively. We show that this strategy effectively results in an opponent process, albeit not at the perceptual level-such as those proposed for colour vision or polarisation detection-but at the level of the environmental space. This opponent process results in a signal that strongly correlates with the angular error of the current body orientation so that a single view of the world now suffices to indicate whether the agent should turn or not. By incorporating this principle into a simple agent navigating in reconstructed natural environments, we show that it overcomes the usual shortcomings and produces a step-increase in navigation effectiveness and robustness. Our findings provide a functional explanation to recent behavioural observations in ants and why and how so-called aversive and appetitive memories must be combined. We propose a likely neural implementation based on insects' mushroom bodies' circuitry that produces behavioural and neural predictions contrasting with previous models.

Nitric oxide acts as a cotransmitter in a subset of dopaminergic neurons to diversify memory dynamics

Animals employ diverse learning rules and synaptic plasticity dynamics to record temporal and statistical information about the world. However, the molecular mechanisms underlying this diversity are poorly understood. The anatomically defined compartments of the insect mushroom body function as parallel units of associative learning, with different learning rates, memory decay dynamics and flexibility (Aso and Rubin, 2016). Here, we show that nitric oxide (NO) acts as a neurotransmitter in a subset of dopaminergic neurons in Drosophila. NO's effects develop more slowly than those of dopamine and depend on soluble guanylate cyclase in postsynaptic Kenyon cells. NO acts antagonistically to dopamine; it shortens memory retention and facilitates the rapid updating of memories. The interplay of NO and dopamine enables memories stored in local domains along Kenyon cell axons to be specialized for predicting the value of odors based only on recent events. Our results provide key mechanistic insights into how diverse memory dynamics are established in parallel memory systems.

An electro-photo-sensitive synaptic transistor for edge neuromorphic visual systems

The practical application of optoelectronic artificial synapses in neuromorphic visual systems is still hindered by their limited functionality, reliability and the challenge of mass production. Here, an electro-photo-sensitive synapse based on a highly reliable amorphous InGaZnO thin-film transistor is demonstrated. Not only does the synapse respond to electrical voltage spikes due to charge trapping/detrapping, but also the weight is modified directly by persistent photocurrent effects under UV-light stimulation. Representative forms of synaptic plasticity, including inhibitory and excitatory postsynaptic currents, frequency-dependent characteristics, short-term to long-term plasticity transitions, and summation effects, are successfully demonstrated. In particular, optoelectronic synergetic modulation leads to reconfigurable excitatory and inhibitory synaptic behaviors, which provides a promising way to achieve the homeostatic regulation of synaptic weights. Moreover, the analogue channel conductance with 100 states is used as the weight update rule to perform MNIST handwritten digit recognition, using system-level LeNet-5 convolutional neural network simulations. The network shows a high recognition accuracy of 95.99% and good tolerance to noisy input patterns. This study highlights the commercial potential of mature optoelectronic InGaZnO transistor technology in edge neuromorphic systems.

Separate But Interactive Parallel Olfactory Processing Streams Governed by Different Types of GABAergic Feedback Neurons in the Mushroom Body of a Basal Insect

The basic organization of the olfactory system has been the subject of extensive studies in vertebrates and invertebrates. In many animals, GABA-ergic neurons inhibit spike activities of higher-order olfactory neurons and help sparsening of their odor representations. In the cockroach, two different types of GABA-immunoreactive interneurons (calyceal giants [CGs]) mainly project to the base and lip regions of the calyces (input areas) of the mushroom body (MB), a second-order olfactory center. The base and lip regions receive axon terminals of two different types of projection neurons, which receive synapses from different classes of olfactory sensory neurons (OSNs), and receive dendrites of different classes of Kenyon cells, MB intrinsic neurons. We performed intracellular recordings from pairs of CGs and MB output neurons (MBONs) of male American cockroaches, the latter receiving synapses from Kenyon cells, and we found that a CG receives excitatory synapses from an MBON and that odor responses of the MBON are changed by current injection into the CG. Such feedback effects, however, were often weak or absent in pairs of neurons that belong to different streams, suggesting parallel organization of the recurrent pathways, although interactions between different streams were also evident. Cross-covariance analysis of the spike activities of CGs and MBONs suggested that odor stimulation produces synchronized spike activities in MBONs and then in CGs. We suggest that there are separate but interactive parallel streams to process odors detected by different OSNs throughout the olfactory processing system in cockroaches.SIGNIFICANCE STATEMENT Organizational principles of the olfactory system have been the subject of extensive studies. In cockroaches, signals from olfactory sensory neurons (OSNs) in two different classes of sensilla are sent to two different classes of projection neurons, which terminate in different areas of the mushroom body (MB), each area having dendrites of different classes of MB intrinsic neurons (Kenyon cells) and terminations of different classes of GABAergic neurons. Physiological and morphological assessments derived from simultaneous intracellular recordings/stainings from GABAergic neurons and MB output neurons suggested that GABAergic neurons play feedback roles and that odors detected by OSNs are processed in separate but interactive processing streams throughout the central olfactory system.

Control of synaptic plasticity in deep cortical networks

Humans and many other animals have an enormous capacity to learn about sensory stimuli and to master new skills. However, many of the mechanisms that enable us to learn remain to be understood. One of the greatest challenges of systems neuroscience is to explain how synaptic connections change to support maximally adaptive behaviour. Here, we provide an overview of factors that determine the change in the strength of synapses, with a focus on synaptic plasticity in sensory cortices. We review the influence of neuromodulators and feedback connections in synaptic plasticity and suggest a specific framework in which these factors can interact to improve the functioning of the entire network.

Biological learning curves outperform existing ones in artificial intelligence algorithms

Recently, deep learning algorithms have outperformed human experts in various tasks across several domains; however, their characteristics are distant from current knowledge of neuroscience. The simulation results of biological learning algorithms presented herein outperform state-of-the-art optimal learning curves in supervised learning of feedforward networks. The biological learning algorithms comprise asynchronous input signals with decaying input summation, weights adaptation, and multiple outputs for an input signal. In particular, the generalization error for such biological perceptrons decreases rapidly with increasing number of examples, and it is independent of the size of the input. This is achieved using either synaptic learning, or solely through dendritic adaptation with a mechanism of swinging between reflecting boundaries, without learning steps. The proposed biological learning algorithms outperform the optimal scaling of the learning curve in a traditional perceptron. It also results in a considerable robustness to disparity between weights of two networks with very similar outputs in biological supervised learning scenarios. The simulation results indicate the potency of neurobiological mechanisms and open opportunities for developing a superior class of deep learning algorithms.

Distinct Dopamine Receptor Pathways Underlie the Temporal Sensitivity of Associative Learning

Animals rely on the relative timing of events in their environment to form and update predictive associations, but the molecular and circuit mechanisms for this temporal sensitivity remain incompletely understood. Here, we show that olfactory associations in Drosophila can be written and reversed on a trial-by-trial basis depending on the temporal relationship between an odor cue and dopaminergic reinforcement. Through the synchronous recording of neural activity and behavior, we show that reversals in learned odor attraction correlate with bidirectional neural plasticity in the mushroom body, the associative olfactory center of the fly. Two dopamine receptors, DopR1 and DopR2, contribute to this temporal sensitivity by coupling to distinct second messengers and directing either synaptic depression or potentiation. Our results reveal how dopamine-receptor signaling pathways can detect the order of events to instruct opposing forms of synaptic and behavioral plasticity, allowing animals to flexibly update their associations in a dynamic environment.

Neuroscience: Intelligence in the Honeybee Mushroom Body

Intelligence, in most people's conception, involves combining pieces of evidence to reach non-obvious conclusions. A recent theoretical study shows that intelligence-like brain functions can emerge from simple neural circuits, in this case the honeybee mushroom body.

Cortical recruitment determines learning dynamics and strategy

Salience is a broad and widely used concept in neuroscience whose neuronal correlates, however, remain elusive. In behavioral conditioning, salience is used to explain various effects, such as stimulus overshadowing, and refers to how fast and strongly a stimulus can be associated with a conditioned event. Here, we identify sounds of equal intensity and perceptual detectability, which due to their spectro-temporal content recruit different levels of population activity in mouse auditory cortex. When using these sounds as cues in a Go/NoGo discrimination task, the degree of cortical recruitment matches the salience parameter of a reinforcement learning model used to analyze learning speed. We test an essential prediction of this model by training mice to discriminate light-sculpted optogenetic activity patterns in auditory cortex, and verify that cortical recruitment causally determines association or overshadowing of the stimulus components. This demonstrates that cortical recruitment underlies major aspects of stimulus salience during reinforcement learning.

Insect-inspired neuromorphic computing

The steady improvement in the performance of computing systems seen for many decades is levelling off as the miniaturization of semiconducting technology approaches the atomic limit, facing severe physical and technological issues. Neuromorphic computing is an emerging solution which makes use of silicon technology in a different way, inline with the computational principles observed in animal nervous systems. In this article, we argue that the nervous systems of insects in particular offer themselves as an ideal starting point for incorporation into realistic neuromorphic systems and review research in developing insect-inspired neuromorphic systems. We conclude with an exciting yet tangible vision of a full neuromorphic sensory-motor system where a liquid state machine modelling the function of the insect mushroom body links input to output and allows for amalgamation of the work discussed in a hierarchical framework of a full system inspired by the concept of how information flows through insects.