Sequential neuromodulation of Hebbian plasticity offers mechanism for effective reward-based navigation

Eligibility traces as a synaptic substrate for learning

Animals can learn to associate a behavior or a stimulus with a delayed reward, this is essential for survival. A mechanism proposed for bridging this gap are synaptic eligibility traces, which are slowly decaying tags, which can lead to synaptic plasticity if followed by rewards. Recently, experiments have demonstrated the existence of synaptic eligibility traces in diverse neural systems, depending on either neuromodulators or plateau potentials. Evidence for both eligibility trace-dependent potentiation and depression of synaptic efficacies has emerged. We discuss the commonalities and differences of these different results. We show why the existence of both potentiation and depression is important because these opposing forces can lead to a synaptic stopping rule. Without a stopping rule, synapses would saturate at their upper bound thus leading to a loss of selectivity and representational power. We discuss the possible underlying mechanisms of the eligibility traces as well as their functional and theoretical significance.

Teucrium Polium ameliorates amyloid β-induced brain network disorders in rats: electrophysiological and behavioral studies

Synaptic failure in specific cholinergic networks in rat brains has been implicated in amyloid β-induced neurodegeneration. Teucrium polium is a promising candidate for drug development against Alzheimer's disease (AD) and similar disorders. However, the protective effect of Teucrium polium against amyloid β-induced impairment of short-term synaptic plasticity is still poorly understood. In this study, we used in vivo extracellular single-unit recordings to investigate the preventive efficacy of Teucrium polium on Aβ(25-35)-induced aberrant neuronal activity in the hippocampus and basolateral amygdala of rats, in response to high-frequency stimulation of the cholinergic nucleus basalis magnocellularis (NBM). After 12 weeks of intracerebroventricular administration of Aβ(25-35), alterations such as decreased excitatory responses and increased inhibitory synaptic activity were observed in the NBM-hippocampus and NBM-basolateral amygdala cholinergic circuits. Treatment with Teucrium polium improved the balance of excitatory and inhibitory responses by modulating synaptic transmission strength and restoring short-term plasticity. Acute injection of a therapeutic dose of Teucrium temporarily inhibited spiking activity in single NBM neurons. Open field tests revealed that amyloid-injected rats displayed anxiety and reduced exploratory drive. Treatment with Teucrium polium improved these behaviors, reducing anxiety and increasing exploration. Teucrium polium mitigated amyloid β-induced alterations in cholinergic circuits by enhancing the adaptive capacity of short-term synaptic plasticity. These findings suggest that Teucrium polium could serve as a preventive strategy to delay the progression of cholinergic neurodegeneration.

Non-synaptic plasticity enables memory-dependent local learning

Synaptic plasticity is essential for memory formation and learning in the brain. In addition, recent results indicate that non-synaptic plasticity processes such as the regulation of neural membrane properties contribute to memory formation, its functional role in memory and learning has however remained elusive. Here, we propose that non-synaptic and synaptic plasticity are both essential components to enable memory-dependent processing in neuronal networks. While the former acts on a fast timescale for rapid information storage, the latter shapes network processing on a slower timescale to harness this memory as a functional component. We analyse this concept in a network model where pyramidal neurons regulate their apical trunk excitability in a Hebbian manner. We find that local synaptic plasticity rules can be derived for this model and show that the interplay between this synaptic plasticity and the non-synaptic trunk plasticity enables the model to successfully accommodate memory-dependent processing capabilities in a number of tasks, ranging from simple memory tests to question answering.

Data-driven synapse classification reveals a logic of glutamate receptor diversity

The rich diversity of synapses facilitates the capacity of neural circuits to transmit, process and store information. We used multiplex super-resolution proteometric imaging through array tomography to define features of single synapses in mouse neocortex. We find that glutamatergic synapses cluster into subclasses that parallel the distinct biochemical and functional categories of receptor subunits: GluA1/4, GluA2/3 and GluN1/GluN2B. Two of these subclasses align with physiological expectations based on synaptic plasticity: large AMPAR-rich synapses may represent potentiated synapses, whereas small NMDAR-rich synapses suggest "silent" synapses. The NMDA receptor content of large synapses correlates with spine neck diameter, and thus the potential for coupling to the parent dendrite. Overall, ultrastructural features predict receptor content of synapses better than parent neuron identity does, suggesting synapse subclasses act as fundamental elements of neuronal circuits. No barriers prevent future generalization of this approach to other species, or to study of human disorders and therapeutics.

Ventral Pallidum Neurons Are Necessary to Generalize and Express Fear-Related Responding in a Minimal Threat Setting

Fear generalization is a hallmark of anxiety disorders. Experimentally, fear generalization can be difficult to dissociate from its counterpart, fear discrimination. Here, we use minimal threat learning procedures to reveal such a dissociation. We show that in Long-Evans rats, an auditory threat cue predicting footshock on 10% of trials produces a discriminated fear response that does not generalize to a neutral auditory cue. In contrast, even slightly higher footshock probabilities (30 and 20%) produce fear generalization. AAV-mediated, caspase-3 deletion of ventral pallidum neurons abolishes fear generalization and reduces threat cue responding during extinction. The ventral pallidum's contribution to fear generalization and extinction threat responding does not depend on inputs from the nucleus accumbens. The results demonstrate a minimal threat learning approach to dissociate fear discrimination from fear generalization and a novel role for the ventral pallidum in generalizing and expressing fear.

The role of astrocytes from synaptic to non-synaptic plasticity

Information storage and transfer in the brain require a high computational power. Neuronal network display various local or global mechanisms to allow information storage and transfer in the brain. From synaptic to intrinsic plasticity, the rules of input-output function modulation have been well characterized in neurons. In the past years, astrocytes have been suggested to increase the computational power of the brain and we are only just starting to uncover their role in information processing. Astrocytes maintain a close bidirectional communication with neurons to modify neuronal network excitability, transmission, axonal conduction, and plasticity through various mechanisms including the release of gliotransmitters or local ion homeostasis. Astrocytes have been significantly studied in the context of long-term or short-term synaptic plasticity, but this is not the only mechanism involved in memory formation. Plasticity of intrinsic neuronal excitability also participates in memory storage through regulation of voltage-gated ion channels or axonal morphological changes. Yet, the contribution of astrocytes to these other forms of non-synaptic plasticity remains to be investigated. In this review, we summarized the recent advances on the role of astrocytes in different forms of plasticity and discuss new directions and ideas to be explored regarding astrocytes-neuronal communication and regulation of plasticity.

Astrocytes enhance plasticity response during reversal learning

Astrocytes play a key role in the regulation of synaptic strength and are thought to orchestrate synaptic plasticity and memory. Yet, how specifically astrocytes and their neuroactive transmitters control learning and memory is currently an open question. Recent experiments have uncovered an astrocyte-mediated feedback loop in CA1 pyramidal neurons which is started by the release of endocannabinoids by active neurons and closed by astrocytic regulation of the D-serine levels at the dendrites. D-serine is a co-agonist for the NMDA receptor regulating the strength and direction of synaptic plasticity. Activity-dependent D-serine release mediated by astrocytes is therefore a candidate for mediating between long-term synaptic depression (LTD) and potentiation (LTP) during learning. Here, we show that the mathematical description of this mechanism leads to a biophysical model of synaptic plasticity consistent with the phenomenological model known as the BCM model. The resulting mathematical framework can explain the learning deficit observed in mice upon disruption of the D-serine regulatory mechanism. It shows that D-serine enhances plasticity during reversal learning, ensuring fast responses to changes in the external environment. The model provides new testable predictions about the learning process, driving our understanding of the functional role of neuron-glia interaction in learning.

Anti-Hebbian plasticity drives sequence learning in striatum

Spatio-temporal activity patterns have been observed in a variety of brain areas in spontaneous activity, prior to or during action, or in response to stimuli. Biological mechanisms endowing neurons with the ability to distinguish between different sequences remain largely unknown. Learning sequences of spikes raises multiple challenges, such as maintaining in memory spike history and discriminating partially overlapping sequences. Here, we show that anti-Hebbian spike-timing dependent plasticity (STDP), as observed at cortico-striatal synapses, can naturally lead to learning spike sequences. We design a spiking model of the striatal output neuron receiving spike patterns defined as sequential input from a fixed set of cortical neurons. We use a simple synaptic plasticity rule that combines anti-Hebbian STDP and non-associative potentiation for a subset of the presented patterns called rewarded patterns. We study the ability of striatal output neurons to discriminate rewarded from non-rewarded patterns by firing only after the presentation of a rewarded pattern. In particular, we show that two biological properties of striatal networks, spiking latency and collateral inhibition, contribute to an increase in accuracy, by allowing a better discrimination of partially overlapping sequences. These results suggest that anti-Hebbian STDP may serve as a biological substrate for learning sequences of spikes.

Navigation and the efficiency of spatial coding: insights from closed-loop simulations

Spatial learning is critical for survival and its underlying neuronal mechanisms have been studied extensively. These studies have revealed a wealth of information about the neural representations of space, such as place cells and boundary cells. While many studies have focused on how these representations emerge in the brain, their functional role in driving spatial learning and navigation has received much less attention. We extended an existing computational modeling tool-chain to study the functional role of spatial representations using closed-loop simulations of spatial learning. At the heart of the model agent was a spiking neural network that formed a ring attractor. This network received inputs from place and boundary cells and the location of the activity bump in this network was the output. This output determined the movement directions of the agent. We found that the navigation performance depended on the parameters of the place cell input, such as their number, the place field sizes, and peak firing rate, as well as, unsurprisingly, the size of the goal zone. The dependence on the place cell parameters could be accounted for by just a single variable, the overlap index, but this dependence was nonmonotonic. By contrast, performance scaled monotonically with the Fisher information of the place cell population. Our results therefore demonstrate that efficiently encoding spatial information is critical for navigation performance.

Distinct GABAergic modulation of timing-dependent LTP in CA1 pyramidal neurons along the longitudinal axis of the mouse hippocampus

Synaptic plasticity in the hippocampus underlies episodic memory formation, with dorsal hippocampus being instrumental for spatial memory whereas ventral hippocampus is crucial for emotional learning. Here, we studied how GABAergic inhibition regulates physiologically relevant low repeat spike timing-dependent LTP (t-LTP) at Schaffer collateral-CA1 synapses along the dorsoventral hippocampal axis. We used two t-LTP protocols relying on only 6 repeats of paired spike-firing in pre- and postsynaptic cells within 10 s that differ in postsynaptic firing patterns. GABAA receptor mechanisms played a greater role in blocking 6× 1:1 t-LTP that recruits single postsynaptic action potentials. 6× 1:4 t-LTP that depends on postsynaptic burst-firing unexpectedly required intact GABAB receptor signaling. The magnitude of both t-LTP-forms decreased along the dorsoventral axis, despite increasing excitability and basal synaptic strength in this direction. This suggests that GABAergic inhibition contributes to the distinct roles of dorsal and ventral hippocampus in memory formation.

Divergent Learning-Related Transcriptional States of Cortical Glutamatergic Neurons

Experience-dependent gene expression reshapes neural circuits, permitting the learning of knowledge and skills. Most learning involves repetitive experiences during which neurons undergo multiple stages of functional and structural plasticity. Currently, the diversity of transcriptional responses underlying dynamic plasticity during repetition-based learning is poorly understood. To close this gap, we analyzed single-nucleus transcriptomes of L2/3 glutamatergic neurons of the primary motor cortex after 3 d motor skill training or home cage control in water-restricted male mice. "Train" and "control" neurons could be discriminated with high accuracy based on expression patterns of many genes, indicating that recent experience leaves a widespread transcriptional signature across L2/3 neurons. These discriminating genes exhibited divergent modes of coregulation, differentiating neurons into discrete clusters of transcriptional states. Several states showed gene expressions associated with activity-dependent plasticity. Some of these states were also prominent in the previously published reference, suggesting that they represent both spontaneous and task-related plasticity events. Markedly, however, two states were unique to our dataset. The first state, further enriched by motor training, showed gene expression suggestive of late-stage plasticity with repeated activation, which is suitable for expected emergent neuronal ensembles that stably retain motor learning. The second state, equally found in both train and control mice, showed elevated levels of metabolic pathways and norepinephrine sensitivity, suggesting a response to common experiences specific to our experimental conditions, such as water restriction or circadian rhythm. Together, we uncovered divergent transcriptional responses across L2/3 neurons, each potentially linked with distinct features of repetition-based motor learning such as plasticity, memory, and motivation.

[Spatial navigation method based on the entorhinal-hippocampal-prefrontal information transmission circuit of rat's brain]

Physiological studies have revealed that rats perform spatial localization relying on grid cells and place cells in the entorhinal-hippocampal CA3 structure. The dynamic connection between the entorhinal-hippocampal structure and the prefrontal cortex is crucial for navigation. Based on these findings, this paper proposes a spatial navigation method based on the entorhinal-hippocampal-prefrontal information transmission circuit of the rat's brain, with the aim of endowing the mobile robot with strong spatial navigation capability. Using the hippocampal CA3-prefrontal spatial navigation model as a foundation, this paper constructed a dynamic self-organizing model with the hippocampal CA1 place cells as the basic unit to optimize the navigation path. The path information was then fed back to the impulse neural network via hippocampal CA3 place cells and prefrontal cortex action neurons, improving the convergence speed of the model and helping to establish long-term memory of navigation habits. To verify the validity of the method, two-dimensional simulation experiments and three-dimensional simulation robot experiments were designed in this paper. The experimental results showed that the method presented in this paper not only surpassed other algorithms in terms of navigation efficiency and convergence speed, but also exhibited good adaptability to dynamic navigation tasks. Furthermore, our method can be effectively applied to mobile robots.

Acetylcholine facilitates localized synaptic potentiation and location specific feature binding

Forebrain acetylcholine (ACh) signaling has been shown to drive attention and learning. Recent experimental evidence of spatially and temporally constrained cholinergic signaling has sparked interest to investigate how it facilitates stimulus-induced learning. We use biophysical excitatory-inhibitory (E-I) multi-module neural network models to show that external stimuli and ACh signaling can mediate spatially constrained synaptic potentiation patterns. The effects of ACh on neural excitability are simulated by varying the conductance of a muscarinic receptor-regulated hyperpolarizing slow K+ current (m-current). Each network module consists of an E-I network with local excitatory connectivity and global inhibitory connectivity. The modules are interconnected with plastic excitatory synaptic connections, that change via a spike-timing-dependent plasticity (STDP) rule. Our results indicate that spatially constrained ACh release influences the information flow represented by network dynamics resulting in selective reorganization of inter-module interactions. Moreover the information flow depends on the level of synchrony in the network. For highly synchronous networks, the more excitable module leads firing in the less excitable one resulting in strengthening of the outgoing connections from the former and weakening of its incoming synapses. For networks with more noisy firing patterns, activity in high ACh regions is prone to induce feedback firing of synchronous volleys and thus strengthening of the incoming synapses to the more excitable region and weakening of outgoing synapses. Overall, these results suggest that spatially and directionally specific plasticity patterns, as are presumed necessary for feature binding, can be mediated by spatially constrained ACh release.

A Navigation Path Search and Optimization Method for Mobile Robots Based on the Rat Brain's Cognitive Mechanism

Rats possess exceptional navigational abilities, allowing them to adaptively adjust their navigation paths based on the environmental structure. This remarkable ability is attributed to the interactions and regulatory mechanisms among various spatial cells within the rat's brain. Based on these, this paper proposes a navigation path search and optimization method for mobile robots based on the rat brain's cognitive mechanism. The aim is to enhance the navigation efficiency of mobile robots. The mechanism of this method is based on developing a navigation habit. Firstly, the robot explores the environment to search for the navigation goal. Then, with the assistance of boundary vector cells, the greedy strategy is used to guide the robot in generating a locally optimal path. Once the navigation path is generated, a dynamic self-organizing model based on the hippocampal CA1 place cells is constructed to further optimize the navigation path. To validate the effectiveness of the method, this paper designs several 2D simulation experiments and 3D robot simulation experiments, and compares the proposed method with various algorithms. The experimental results demonstrate that the proposed method not only surpasses other algorithms in terms of path planning efficiency but also yields the shortest navigation path. Moreover, the method exhibits good adaptability to dynamic navigation tasks.

Prediction errors drive dynamic changes in neural patterns that guide behavior

Learning describes the process by which our internal expectation models of the world are updated by surprising outcomes (prediction errors [PEs]) to improve predictions of future events. However, the mechanisms through which error signals dynamically influence existing neural representations are unknown. Here, we use functional magnetic resonance imaging (fMRI) in humans solving a two-step Markov decision task to investigate changes in neural activation patterns following PEs. Using a dynamic multivariate pattern analysis, we can show that PE-related fMRI responses in error-coding regions predict trial-by-trial changes in multivariate neural patterns in the orbitofrontal cortex, the precuneus, and the ventromedial prefrontal cortex (vmPFC). Importantly, the dynamics of these pattern changes in the vmPFC also predicted upcoming changes in choice strategies and thus highlight the importance of these pattern changes for behavior.

[A bio-inspired hierarchical spiking neural network with biological synaptic plasticity for event camera object recognition]

With inherent sparse spike-based coding and asynchronous event-driven computation, spiking neural network (SNN) is naturally suitable for processing event stream data of event cameras. In order to improve the feature extraction and classification performance of bio-inspired hierarchical SNNs, in this paper an event camera object recognition system based on biological synaptic plasticity is proposed. In our system input event streams were firstly segmented adaptively using spiking neuron potential to improve computational efficiency of the system. Multi-layer feature learning and classification are implemented by our bio-inspired hierarchical SNN with synaptic plasticity. After Gabor filter-based event-driven convolution layer which extracted primary visual features of event streams, we used a feature learning layer with unsupervised spiking timing dependent plasticity (STDP) rule to help the network extract frequent salient features, and a feature learning layer with reward-modulated STDP rule to help the network learn diagnostic features. The classification accuracies of the network proposed in this paper on the four benchmark event stream datasets were better than the existing bio-inspired hierarchical SNNs. Moreover, our method showed good classification ability for short event stream input data, and was robust to input event stream noise. The results show that our method can improve the feature extraction and classification performance of this kind of SNNs for event camera object recognition.

New perspectives on the basal forebrain cholinergic system in Alzheimer's disease

The basal forebrain cholinergic system (BFCS) has long been implicated in age-related cognitive changes and the pathophysiology of Alzheimer's disease (AD). Limitations of cholinergic interventions helped to inspire a shift away from BFCS in AD research. A resurgence in interest in the BFCS following methodological and analytical advances has resulted in a call for the BFCS to be examined in novel frameworks. We outline the basic structure and function of the BFCS, its role in supporting cognitive and affective function, and its vulnerability to aging and AD. We consider the BFCS in the context of the amyloid hypothesis and evolving concepts in AD research: resilience and resistance to pathology, selective neuronal vulnerability, trans-synaptic pathology spread and sleep health. We highlight 1) the potential role of the BFCS in cognitive resilience, 2) recent work refining understanding about the selective vulnerability of BFCS to AD, 3) BFCS connectivity that suggests it is related to tau spreading and neurodegeneration and 4) the gap between BFCS involvement in AD and sleep-wake cycles.

Biphasic Cholinergic Modulation of Reverberatory Activity in Neuronal Networks

Acetylcholine (ACh) is an important neuromodulator in various cognitive functions. However, it is unclear how ACh influences neural circuit dynamics by altering cellular properties. Here, we investigated how ACh influences reverberatory activity in cultured neuronal networks. We found that ACh suppressed the occurrence of evoked reverberation at low to moderate doses, but to a much lesser extent at high doses. Moreover, high doses of ACh caused a longer duration of evoked reverberation, and a higher occurrence of spontaneous activity. With whole-cell recording from single neurons, we found that ACh inhibited excitatory postsynaptic currents (EPSCs) while elevating neuronal firing in a dose-dependent manner. Furthermore, all ACh-induced cellular and network changes were blocked by muscarinic, but not nicotinic receptor antagonists. With computational modeling, we found that simulated changes in EPSCs and the excitability of single cells mimicking the effects of ACh indeed modulated the evoked network reverberation similar to experimental observations. Thus, ACh modulates network dynamics in a biphasic fashion, probably by inhibiting excitatory synaptic transmission and facilitating neuronal excitability through muscarinic signaling pathways.

Adaptive control of synaptic plasticity integrates micro- and macroscopic network function

Synaptic plasticity configures interactions between neurons and is therefore likely to be a primary driver of behavioral learning and development. How this microscopic-macroscopic interaction occurs is poorly understood, as researchers frequently examine models within particular ranges of abstraction and scale. Computational neuroscience and machine learning models offer theoretically powerful analyses of plasticity in neural networks, but results are often siloed and only coarsely linked to biology. In this review, we examine connections between these areas, asking how network computations change as a function of diverse features of plasticity and vice versa. We review how plasticity can be controlled at synapses by calcium dynamics and neuromodulatory signals, the manifestation of these changes in networks, and their impacts in specialized circuits. We conclude that metaplasticity-defined broadly as the adaptive control of plasticity-forges connections across scales by governing what groups of synapses can and can't learn about, when, and to what ends. The metaplasticity we discuss acts by co-opting Hebbian mechanisms, shifting network properties, and routing activity within and across brain systems. Asking how these operations can go awry should also be useful for understanding pathology, which we address in the context of autism, schizophrenia and Parkinson's disease.

Continuous cholinergic-dopaminergic updating in the nucleus accumbens underlies approaches to reward-predicting cues

The ability to learn Pavlovian associations from environmental cues predicting positive outcomes is critical for survival, motivating adaptive behaviours. This cued-motivated behaviour depends on the nucleus accumbens (NAc). NAc output activity mediated by spiny projecting neurons (SPNs) is regulated by dopamine, but also by cholinergic interneurons (CINs), which can release acetylcholine and glutamate via the activity of the vesicular acetylcholine transporter (VAChT) or the vesicular glutamate transporter (VGLUT3), respectively. Here we investigated behavioural and neurochemical changes in mice performing a touchscreen Pavlovian approach task by recording dopamine, acetylcholine, and calcium dynamics from D1- and D2-SPNs using fibre photometry in control, VAChT or VGLUT3 mutant mice to understand how these signals cooperate in the service of approach behaviours toward reward-predicting cues. We reveal that NAc acetylcholine-dopaminergic signalling is continuously updated to regulate striatal output underlying the acquisition of Pavlovian approach learning toward reward-predicting cues.

Neuromodulator-dependent synaptic tagging and capture retroactively controls neural coding in spiking neural networks

Events that are important to an individual's life trigger neuromodulator release in brain areas responsible for cognitive and behavioral function. While it is well known that the presence of neuromodulators such as dopamine and norepinephrine is required for memory consolidation, the impact of neuromodulator concentration is, however, less understood. In a recurrent spiking neural network model featuring neuromodulator-dependent synaptic tagging and capture, we study how synaptic memory consolidation depends on the amount of neuromodulator present in the minutes to hours after learning. We find that the storage of rate-based and spike timing-based information is controlled by the level of neuromodulation. Specifically, we find better recall of temporal information for high levels of neuromodulation, while we find better recall of rate-coded spatial patterns for lower neuromodulation, mediated by the selection of different groups of synapses for consolidation. Hence, our results indicate that in minutes to hours after learning, the level of neuromodulation may alter the process of synaptic consolidation to ultimately control which type of information becomes consolidated in the recurrent neural network.

Continuous learning of spiking networks trained with local rules

Artificial neural networks (ANNs) experience catastrophic forgetting (CF) during sequential learning. In contrast, the brain can learn continuously without any signs of catastrophic forgetting. Spiking neural networks (SNNs) are the next generation of ANNs with many features borrowed from biological neural networks. Thus, SNNs potentially promise better resilience to CF. In this paper, we study the susceptibility of SNNs to CF and test several biologically inspired methods for mitigating catastrophic forgetting. SNNs are trained with biologically plausible local training rules based on spike-timing-dependent plasticity (STDP). Local training prohibits the direct use of CF prevention methods based on gradients of a global loss function. We developed and tested the method to determine the importance of synapses (weights) based on stochastic Langevin dynamics without the need for the gradients. Several other methods of catastrophic forgetting prevention adapted from analog neural networks were tested as well. The experiments were performed on freely available datasets in the SpykeTorch environment.

Memory trace imbalance in reinforcement and punishment systems can reinforce implicit choices leading to obsessive-compulsive behavior

We may view most of our daily activities as rational action selections; however, we sometimes reinforce maladaptive behaviors despite having explicit environmental knowledge. In this study, we model obsessive-compulsive disorder (OCD) symptoms as implicitly learned maladaptive behaviors. Simulations in the reinforcement learning framework show that agents implicitly learn to respond to intrusive thoughts when the memory trace signal for past actions decays differently for positive and negative prediction errors. Moreover, this model extends our understanding of therapeutic effects of behavioral therapy in OCD. Using empirical data, we confirm that patients with OCD show extremely imbalanced traces, which are normalized by serotonin enhancers. We find that healthy participants also vary in their obsessive-compulsive tendencies, consistent with the degree of imbalanced traces. These behavioral characteristics can be generalized to variations in the healthy population beyond the spectrum of clinical phenotypes.

Norepinephrine potentiates and serotonin depresses visual cortical responses by transforming eligibility traces

Reinforcement allows organisms to learn which stimuli predict subsequent biological relevance. Hebbian mechanisms of synaptic plasticity are insufficient to account for reinforced learning because neuromodulators signaling biological relevance are delayed with respect to the neural activity associated with the stimulus. A theoretical solution is the concept of eligibility traces (eTraces), silent synaptic processes elicited by activity which upon arrival of a neuromodulator are converted into a lasting change in synaptic strength. Previously we demonstrated in visual cortical slices the Hebbian induction of eTraces and their conversion into LTP and LTD by the retroactive action of norepinephrine and serotonin Here we show in vivo in mouse V1 that the induction of eTraces and their conversion to LTP/D by norepinephrine and serotonin respectively potentiates and depresses visual responses. We also show that the integrity of this process is crucial for ocular dominance plasticity, a canonical model of experience-dependent plasticity.

Modulation of hippocampal plasticity in learning and memory

Synaptic plasticity plays a central role in the study of neural mechanisms of learning and memory. Plasticity rules are not invariant over time but are under neuromodulatory control, enabling behavioral states to influence memory formation. Neuromodulation controls synaptic plasticity at network level by directing information flow, at circuit level through changes in excitation/inhibition balance, and at synaptic level through modulation of intracellular signaling cascades. Although most research has focused on modulation of principal neurons, recent progress has uncovered important roles for interneurons in not only routing information, but also setting conditions for synaptic plasticity. Moreover, astrocytes have been shown to both gate and mediate plasticity. These additional mechanisms must be considered for a comprehensive mechanistic understanding of learning and memory.

Training a spiking neuronal network model of visual-motor cortex to play a virtual racket-ball game using reinforcement learning

Recent models of spiking neuronal networks have been trained to perform behaviors in static environments using a variety of learning rules, with varying degrees of biological realism. Most of these models have not been tested in dynamic visual environments where models must make predictions on future states and adjust their behavior accordingly. The models using these learning rules are often treated as black boxes, with little analysis on circuit architectures and learning mechanisms supporting optimal performance. Here we developed visual/motor spiking neuronal network models and trained them to play a virtual racket-ball game using several reinforcement learning algorithms inspired by the dopaminergic reward system. We systematically investigated how different architectures and circuit-motifs (feed-forward, recurrent, feedback) contributed to learning and performance. We also developed a new biologically-inspired learning rule that significantly enhanced performance, while reducing training time. Our models included visual areas encoding game inputs and relaying the information to motor areas, which used this information to learn to move the racket to hit the ball. Neurons in the early visual area relayed information encoding object location and motion direction across the network. Neuronal association areas encoded spatial relationships between objects in the visual scene. Motor populations received inputs from visual and association areas representing the dorsal pathway. Two populations of motor neurons generated commands to move the racket up or down. Model-generated actions updated the environment and triggered reward or punishment signals that adjusted synaptic weights so that the models could learn which actions led to reward. Here we demonstrate that our biologically-plausible learning rules were effective in training spiking neuronal network models to solve problems in dynamic environments. We used our models to dissect the circuit architectures and learning rules most effective for learning. Our model shows that learning mechanisms involving different neural circuits produce similar performance in sensory-motor tasks. In biological networks, all learning mechanisms may complement one another, accelerating the learning capabilities of animals. Furthermore, this also highlights the resilience and redundancy in biological systems.

Chalcogenide optomemristors for multi-factor neuromorphic computation

Neuromorphic hardware that emulates biological computations is a key driver of progress in AI. For example, memristive technologies, including chalcogenide-based in-memory computing concepts, have been employed to dramatically accelerate and increase the efficiency of basic neural operations. However, powerful mechanisms such as reinforcement learning and dendritic computation require more advanced device operations involving multiple interacting signals. Here we show that nano-scaled films of chalcogenide semiconductors can perform such multi-factor in-memory computation where their tunable electronic and optical properties are jointly exploited. We demonstrate that ultrathin photoactive cavities of Ge-doped Selenide can emulate synapses with three-factor neo-Hebbian plasticity and dendrites with shunting inhibition. We apply these properties to solve a maze game through on-device reinforcement learning, as well as to provide a single-neuron solution to linearly inseparable XOR implementation.

Some types of machine learning rely on the interaction between multiple signals, which requires new devices for efficient implementation. Here, Sarwat et al demonstrate a memristor that is both optically and electronically active, enabling computational models such as three factor learning.

Dopamine and serotonin interplay for valence-based spatial learning

Dopamine (DA) and serotonin (5-HT) are important neuromodulators of synaptic plasticity that have been linked to learning from positive or negative outcomes or valence-based learning. In the hippocampus, both affect long-term plasticity but play different roles in encoding uncertainty or predicted reward. DA has been related to positive valence, from reward consumption or avoidance behavior, and 5-HT to aversive encoding. We propose DA produces overall LTP while 5-HT elicits LTD. Here, we compare two reward-modulated spike timing-dependent plasticity (R-STDP) rules to describe the action of these neuromodulators. We examined their role in cognitive performance and flexibility for computational models of the Morris water maze task and reversal learning. Our results show that the interplay of DA and 5-HT improves learning performance and can explain experimental evidence. This study reinforces the importance of neuromodulation in determining the direction of plasticity.

A Nonlinear Hidden Layer Enables Actor-Critic Agents to Learn Multiple Paired Association Navigation

Navigation to multiple cued reward locations has been increasingly used to study rodent learning. Though deep reinforcement learning agents have been shown to be able to learn the task, they are not biologically plausible. Biologically plausible classic actor-critic agents have been shown to learn to navigate to single reward locations, but which biologically plausible agents are able to learn multiple cue-reward location tasks has remained unclear. In this computational study, we show versions of classic agents that learn to navigate to a single reward location, and adapt to reward location displacement, but are not able to learn multiple paired association navigation. The limitation is overcome by an agent in which place cell and cue information are first processed by a feedforward nonlinear hidden layer with synapses to the actor and critic subject to temporal difference error-modulated plasticity. Faster learning is obtained when the feedforward layer is replaced by a recurrent reservoir network.

Canonical neural networks perform active inference

This work considers a class of canonical neural networks comprising rate coding models, wherein neural activity and plasticity minimise a common cost function-and plasticity is modulated with a certain delay. We show that such neural networks implicitly perform active inference and learning to minimise the risk associated with future outcomes. Mathematical analyses demonstrate that this biological optimisation can be cast as maximisation of model evidence, or equivalently minimisation of variational free energy, under the well-known form of a partially observed Markov decision process model. This equivalence indicates that the delayed modulation of Hebbian plasticity-accompanied with adaptation of firing thresholds-is a sufficient neuronal substrate to attain Bayes optimal inference and control. We corroborated this proposition using numerical analyses of maze tasks. This theory offers a universal characterisation of canonical neural networks in terms of Bayesian belief updating and provides insight into the neuronal mechanisms underlying planning and adaptive behavioural control.

Active inference leads to Bayesian neurophysiology

The neuronal substrates that implement the free-energy principle and ensuing active inference at the neuron and synapse level have not been fully elucidated. This Review considers possible neuronal substrates underlying the principle. First, the foundations of the free-energy principle are introduced, and then its ability to empirically explain various brain functions and psychological and biological phenomena in terms of Bayesian inference is described. Mathematically, the dynamics of neural activity and plasticity that minimise a cost function can be cast as performing Bayesian inference that minimises variational free energy. This equivalence licenses the adoption of the free-energy principle as a universal characterisation of neural networks. Further, the neural network structure itself represents a generative model under which an agent operates. A virtue of this perspective is that it enables the formal association of neural network properties with prior beliefs that regulate inference and learning. The possible neuronal substrates that implement prior beliefs and how to empirically examine the theory are discussed. This perspective renders brain activity explainable, leading to a deeper understanding of the neuronal mechanisms underlying basic psychology and psychiatric disorders in terms of an implicit generative model.

Bidirectional synaptic plasticity rapidly modifies hippocampal representations

Learning requires neural adaptations thought to be mediated by activity-dependent synaptic plasticity. A relatively non-standard form of synaptic plasticity driven by dendritic calcium spikes, or plateau potentials, has been reported to underlie place field formation in rodent hippocampal CA1 neurons. Here, we found that this behavioral timescale synaptic plasticity (BTSP) can also reshape existing place fields via bidirectional synaptic weight changes that depend on the temporal proximity of plateau potentials to pre-existing place fields. When evoked near an existing place field, plateau potentials induced less synaptic potentiation and more depression, suggesting BTSP might depend inversely on postsynaptic activation. However, manipulations of place cell membrane potential and computational modeling indicated that this anti-correlation actually results from a dependence on current synaptic weight such that weak inputs potentiate and strong inputs depress. A network model implementing this bidirectional synaptic learning rule suggested that BTSP enables population activity, rather than pairwise neuronal correlations, to drive neural adaptations to experience.

[A review of brain-like spiking neural network and its neuromorphic chip research]

Under the current situation of the rapid development of brain-like artificial intelligence and the increasingly complex electromagnetic environment, the most bionic and anti-interference spiking neural network has shown great potential in computing speed, real-time information processing, and spatiotemporal data processing. Spiking neural network is the core part of brain-like artificial intelligence, which realizes brain-like computing by simulating the structure of biological neural network and the way of information transmission. This article first summarizes the advantages and disadvantages of the five models, and analyzes the characteristics of several network topologies. Then, it summarizes the spiking neural network algorithms. The unsupervised learning based on spike timing dependent plasticity (STDP) rules and four types of supervised learning algorithms are analyzed. Finally, the research on brain-like neuromorphic chips at home and abroad are reviewed. This paper aims to provide learning ideas and research directions for new colleagues in the field of spiking neural network.

A Multisubcellular Compartment Model of AMPA Receptor Trafficking for Neuromodulation of Hebbian Synaptic Plasticity

Neuromodulation can profoundly impact the gain and polarity of postsynaptic changes in Hebbian synaptic plasticity. An emerging pattern observed in multiple central synapses is a pull–push type of control in which activation of receptors coupled to the G-protein Gs promote long-term potentiation (LTP) at the expense of long-term depression (LTD), whereas receptors coupled to Gq promote LTD at the expense of LTP. Notably, coactivation of both Gs- and Gq-coupled receptors enhances the gain of both LTP and LTD. To account for these observations, we propose a simple kinetic model in which AMPA receptors (AMPARs) are trafficked between multiple subcompartments in and around the postsynaptic spine. In the model AMPARs in the postsynaptic density compartment (PSD) are the primary contributors to synaptic conductance. During LTP induction, AMPARs are trafficked to the PSD primarily from a relatively small perisynaptic (peri-PSD) compartment. Gs-coupled receptors promote LTP by replenishing peri-PSD through increased AMPAR exocytosis from a pool of endocytic AMPAR. During LTD induction AMPARs are trafficked in the reverse direction, from the PSD to the peri-PSD compartment, and Gq-coupled receptors promote LTD by clearing the peri-PSD compartment through increased AMPAR endocytosis. We claim that the model not only captures essential features of the pull–push neuromodulation of synaptic plasticity, but it is also consistent with other actions of neuromodulators observed in slice experiments and is compatible with the current understanding of AMPAR trafficking.

A computational model of dopaminergic modulation of hippocampal Schaffer collateral-CA1 long-term plasticity

Dopamine plays a critical role in modulating the long-term synaptic plasticity of the hippocampal Schaffer collateral-CA1 pyramidal neuron synapses (SC-CA1), a widely accepted cellular model of learning and memory. Limited results from hippocampal slice experiments over the last four decades have shown that the timing of the activation of dopamine D1/D5 receptors relative to a high/low-frequency stimulation (HFS/LFS) in SC-CA1 synapses regulates the modulation of HFS/LFS-induced long-term potentiation/depression (LTP/LTD) in these synapses. However, the existing literature lacks a complete picture of how various concentrations of D1/D5 agonists and the relative timing between the activation of D1/D5 receptors and LTP/LTD induction by HFS/LFS, affect the spatiotemporal modulation of SC-CA1 synaptic dynamics. In this paper, we have developed a computational model, a first of its kind, to make quantitative predictions of the temporal dose-dependent modulation of the HFS/LFS induced LTP/LTD in SC-CA1 synapses by various D1/D5 agonists. Our model combines the biochemical effects with the electrical effects at the electrophysiological level. We have estimated the model parameters from the published electrophysiological data, available from diverse hippocampal CA1 slice experiments, in a Bayesian framework. Our modeling results demonstrate the capability of our model in making quantitative predictions of the available experimental results under diverse HFS/LFS protocols. The predictions from our model show a strong nonlinear dependency of the modulated LTP/LTD by D1/D5 agonists on the relative timing between the activated D1/D5 receptors and the HFS/LFS protocol and the applied concentration of D1/D5 agonists.

Cholinergic Modulation of Dendritic Signaling in Hippocampal GABAergic Inhibitory Interneurons

Dendrites represent the "reception hub" of the neuron as they collect thousands of different inputs and send a coherent response to the cell body. A considerable portion of these signals, especially in vivo, arises from neuromodulatory sources, which affect dendritic computations and cellular activity. In this context, acetylcholine (ACh) exerts a coordinating role of different brain structures, contributing to goal-driven behaviors and sleep-wake cycles. Specifically, cholinergic neurons from the medial septum-diagonal band of Broca complex send numerous projections to glutamatergic principal cells and GABAergic inhibitory neurons in the hippocampus, differentially entraining them during network oscillations. Interneurons display abundant expression of cholinergic receptors and marked responses to stimulation by ACh. Nonetheless, the precise localization of ACh inputs is largely unknown, and evidence for cholinergic modulation of interneuronal dendritic signaling remains elusive. In this article, we review evidence that suggests modulatory effects of ACh on dendritic computations in three hippocampal interneuron subtypes: fast-spiking parvalbumin-positive (PV⁺) cells, somatostatin-expressing (SOM⁺) oriens lacunosum moleculare cells and vasoactive intestinal polypeptide-expressing (VIP⁺) interneuron-selective interneurons. We consider the distribution of cholinergic receptors on these interneurons, including information about their specific somatodendritic location, and discuss how the action of these receptors can modulate dendritic Ca²⁺ signaling and activity of interneurons. The implications of ACh-dependent Ca²⁺ signaling for dendritic plasticity are also discussed. We propose that cholinergic modulation can shape the dendritic integration and plasticity in interneurons in a cell type-specific manner, and the elucidation of these mechanisms will be required to understand the contribution of each cell type to large-scale network activity.

The functional role of sequentially neuromodulated synaptic plasticity in behavioural learning

To survive, animals have to quickly modify their behaviour when the reward changes. The internal representations responsible for this are updated through synaptic weight changes, mediated by certain neuromodulators conveying feedback from the environment. In previous experiments, we discovered a form of hippocampal Spike-Timing-Dependent-Plasticity (STDP) that is sequentially modulated by acetylcholine and dopamine. Acetylcholine facilitates synaptic depression, while dopamine retroactively converts the depression into potentiation. When these experimental findings were implemented as a learning rule in a computational model, our simulations showed that cholinergic-facilitated depression is important for reversal learning. In the present study, we tested the model's prediction by optogenetically inactivating cholinergic neurons in mice during a hippocampus-dependent spatial learning task with changing rewards. We found that reversal learning, but not initial place learning, was impaired, verifying our computational prediction that acetylcholine-modulated plasticity promotes the unlearning of old reward locations. Further, differences in neuromodulator concentrations in the model captured mouse-by-mouse performance variability in the optogenetic experiments. Our line of work sheds light on how neuromodulators enable the learning of new contingencies.

Dopaminergic Neuromodulation of Spike Timing Dependent Plasticity in Mature Adult Rodent and Human Cortical Neurons

Synapses in the cerebral cortex constantly change and this dynamic property regulated by the action of neuromodulators such as dopamine (DA), is essential for reward learning and memory. DA modulates spike-timing-dependent plasticity (STDP), a cellular model of learning and memory, in juvenile rodent cortical neurons. However, it is unknown whether this neuromodulation also occurs at excitatory synapses of cortical neurons in mature adult mice or in humans. Cortical layer V pyramidal neurons were recorded with whole cell patch clamp electrophysiology and an extracellular stimulating electrode was used to induce STDP. DA was either bath-applied or optogenetically released in slices from mice. Classical STDP induction protocols triggered non-hebbian excitatory synaptic depression in the mouse or no plasticity at human cortical synapses. DA reverted long term synaptic depression to baseline in mouse via dopamine 2 type receptors or elicited long term synaptic potentiation in human cortical synapses. Furthermore, when DA was applied during an STDP protocol it depressed presynaptic inhibition in the mouse but not in the human cortex. Thus, DA modulates excitatory synaptic plasticity differently in human vs. mouse cortex. The data strengthens the importance of DA in gating cognition in humans, and may inform on therapeutic interventions to recover brain function from diseases.

Frozen algorithms: how the brain's wiring facilitates learning

Synapses and neural connectivity are plastic and shaped by experience. But to what extent does connectivity itself influence the ability of a neural circuit to learn? Insights from optimization theory and AI shed light on how learning can be implemented in neural circuits. Though abstract in their nature, learning algorithms provide a principled set of hypotheses on the necessary ingredients for learning in neural circuits. These include the kinds of signals and circuit motifs that enable learning from experience, as well as an appreciation of the constraints that make learning challenging in a biological setting. Remarkably, some simple connectivity patterns can boost the efficiency of relatively crude learning rules, showing how the brain can use anatomy to compensate for the biological constraints of known synaptic plasticity mechanisms. Modern connectomics provides rich data for exploring this principle, and may reveal how brain connectivity is constrained by the requirement to learn efficiently.

Learning precise spatiotemporal sequences via biophysically realistic learning rules in a modular, spiking network

Multiple brain regions are able to learn and express temporal sequences, and this functionality is an essential component of learning and memory. We propose a substrate for such representations via a network model that learns and recalls discrete sequences of variable order and duration. The model consists of a network of spiking neurons placed in a modular microcolumn based architecture. Learning is performed via a biophysically realistic learning rule that depends on synaptic 'eligibility traces'. Before training, the network contains no memory of any particular sequence. After training, presentation of only the first element in that sequence is sufficient for the network to recall an entire learned representation of the sequence. An extended version of the model also demonstrates the ability to successfully learn and recall non-Markovian sequences. This model provides a possible framework for biologically plausible sequence learning and memory, in agreement with recent experimental results.

Prioritized experience replays on a hippocampal predictive map for learning

The hippocampus is crucial for spatial learning but their neuronal mechanisms remain unknown. Here, we found that hippocampal cell ensembles preferentially replayed salient reward-related locations as rats learned a new behavioral trajectory for reward. The contents of these hippocampal replays progressively varied with learning. Especially during learning, hippocampal neurons even replayed an optimized path that had never been exploited by the animals. The learning-related hippocampal activity was necessary for the stabilization of spatial behavior. These results suggest prioritized replays of significant experiences on a predictive map by hippocampal neurons.

Hippocampal cells are central to spatial and predictive representations, and experience replays by place cells are crucial for learning and memory. Nonetheless, how hippocampal replay patterns dynamically change during the learning process remains to be elucidated. Here, we designed a spatial task in which rats learned a new behavioral trajectory for reward. We found that as rats updated their behavioral strategies for a novel salient location, hippocampal cell ensembles increased theta-sequences and sharp wave ripple-associated synchronous spikes that preferentially replayed salient locations and reward-related contexts in reverse order. The directionality and contents of the replays progressively varied with learning, including an optimized path that had never been exploited by the animals, suggesting prioritized replays of significant experiences on a predictive map. Online feedback blockade of sharp wave ripples during a learning process inhibited stabilizing optimized behavior. These results implicate learning-associated experience replays that act to learn and reinforce specific behavioral strategies.

Muscarinic Regulation of Spike Timing Dependent Synaptic Plasticity in the Hippocampus

Long-term changes in synaptic transmission between neurons in the brain are considered the cellular basis of learning and memory. Over the last few decades, many studies have revealed that the precise order and timing of activity between pre- and post-synaptic cells ("spike-timing-dependent plasticity; STDP") is crucial for the sign and magnitude of long-term changes at many central synapses. Acetylcholine (ACh) via the recruitment of diverse muscarinic receptors is known to influence STDP in a variety of ways, enabling flexibility and adaptability in brain network activity during complex behaviors. In this review, we will summarize and discuss different mechanistic aspects of muscarinic modulation of timing-dependent plasticity at both excitatory and inhibitory synapses in the hippocampus to shape learning and memory.

Dopamine agonist treatment increases sensitivity to gamble outcomes in the hippocampus in de novo Parkinson's disease

BACKGROUND

Parkinson's disease is associated with severe nigro-striatal dopamine depletion, leading to motor dysfunction and altered reward processing. We previously showed that drug-naïve patients with Parkinson's disease had a consistent attenuation of reward signalling in the mesolimbic and mesocortical system. Here, we address the neurobiological effects of dopaminergic therapy on reward sensitivity in the mesolimbic circuitry, and how this may contribute to neuropsychiatric symptoms.

OBJECTIVES

We tested the hypothesis that (1) dopaminergic treatment would restore the attenuated, mesolimbic and mesocortical responses to reward; and (2) restoration of reward responsivity by dopaminergic treatment would predict motor performance and the emergence of impulse control symptoms.

METHODS

In 11 drug-naïve Parkinson patients, we prospectively assessed treatment-induced changes in reward processing before, and eight weeks after initiation of monotherapy with dopamine agonists. They were compared to 10 non-medicated healthy controls who were also measured longitudinally. We used whole-brain functional magnetic resonance imaging at 3 Tesla to assess the reward responsivity of the brain to monetary gains and losses, while participants performed a simple consequential gambling task.

RESULTS

In patients, dopaminergic treatment improved clinical motor symptoms without significantly changing task performance. Dopamine agonist therapy induced a stronger reward responsivity in the right hippocampus with higher doses being less effective. None of the patients developed impulse control disorders in the follow-up period of four years.

CONCLUSIONS

Short-term treatment with first-ever dopaminergic medication partially restores deficient reward-related processing in the hippocampus in de novo Parkinson's disease.

Synaptic Plasticity Forms and Functions

Synaptic plasticity, the activity-dependent change in neuronal connection strength, has long been considered an important component of learning and memory. Computational and engineering work corroborate the power of learning through the directed adjustment of connection weights. Here we review the fundamental elements of four broadly categorized forms of synaptic plasticity and discuss their functional capabilities and limitations. Although standard, correlation-based, Hebbian synaptic plasticity has been the primary focus of neuroscientists for decades, it is inherently limited. Three-factor plasticity rules supplement Hebbian forms with neuromodulation and eligibility traces, while true supervised types go even further by adding objectives and instructive signals. Finally, a recently discovered hippocampal form of synaptic plasticity combines the above elements, while leaving behind the primary Hebbian requirement. We suggest that the effort to determine the neural basis of adaptive behavior could benefit from renewed experimental and theoretical investigation of more powerful directed types of synaptic plasticity.

Distinct effects of reward and navigation history on hippocampal forward and reverse replays

To better understand the functional roles of hippocampal forward and reverse replays, we trained rats in a spatial sequence memory task and examined how these replays are modulated by reward and navigation history. We found that reward enhances both forward and reverse replays during the awake state, but in different ways. Reward enhances the rate of reverse replays, but it increases the fidelity of forward replays for recently traveled as well as other alternative trajectories heading toward a rewarding location. This suggests roles for forward and reverse replays in reinforcing representations for all potential rewarding trajectories. We also found more faithful reactivation of upcoming than already rewarded trajectories in forward replays. This suggests a role for forward replays in preferentially reinforcing representations for high-value trajectories. We propose that hippocampal forward and reverse replays might contribute to constructing a map of potential navigation trajectories and their associated values (a "value map") via distinct mechanisms.

One-shot learning and behavioral eligibility traces in sequential decision making

In many daily tasks, we make multiple decisions before reaching a goal. In order to learn such sequences of decisions, a mechanism to link earlier actions to later reward is necessary. Reinforcement learning (RL) theory suggests two classes of algorithms solving this credit assignment problem: In classic temporal-difference learning, earlier actions receive reward information only after multiple repetitions of the task, whereas models with eligibility traces reinforce entire sequences of actions from a single experience (one-shot). Here, we show one-shot learning of sequences. We developed a novel paradigm to directly observe which actions and states along a multi-step sequence are reinforced after a single reward. By focusing our analysis on those states for which RL with and without eligibility trace make qualitatively distinct predictions, we find direct behavioral (choice probability) and physiological (pupil dilation) signatures of reinforcement learning with eligibility trace across multiple sensory modalities.

Rewards interact with repetition-dependent learning to enhance long-term retention of motor memories

The combination of behavioral experiences that enhance long-term retention remains largely unknown. Informed by neurophysiological lines of work, this study tested the hypothesis that performance-contingent monetary rewards potentiate repetition-dependent forms of learning, as induced by extensive practice at asymptote, to enhance long-term retention of motor memories. To this end, six groups of 14 participants (n = 84) acquired novel motor behaviors by adapting to a gradual visuomotor rotation while these factors were manipulated. Retention was assessed 24 h later. While all groups similarly acquired the novel motor behaviors, results from the retention session revealed an interaction indicating that rewards enhanced long-term retention, but only when practice was extended to asymptote. Specifically, the interaction indicated that this effect selectively occurred when rewards were intermittently available (i.e., 50%), but not when they were absent (i.e., 0%) or continuously available (i.e., 100%) during acquisition. This suggests that the influence of rewards on extensive practice and long-term retention is nonlinear, as continuous rewards did not further enhance retention as compared with intermittent rewards. One possibility is that rewards' intermittent availability allows to maintain their subjective value during acquisition, which may be key to potentiate long-term retention.

SpykeTorch: Efficient Simulation of Convolutional Spiking Neural Networks With at Most One Spike per Neuron

Application of deep convolutional spiking neural networks (SNNs) to artificial intelligence (AI) tasks has recently gained a lot of interest since SNNs are hardware-friendly and energy-efficient. Unlike the non-spiking counterparts, most of the existing SNN simulation frameworks are not practically efficient enough for large-scale AI tasks. In this paper, we introduce SpykeTorch, an open-source high-speed simulation framework based on PyTorch. This framework simulates convolutional SNNs with at most one spike per neuron and the rank-order encoding scheme. In terms of learning rules, both spike-timing-dependent plasticity (STDP) and reward-modulated STDP (R-STDP) are implemented, but other rules could be implemented easily. Apart from the aforementioned properties, SpykeTorch is highly generic and capable of reproducing the results of various studies. Computations in the proposed framework are tensor-based and totally done by PyTorch functions, which in turn brings the ability of just-in-time optimization for running on CPUs, GPUs, or Multi-GPU platforms.