A saturation hypothesis to explain both enhanced and impaired learning with enhanced plasticity

Energy-aware bio-inspired spiking reinforcement learning system architecture for real-time autonomous edge applications

Mobile, low-cost, and energy-aware operation of Artificial Intelligence (AI) computations in smart circuits and autonomous robots will play an important role in the next industrial leap in intelligent automation and assistive devices. Neuromorphic hardware with spiking neural network (SNN) architecture utilizes insights from biological phenomena to offer encouraging solutions. Previous studies have proposed reinforcement learning (RL) models for SNN responses in the rat hippocampus to an environment where rewards depend on the context. The scale of these models matches the scope and capacity of small embedded systems in the framework of Internet-of-Bodies (IoB), autonomous sensor nodes, and other edge applications. Addressing energy-efficient artificial learning problems in such systems enables smart micro-systems with edge intelligence. A novel bio-inspired RL system architecture is presented in this work, leading to significant energy consumption benefits without foregoing real-time autonomous processing and accuracy requirements of the context-dependent task. The hardware architecture successfully models features analogous to synaptic tagging, changes in the exploration schemes, synapse saturation, and spatially localized task-based activation observed in the brain. The design has been synthesized, simulated, and tested on Intel MAX10 Field-Programmable Gate Array (FPGA). The problem-based bio-inspired approach to SNN edge architectural design results in 25X reduction in average power compared to the state-of-the-art for a test with real-time context learning and 30 trials. Furthermore, 940x lower energy consumption is achieved due to improvement in the execution time.

Restoring cerebellar-dependent learning

Behavioral and pharmaceutical interventions reverse defects associated with increased cerebellar long-term depression in a mouse model of Fragile X syndrome.

Systemic pharmacological suppression of neural activity reverses learning impairment in a mouse model of Fragile X syndrome

The enhancement of associative synaptic plasticity often results in impaired rather than enhanced learning. Previously, we proposed that such learning impairments can result from saturation of the plasticity mechanism (Nguyen-Vu et al., 2017), or, more generally, from a history-dependent change in the threshold for plasticity. This hypothesis was based on experimental results from mice lacking two class I major histocompatibility molecules, MHCI H2-Kb and H2-Db (MHCI KbDb-/-), which have enhanced associative long-term depression at the parallel fiber-Purkinje cell synapses in the cerebellum (PF-Purkinje cell LTD). Here, we extend this work by testing predictions of the threshold metaplasticity hypothesis in a second mouse line with enhanced PF-Purkinje cell LTD, the Fmr1 knockout mouse model of Fragile X syndrome (FXS). Mice lacking Fmr1 gene expression in cerebellar Purkinje cells (L7-Fmr1 KO) were selectively impaired on two oculomotor learning tasks in which PF-Purkinje cell LTD has been implicated, with no impairment on LTD-independent oculomotor learning tasks. Consistent with the threshold metaplasticity hypothesis, behavioral pre-training designed to reverse LTD at the PF-Purkinje cell synapses eliminated the oculomotor learning deficit in the L7-Fmr1 KO mice, as previously reported in MHCI KbDb-/-mice. In addition, diazepam treatment to suppress neural activity and thereby limit the induction of associative LTD during the pre-training period also eliminated the learning deficits in L7-Fmr1 KO mice. These results support the hypothesis that cerebellar LTD-dependent learning is governed by an experience-dependent sliding threshold for plasticity. An increased threshold for LTD in response to elevated neural activity would tend to oppose firing rate stability, but could serve to stabilize synaptic weights and recently acquired memories. The metaplasticity perspective could inform the development of new clinical approaches for addressing learning impairments in autism and other disorders of the nervous system.

Enhancement of Motor Learning and Corticospinal Excitability: The Role of Electroacupuncture and Motor Training in Healthy Volunteers

BACKGROUND This study embarked on an innovative exploration to elucidate the effects of integrating electroacupuncture (EA) with motor training (MT) on enhancing corticospinal excitability and motor learning. Central to this investigation is the interplay between homeostatic and non-homeostatic metaplasticity processes, providing insights into how these combined interventions may influence neural plasticity and motor skill acquisition. MATERIAL AND METHODS The investigation enrolled 20 healthy volunteers, subjecting them to 4 distinct interventions to parse out the individual and combined effects of EA and MT. These interventions were EA alone, MT alone, EA-priming followed by MT, and MT-priming followed by EA. The assessment of changes in primary motor cortex (M1) excitability was conducted through motor-evoked potentials (MEPs), while the grooved pegboard test (GPT) was used to evaluate alterations in motor performance. RESULTS The findings revealed that EA and MT independently contributed to enhanced M1 excitability and motor performance. However, the additional priming with EA or MT did not yield further modulation in MEPs amplitudes. Notably, EA-priming was associated with improved GPT completion times, underscoring its potential in facilitating motor learning. CONCLUSIONS The study underscores that while EA and MT individually augment motor cortex excitability and performance, their synergistic application does not further enhance or inhibit cortical excitability. This points to the involvement of non-homeostatic metaplasticity mechanisms. Nonetheless, EA emerges as a critical tool in preventing M1 overstimulation, thereby continuously fostering motor learning. The findings call for further research into the strategic application of EA, whether in isolation or with MT, within clinical settings to optimize rehabilitation outcomes.

Interference underlies attenuation upon relearning in sensorimotor adaptation

Savings refers to the gain in performance upon relearning a task. In sensorimotor adaptation, savings is tested by having participants adapt to perturbed feedback and, following a washout block during which the system resets to baseline, presenting the same perturbation again. While savings has been observed with these tasks, we have shown that the contribution from implicit sensorimotor adaptation, a process that uses sensory prediction errors to recalibrate the sensorimotor map, is actually attenuated upon relearning (Avraham et al., 2021). In the present study, we test the hypothesis that this attenuation is due to interference arising from the washout block, and more generally, from experience with a different relationship between the movement and the feedback. In standard adaptation studies, removing the perturbation at the start of the washout block results in a salient error signal in the opposite direction to that observed during learning. As a starting point, we replicated the finding that implicit adaptation is attenuated following a washout period in which the feedback now signals a salient opposite error. When we eliminated visual feedback during washout, implicit adaptation was no longer attenuated upon relearning, consistent with the interference hypothesis. Next, we eliminated the salient error during washout by gradually decreasing the perturbation, creating a scenario in which the perceived errors fell within the range associated with motor noise. Nonetheless, attenuation was still prominent. Inspired by this observation, we tested participants with an extended experience with veridical feedback during an initial baseline phase and found that this was sufficient to cause robust attenuation of implicit adaptation during the first exposure to the perturbation. This effect was context-specific: It did not generalize to movements that were not associated with the interfering feedback. Taken together, these results show that the implicit sensorimotor adaptation system is highly sensitive to memory interference from a recent experience with a discrepant action-outcome contingency.

Systemic pharmacological suppression of neural activity reverses learning impairment in a mouse model of Fragile X syndrome

The enhancement of associative synaptic plasticity often results in impaired rather than enhanced learning. Previously, we proposed that such learning impairments can result from saturation of the plasticity mechanism (Nguyen-Vu et al., 2017), or, more generally, from a history-dependent change in the threshold for plasticity. This hypothesis was based on experimental results from mice lacking two class I major histocompatibility molecules, MHCI H2-Kb and H2Db (MH-CI KbDb-/-), which have enhanced associative long-term depression at the parallel fiber-Purkinje cell synapses in the cerebellum (PF-Purkinje cell LTD). Here, we extend this work by testing predictions of the threshold metaplasticity hypothesis in a second mouse line with enhanced PF-Purkinje cell LTD, the Fmr1 knockout mouse model of Fragile X syndrome (FXS). Mice lacking Fmr1 gene expression in cerebellar Purkinje cells (L7-Fmr1 KO) were selectively impaired on two oculomotor learning tasks in which PF-Purkinje cell LTD has been implicated, with no impairment on LTD-independent oculomotor learning tasks. Consistent with the threshold metaplasticity hypothesis, behavioral pre-training designed to reverse LTD at the PF-Purkinje cell synapses eliminated the oculomotor learning deficit in the L7-Fmr1 KO mice, as previously reported in MHCI KbDb-/-mice. In addition, diazepam treatment to suppress neural activity and thereby limit the induction of associative LTD during the pre-training period also eliminated the learning deficits in L7-Fmr1 KO mice. These results support the hypothesis that cerebellar LTD-dependent learning is governed by an experience-dependent sliding threshold for plasticity. An increased threshold for LTD in response to elevated neural activity would tend to oppose firing rate stability, but could serve to stabilize synaptic weights and recently acquired memories. The metaplasticity perspective could inform the development of new clinical approaches for addressing learning impairments in autism and other disorders of the nervous system.

Glutamate-specific gene linked to human brain evolution enhances synaptic plasticity and cognitive processes

The human brain is characterized by the upregulation of synaptic, mainly glutamatergic, transmission, but its evolutionary origin(s) remain elusive. Here we approached this fundamental question by studying mice transgenic (Tg) for GLUD2, a human gene involved in glutamate metabolism that emerged in the hominoid and evolved concomitantly with brain expansion. We demonstrate that Tg mice express the human enzyme in hippocampal astrocytes and CA1-CA3 pyramidal neurons. LTP, evoked by theta-burst stimulation, is markedly enhanced in the CA3-CA1 synapses of Tg mice, with patch-clamp recordings from CA1 pyramidal neurons revealing increased sNMDA currents. LTP enhancement is blocked by D-lactate, implying that GLUD2 potentiates L-lactate-mediated astrocyte-neuron interaction. Dendritic spine density and synaptogenesis are increased in the hippocampus of Tg mice, which exhibit enhanced responses to sensory stimuli and improved performance on complex memory tasks. Hence, GLUD2 likely contributed to human brain evolution by enhancing synaptic plasticity and metabolic processes central to cognitive functions.

Molecular layer disinhibition unlocks climbing-fiber-instructed motor learning in the cerebellum

Climbing fibers supervise cerebellar learning by providing signals to Purkinje cells (PCs) that instruct adaptive changes to mistakenly performed movements. Yet, climbing fibers are regularly active, even during well performed movements, suggesting that a mechanism dynamically regulates the ability of climbing fibers to induce corrective plasticity in response to motor errors. We found that molecular layer interneurons (MLIs), whose inhibition of PCs powerfully opposes climbing-fiber-mediated excitation, serve this function. Optogenetically suppressing the activity of floccular MLIs in mice during the vestibulo-ocular reflex (VOR) induces a learned increase in gain despite the absence of performance errors. Suppressing MLIs when the VOR is mistakenly underperformed reveled that their inhibitory output is necessary to orchestrate gain-increase learning by conditionally permitting climbing fibers to instruct plasticity induction during ipsiversive head turns. Ablation of an MLI circuit for PC disinhibition prevents gain-increase learning during VOR performance errors which was rescued by re-imposing PC disinhibition through MLI activity suppression. Our findings point to a decisive role for MLIs in gating climbing-fiber-mediated learning through their context-dependent inhibition of PCs.

Increased paired stimuli enhance corticospinal-motoneuronal plasticity in humans with spinal cord injury

Paired corticospinal-motoneuronal stimulation (PCMS) has been used to enhance corticospinal excitability and functional outcomes in humans with spinal cord injury (SCI). Here, we examined the effect of increasing the number of paired pulses on PCMS-induced plasticity. During PCMS, corticospinal volleys evoked by transcranial magnetic stimulation (TMS) over the hand motor cortex were timed to arrive at corticospinal-motoneuronal synapses of the first dorsal interosseous (FDI) muscle 1-2 ms before the arrival of antidromic potentials elicited in motoneurons by electrical stimulation of the ulnar nerve. We tested motor-evoked potentials (MEPs) elicited by TMS over the hand motor cortex and electrical stimulation at the cervicomedullary junction (CMEPs) in the FDI muscle before and after 180 paired pulses (PCMS-180) followed up by another 180 paired pulses (PCMS-360) in humans with and without chronic incomplete cervical SCI. The nine-hole-peg-test (9HPT) was measured before and after PCMS paired pulses in individuals with SCI. We found that the size of MEPs and CMEPs increased after PCMS-180 in both groups compared with baseline and further increased after PCMS-360 in participants with SCI, suggesting a spinal origin for these effects. Notably, in people with SCI, the time to complete the 9HPT decreased after PCMS-180 and further decreased after PCMS-360 compared with baseline but not when the 9HPT was repeated overtime. Our findings demonstrate that increasing the number of PCMS paired pulses potentiates corticospinal excitability and voluntary motor output after SCI, likely through spinal plasticity. This proof-of-principle study suggests that increasing the PCMS dose represents a strategy to boost voluntary motor output after SCI.NEW & NOTEWORTHY Paired corticospinal-motoneuronal stimulation (PCMS) has been used to enhance corticospinal excitability and functional outcomes in humans with spinal cord injury (SCI). Here, we demonstrate that 360 paired pulses resulted in larger increases in motor-evoked potential size in a hand muscle and in a better ability to complete the nine-hold-peg-test compared with 180 paired pulses in people with SCI. This proof-of-principle study suggests that increasing the PCMS dose represents a strategy to boost motor output after SCI.

Computational epidemiology study of homeostatic compensation during sensorimotor aging

The vestibulo-ocular reflex (VOR) stabilizes vision during head motion. Age-related changes of vestibular neuroanatomical properties predict a linear decay of VOR function. Nonetheless, human epidemiological data show a stable VOR function across the life span. In this study, we model cerebellum-dependent VOR adaptation to relate structural and functional changes throughout aging. We consider three neurosynaptic factors that may codetermine VOR adaptation during aging: the electrical coupling of inferior olive neurons, the long-term spike timing-dependent plasticity at parallel fiber - Purkinje cell synapses and mossy fiber - medial vestibular nuclei synapses, and the intrinsic plasticity of Purkinje cell synapses Our cross-sectional aging analyses suggest that long-term plasticity acts as a global homeostatic mechanism that underpins the stable temporal profile of VOR function. The results also suggest that the intrinsic plasticity of Purkinje cell synapses operates as a local homeostatic mechanism that further sustains the VOR at older ages. Importantly, the computational epidemiology approach presented in this study allows discrepancies among human cross-sectional studies to be understood in terms of interindividual variability in older individuals. Finally, our longitudinal aging simulations show that the amount of residual fibers coding for the peak and trough of the VOR cycle constitutes a predictive hallmark of VOR trajectories over a lifetime.

Lack of the peroxiredoxin 6 gene causes impaired spatial memory and abnormal synaptic plasticity

Peroxiredoxin 6 (PRDX6) is expressed dominantly in the astrocytes and exerts either neuroprotective or neurotoxic effects in the brain. Although PRDX6 can modulate several signaling cascades involving cognitive functions, its physiological role in spatial memory has not been investigated yet. This study aims to explore the function of the Prdx6 gene in spatial memory formation and synaptic plasticity. We first tested Prdx6-/- mice on a Morris water maze task and found that their memory performance was defective, along with reduced long-term potentiation (LTP) in CA3-CA1 hippocampal synapses recorded from hippocampal sections of home-caged mice. Surprisingly, after the probe test, these knockout mice exhibited elevated hippocampal LTP, higher phosphorylated ERK1/2 level, and decreased reactive astrocyte markers. We further reduced ERK1/2 phosphorylation by administering MEK inhibitor, U0126, into Prdx6-/- mice before the probe test, which reversed their spatial memory deficit. This study is the first one to report the role of PRDX6 in spatial memory and synaptic plasticity. Our results revealed that PRDX6 is necessary for maintaining spatial memory by modulating ERK1/2 phosphorylation and astrocyte activation.

Reexposure to a sensorimotor perturbation produces opposite effects on explicit and implicit learning processes

The motor system demonstrates an exquisite ability to adapt to changes in the environment and to quickly reset when these changes prove transient. If similar environmental changes are encountered in the future, learning may be faster, a phenomenon known as savings. In studies of sensorimotor learning, a central component of savings is attributed to the explicit recall of the task structure and appropriate compensatory strategies. Whether implicit adaptation also contributes to savings remains subject to debate. We tackled this question by measuring, in parallel, explicit and implicit adaptive responses in a visuomotor rotation task, employing a protocol that typically elicits savings. While the initial rate of learning was faster in the second exposure to the perturbation, an analysis decomposing the 2 processes showed the benefit to be solely associated with explicit re-aiming. Surprisingly, we found a significant decrease after relearning in aftereffect magnitudes during no-feedback trials, a direct measure of implicit adaptation. In a second experiment, we isolated implicit adaptation using clamped visual feedback, a method known to eliminate the contribution of explicit learning processes. Consistent with the results of the first experiment, participants exhibited a marked reduction in the adaptation function, as well as an attenuated aftereffect when relearning from the clamped feedback. Motivated by these results, we reanalyzed data from prior studies and observed a consistent, yet unappreciated pattern of attenuation of implicit adaptation during relearning. These results indicate that explicit and implicit sensorimotor processes exhibit opposite effects upon relearning: Explicit learning shows savings, while implicit adaptation becomes attenuated.

Computational Principles of Supervised Learning in the Cerebellum

Supervised learning plays a key role in the operation of many biological and artificial neural networks. Analysis of the computations underlying supervised learning is facilitated by the relatively simple and uniform architecture of the cerebellum, a brain area that supports numerous motor, sensory, and cognitive functions. We highlight recent discoveries indicating that the cerebellum implements supervised learning using the following organizational principles: ( a) extensive preprocessing of input representations (i.e., feature engineering), ( b) massively recurrent circuit architecture, ( c) linear input-output computations, ( d) sophisticated instructive signals that can be regulated and are predictive, ( e) adaptive mechanisms of plasticity with multiple timescales, and ( f) task-specific hardware specializations. The principles emerging from studies of the cerebellum have striking parallels with those in other brain areas and in artificial neural networks, as well as some notable differences, which can inform future research on supervised learning and inspire next-generation machine-based algorithms.

Depressed by Learning-Heterogeneity of the Plasticity Rules at Parallel Fiber Synapses onto Purkinje Cells

Climbing fiber-driven long-term depression (LTD) of parallel fiber synapses onto cerebellar Purkinje cells has long been investigated as a putative mechanism of motor learning. We recently discovered that the rules governing the induction of LTD at these synapses vary across different regions of the cerebellum. Here, we discuss the design of LTD induction protocols in light of this heterogeneity in plasticity rules. The analytical advantages of the cerebellum provide an opportunity to develop a deeper understanding of how the specific plasticity rules at synapses support the implementation of learning.

Inhibition gates supralinear Ca2+ signaling in Purkinje cell dendrites during practiced movements

Motor learning involves neural circuit modifications in the cerebellar cortex, likely through re-weighting of parallel fiber inputs onto Purkinje cells (PCs). Climbing fibers instruct these synaptic modifications when they excite PCs in conjunction with parallel fiber activity, a pairing that enhances climbing fiber-evoked Ca²⁺ signaling in PC dendrites. In vivo, climbing fibers spike continuously, including during movements when parallel fibers are simultaneously conveying sensorimotor information to PCs. Whether parallel fiber activity enhances climbing fiber Ca²⁺ signaling during motor behaviors is unknown. In mice, we found that inhibitory molecular layer interneurons (MLIs), activated by parallel fibers during practiced movements, suppressed parallel fiber enhancement of climbing fiber Ca²⁺ signaling in PCs. Similar results were obtained in acute slices for brief parallel fiber stimuli. Interestingly, more prolonged parallel fiber excitation revealed latent supralinear Ca²⁺ signaling. Therefore, the balance of parallel fiber and MLI input onto PCs regulates concomitant climbing fiber Ca²⁺ signaling.

The Emerging Concept of Intrinsic Plasticity: Activity-dependent Modulation of Intrinsic Excitability in Cerebellar Purkinje Cells and Motor Learning

What is memory? How does the brain process the sensory information and modify an organism's behavior? Many neuroscientists have focused on the activity- and experience-dependent modifications of synaptic functions in order to solve these fundamental questions in neuroscience. Recently, the plasticity of intrinsic excitability (called intrinsic plasticity) has emerged as an important element for information processing and storage in the brain. As the cerebellar Purkinje cells are the sole output neurons in the cerebellar cortex and the information is conveyed from a neuron to its relay neurons by forms of action potential firing, the modulation of the intrinsic firing activity may play a critical role in the cerebellar learning. Many voltage-gated and/or Ca2+-activated ion channels are involved in shaping the spiking output as well as integrating synaptic inputs to finely tune the cerebellar output. Recent studies suggested that the modulation of the intrinsic excitability and its plasticity in the cerebellar Purkinje cells might function as an integrator for information processing and memory formation. Moreover, the intrinsic plasticity might also determine the strength of connectivity to the sub-cortical areas such as deep cerebellar nuclei and vestibular nuclei to trigger the consolidation of the cerebellar-dependent memory by transferring the information.

Occurrence of long-term depression in the cerebellar flocculus during adaptation of optokinetic response

Long-term depression (LTD) at parallel fiber (PF) to Purkinje cell (PC) synapses has been considered as a main cellular mechanism for motor learning. However, the necessity of LTD for motor learning was challenged by demonstration of normal motor learning in the LTD-defective animals. Here, we addressed possible involvement of LTD in motor learning by examining whether LTD occurs during motor learning in the wild-type mice. As a model of motor learning, adaptation of optokinetic response (OKR) was used. OKR is a type of reflex eye movement to suppress blur of visual image during animal motion. OKR shows adaptive change during continuous optokinetic stimulation, which is regulated by the cerebellar flocculus. After OKR adaptation, amplitudes of quantal excitatory postsynaptic currents at PF-PC synapses were decreased, and induction of LTD was suppressed in the flocculus. These results suggest that LTD occurs at PF-PC synapses during OKR adaptation.