Reward signals in the cerebellum: origins, targets, and functional implications

Consensus Paper: Cerebellum and Reward

Cerebellum is a key-structure for the modulation of motor, cognitive, social and affective functions, contributing to automatic behaviours through interactions with the cerebral cortex, basal ganglia and spinal cord. The predictive mechanisms used by the cerebellum cover not only sensorimotor functions but also reward-related tasks. Cerebellar circuits appear to encode temporal difference error and reward prediction error. From a chemical standpoint, cerebellar catecholamines modulate the rate of cerebellar-based cognitive learning, and mediate cerebellar contributions during complex behaviours. Reward processing and its associated emotions are tuned by the cerebellum which operates as a controller of adaptive homeostatic processes based on interoceptive and exteroceptive inputs. Lobules VI-VII/areas of the vermis are candidate regions for the cortico-subcortical signaling pathways associated with loss aversion and reward sensitivity, together with other nodes of the limbic circuitry. There is growing evidence that the cerebellum works as a hub of regional dysconnectivity across all mood states and that mental disorders involve the cerebellar circuitry, including mood and addiction disorders, and impaired eating behaviors where the cerebellum might be involved in longer time scales of prediction as compared to motor operations. Cerebellar patients exhibit aberrant social behaviour, showing aberrant impulsivity/compulsivity. The cerebellum is a master-piece of reward mechanisms, together with the striatum, ventral tegmental area (VTA) and prefrontal cortex (PFC). Critically, studies on reward processing reinforce our view that a fundamental role of the cerebellum is to construct internal models, perform predictions on the impact of future behaviour and compare what is predicted and what actually occurs.

A role for the cerebellum in motor-triggered alleviation of anxiety

Physical exercise is known to reduce anxiety, but the underlying brain mechanisms remain unclear. Here, we explore a hypothalamo-cerebello-amygdalar circuit that may mediate motor-dependent alleviation of anxiety. This three-neuron loop, in which the cerebellar dentate nucleus takes center stage, bridges the motor system with the emotional system. Subjecting animals to a constant rotarod engages glutamatergic cerebellar dentate neurons that drive PKCδ+ amygdalar neurons to elicit an anxiolytic effect. Moreover, challenging animals on an accelerated rather than a constant rotarod engages hypothalamic neurons that provide a superimposed anxiolytic effect via an orexinergic projection to the dentate neurons that activate the amygdala. Our findings reveal a cerebello-limbic pathway that may contribute to motor-triggered alleviation of anxiety and that may be optimally exploited during challenging physical exercise.

Cerebellum Lecture: the Cerebellar Nuclei-Core of the Cerebellum

The cerebellum is a key player in many brain functions and a major topic of neuroscience research. However, the cerebellar nuclei (CN), the main output structures of the cerebellum, are often overlooked. This neglect is because research on the cerebellum typically focuses on the cortex and tends to treat the CN as relatively simple output nuclei conveying an inverted signal from the cerebellar cortex to the rest of the brain. In this review, by adopting a nucleocentric perspective we aim to rectify this impression. First, we describe CN anatomy and modularity and comprehensively integrate CN architecture with its highly organized but complex afferent and efferent connectivity. This is followed by a novel classification of the specific neuronal classes the CN comprise and speculate on the implications of CN structure and physiology for our understanding of adult cerebellar function. Based on this thorough review of the adult literature we provide a comprehensive overview of CN embryonic development and, by comparing cerebellar structures in various chordate clades, propose an interpretation of CN evolution. Despite their critical importance in cerebellar function, from a clinical perspective intriguingly few, if any, neurological disorders appear to primarily affect the CN. To highlight this curious anomaly, and encourage future nucleocentric interpretations, we build on our review to provide a brief overview of the various syndromes in which the CN are currently implicated. Finally, we summarize the specific perspectives that a nucleocentric view of the cerebellum brings, move major outstanding issues in CN biology to the limelight, and provide a roadmap to the key questions that need to be answered in order to create a comprehensive integrated model of CN structure, function, development, and evolution.

Dopamine receptors of the rodent fastigial nucleus support skilled reaching for goal-directed action

The dopaminergic (DA) system regulates both motor function, and learning and memory. The cerebellum supports motor control and the acquisition of procedural memories, including goal-directed behavior, and is subjected to DA control. Its fastigial nucleus (FN) controls and interprets body motion through space. The expression of dopamine receptors has been reported in the deep cerebellar nuclei of mice. However, the presence of dopamine D1-like (D1R) and D2-like (D2R) receptors in the rat FN has not yet been verified. In this study, we first confirmed that DA receptors are expressed in the FN of adult rats and then targeted these receptors to explore to what extent the FN modulates goal-directed behavior. Immunohistochemical assessment revealed expression of both D1R and D2R receptors in the FN, whereby the medial lateral FN exhibited higher receptor expression compared to the other FN subfields. Bilateral treatment of the FN with a D1R antagonist, prior to a goal-directed pellet-reaching task, significantly impaired task acquisition and decreased task engagement. D2R antagonism only reduced late performance post-acquisition. Once task acquisition had occurred, D1R antagonism had no effect on successful reaching, although it significantly decreased reaching speed, task engagement, and promoted errors. Motor coordination and ambulation were, however, unaffected as neither D1R nor D2R antagonism altered rotarod latencies or distance and velocity in an open field. Taken together, these results not only reveal a novel role for the FN in goal-directed skilled reaching, but also show that D1R expressed in FN regulate this process by modulating motivation for action.

Atlas of exercise-induced brain activation in mice

OBJECTIVES

There is significant interest in uncovering the mechanisms through which exercise enhances cognition, memory, and mood, and lowers the risk of neurodegenerative diseases. In this study, we utilize forced treadmill running and distance-matched voluntary wheel running, coupled with light sheet 3D brain imaging and c-Fos immunohistochemistry, to generate a comprehensive atlas of exercise-induced brain activation in mice.

METHODS

To investigate the effects of exercise on brain activity, we compared whole-brain activation profiles of mice subjected to treadmill running with mice subjected to distance-matched wheel running. Male mice were assigned to one of four groups: a) an acute bout of voluntary wheel running, b) confinement to a cage with a locked running wheel, c) forced treadmill running, or d) placement on an inactive treadmill. Immediately following each exercise or control intervention, blood samples were collected for plasma analysis, and brains were collected for whole-brain c-Fos quantification.

RESULTS

Our dataset reveals 255 brain regions activated by acute exercise in mice, the majority of which have not previously been linked to exercise. We find a broad response of 140 regulated brain regions that are shared between voluntary wheel running and treadmill running, while 32 brain regions are uniquely regulated by wheel running and 83 brain regions uniquely regulated by treadmill running. In contrast to voluntary wheel running, forced treadmill running triggers activity in brain regions associated with stress, fear, and pain.

CONCLUSIONS

Our findings demonstrate a significant overlap in neuronal activation signatures between voluntary wheel running and distance-matched forced treadmill running. However, our analysis also reveals notable differences and subtle nuances between these two widely used paradigms. The comprehensive dataset is accessible online at www.neuropedia.dk, with the aim of enabling future research directed towards unraveling the neurobiological response to exercise.

Activating Lobule VI PCTH+-Med Pathway in Cerebellum Blocks the Acquisition of Methamphetamine Conditioned Place Preference in Mice

Cerebellum has been implicated in drug addiction; however, its underlying cellular populations and neuronal circuitry remain largely unknown. In the current study, we identified a neural pathway from tyrosine hydroxylase (TH)-positive Purkinje cells (PCTH+) in cerebellar lobule VI to calcium/calmodulin-dependent protein kinase II (CaMKII)-positive glutamatergic neurons in the medial cerebellar nucleus (MedCaMKII), forming the lobule VI PCTH+-MedCaMKII pathway in male mice. In naive male mice, inhibition of PCTH+ neurons activated Med neurons. During conditioned place preference (CPP) training, exposure to methamphetamine (METH) inhibited lobule VI PCTH+ neurons while excited MedCaMKII neurons in mice. Silencing MedCaMKII using a tetanus toxin light chain (tettox) suppressed the acquisition of METH CPP in mice but resulted in motor coordination deficits in naive mice. In contrast, activating lobule VI PCTH+ terminals within Med inhibited the activity of Med neurons and subsequently blocked the acquisition of METH CPP in mice without affecting motor coordination, locomotor activity, and sucrose reinforcements in naive mice. Our findings identified a novel lobule VI PCTH+-MedCaMKII pathway within the cerebellum and explored its role in mediating the acquisition of METH-preferred behaviors.

The effect of reward on motor learning: different stage, different effect

Motor learning is a prominent and extensively studied subject in rehabilitation following various types of neurological disorders. Motor repair and rehabilitation often extend over months and years post-injury with a slow pace of recovery, particularly affecting the fine movements of the distal extremities. This extended period can diminish the motivation and persistence of patients, a facet that has historically been overlooked in motor learning until recent years. Reward, including monetary compensation, social praise, video gaming, music, and virtual reality, is currently garnering heightened attention for its potential to enhance motor motivation and improve function. Numerous studies have examined the effects and attempted to explore potential mechanisms in various motor paradigms, yet they have yielded inconsistent or even contradictory results and conclusions. A comprehensive review is necessary to summarize studies on the effects of rewards on motor learning and to deduce a central pattern from these existing studies. Therefore, in this review, we initially outline a framework of motor learning considering two major types, two major components, and three stages. Subsequently, we summarize the effects of rewards on different stages of motor learning within the mentioned framework and analyze the underlying mechanisms at the level of behavior or neural circuit. Reward accelerates learning speed and enhances the extent of learning during the acquisition and consolidation stages, possibly by regulating the balance between the direct and indirect pathways (activating more D1-MSN than D2-MSN) of the ventral striatum and by increasing motor dynamics and kinematics. However, the effect varies depending on several experimental conditions. During the retention stage, there is a consensus that reward enhances both short-term and long-term memory retention in both types of motor learning, attributed to the LTP learning mechanism mediated by the VTA-M1 dopaminergic projection. Reward is a promising enhancer to bolster waning confidence and motivation, thereby increasing the efficiency of motor learning and rehabilitation. Further exploration of the circuit and functional connections between reward and the motor loop may provide a novel target for neural modulation to promote motor behavior.

Organization of reward and movement signals in the basal ganglia and cerebellum

The basal ganglia and the cerebellum are major subcortical structures in the motor system. The basal ganglia have been cast as the reward center of the motor system, whereas the cerebellum is thought to be involved in adjusting sensorimotor parameters. Recent findings of reward signals in the cerebellum have challenged this dichotomous view. To compare the basal ganglia and the cerebellum directly, we recorded from oculomotor regions in both structures from the same monkeys. We partitioned the trial-by-trial variability of the neurons into reward and eye-movement signals to compare the coding across structures. Reward expectation and movement signals were the most pronounced in the output structure of the basal ganglia, intermediate in the cerebellum, and the smallest in the input structure of the basal ganglia. These findings suggest that reward and movement information is sharpened through the basal ganglia, resulting in a higher signal-to-noise ratio than in the cerebellum.

The Cerebellar Response to Visual Portion Size Cues Is Associated with the Portion Size Effect in Children

The neural mechanisms underlying susceptibility to eating more in response to large portions (i.e., the portion size effect) remain unclear. Thus, the present study examined how neural responses to portion size relate to changes in weight and energy consumed as portions increase. Associations were examined across brain regions traditionally implicated in appetite control (i.e., an appetitive network) as well as the cerebellum, which has recently been implicated in appetite-related processes. Children without obesity (i.e., BMI-for-age-and-sex percentile < 90; N = 63; 55% female) viewed images of larger and smaller portions of food during fMRI and, in separate sessions, ate four meals that varied in portion size. Individual-level linear and quadratic associations between intake (kcal, grams) and portion size (i.e., portion size slopes) were estimated. The response to portion size in cerebellar lobules IV-VI was associated with the quadratic portion size slope estimated from gram intake; a greater response to images depicting smaller compared to larger portions was associated with steeper increases in intake with increasing portion sizes. Within the appetitive network, neural responses were not associated with portion size slopes. A decreased cerebellar response to larger amounts of food may increase children's susceptibility to overeating when excessively large portions are served.

The cerebellum directly modulates the substantia nigra dopaminergic activity

Evidence of direct reciprocal connections between the cerebellum and basal ganglia has challenged the long-held notion that these structures function independently. While anatomical studies have suggested the presence of cerebellar projections to the substantia nigra pars compacta (SNc), the nature and function of these connections (Cb-SNc) is unknown. Here we show, in mice, that Cb-SNc projections form monosynaptic glutamatergic synapses with dopaminergic and non-dopaminergic neurons in the SNc. Optogenetic activation of Cb-SNc axons in the SNc is associated with increased SNc activity, elevated striatal dopamine levels and increased locomotion. During behavior, Cb-SNc projections are bilaterally activated before ambulation and unilateral lever manipulation. Cb-SNc projections show prominent activation for water reward and higher activation for sweet water, suggesting that the pathway also encodes reward value. Thus, the cerebellum directly, rapidly and effectively modulates basal ganglia dopamine levels and conveys information related to movement initiation, vigor and reward processing.

The Role of the Human Cerebellum for Learning from and Processing of External Feedback in Non-Motor Learning: A Systematic Review

This review aimed to systematically identify and comprehensively review the role of the cerebellum in performance monitoring, focusing on learning from and on processing of external feedback in non-motor learning. While 1078 articles were screened for eligibility, ultimately 36 studies were included in which external feedback was delivered in cognitive tasks and which referenced the cerebellum. These included studies in patient populations with cerebellar damage and studies in healthy subjects applying neuroimaging. Learning performance in patients with different cerebellar diseases was heterogeneous, with only about half of all patients showing alterations. One patient study using EEG demonstrated that damage to the cerebellum was associated with altered neural processing of external feedback. Studies assessing brain activity with task-based fMRI or PET and one resting-state functional imaging study that investigated connectivity changes following feedback-based learning in healthy participants revealed involvement particularly of lateral and posterior cerebellar regions in processing of and learning from external feedback. Cerebellar involvement was found at different stages, e.g., during feedback anticipation and following the onset of the feedback stimuli, substantiating the cerebellum's relevance for different aspects of performance monitoring such as feedback prediction. Future research will need to further elucidate precisely how, where, and when the cerebellum modulates the prediction and processing of external feedback information, which cerebellar subregions are particularly relevant, and to what extent cerebellar diseases alter these processes.

Roles of the cerebellar vermis in predictive postural controls against external disturbances

The central nervous system predictively controls posture against external disturbances; however, the detailed mechanisms remain unclear. We tested the hypothesis that the cerebellar vermis plays a substantial role in acquiring predictive postural control by using a standing task with floor disturbances in rats. The intact, lesioned, and sham groups of rats sequentially underwent 70 conditioned floor-tilting trials, and kinematics were recorded. Six days before these recordings, only the lesion group underwent focal suction surgery targeting vermal lobules IV-VIII. In the naïve stage of the sequential trials, the upright postures and fluctuations due to the disturbance were mostly consistent among the groups. Although the pattern of decrease in postural fluctuation due to learning corresponded among the groups, the learning rate estimated from the lumbar displacement was significantly lower in the lesion group than in the intact and sham groups. These results suggest that the cerebellar vermis contributes to predictive postural controls.

Mild Deficits in Fear Learning: Evidence from Humans and Mice with Cerebellar Cortical Degeneration

Functional brain imaging studies in humans suggest involvement of the cerebellum in fear conditioning but do not allow conclusions about the functional significance. The main aim of the present study was to examine whether patients with cerebellar degeneration show impaired fear conditioning and whether this is accompanied by alterations in cerebellar cortical activations. To this end, a 2 d differential fear conditioning study was conducted in 20 cerebellar patients and 21 control subjects using a 7 tesla (7 T) MRI system. Fear acquisition and extinction training were performed on day 1, followed by recall on day 2. Cerebellar patients learned to differentiate between the CS+ and CS-. Acquisition and consolidation of learned fear, however, was slowed. Additionally, extinction learning appeared to be delayed. The fMRI signal was reduced in relation to the prediction of the aversive stimulus and altered in relation to its unexpected omission. Similarly, mice with cerebellar cortical degeneration (spinocerebellar ataxia type 6, SCA6) were able to learn the fear association, but retrieval of fear memory was reduced. In sum, cerebellar cortical degeneration led to mild abnormalities in the acquisition of learned fear responses in both humans and mice, particularly manifesting postacquisition training. Future research is warranted to investigate the basis of altered fMRI signals related to fear learning.

Disrupted executive cerebro-cerebellar functional connectivity in alcohol use disorder

BACKGROUND

Alcohol use disorder (AUD) affects 283 million people worldwide and its prevalence is increasing. Despite the role of the cerebellum in executive control and its sensitivity to alcohol, few studies have assessed its involvement in AUD-relevant functional networks. The goal of this study is to compare resting-state functional connectivity (FC) patterns in abstinent adults with a history of AUD and controls (CTL). We hypothesized that group differences in cerebro-cerebellar FC would be present, particularly within the frontoparietal/executive control network (FPN).

METHODS

Twenty-eight participants completed a resting-state functional magnetic resonance imaging (rsfMRI) study. CTL participants had no history of AUD, comorbid psychological conditions, or recent heavy drinking and/or drug use. AUD participants had a history of AUD, with sobriety for at least 30 days prior to data collection. Multivariate pattern analysis, an agnostic, whole-brain approach, was used to identify regions with significant differences in FC between groups. Seed-based analyses were then conducted to determine the directionality and extent of these FC differences. Associations between FC strength and executive function were assessed using correlations with Wisconsin Card Sorting Test (WCST) performance.

RESULTS

There were significant group differences in FC in nodes of the FPN, ventral attention network, and default mode network. Post hoc analyses predominantly identified FC differences within the cerebro-cerebellar FPN, with AUD showing significantly less FC within the FPN. In AUD, FC strength between FPN clusters identified in the multivariate pattern analysis (MVPA) analysis (Left Crus II, Right Frontal Cortex) was positively associated with performance on the WCST.

CONCLUSIONS

Our results show less engagement of the FPN in individuals with AUD than in CTL. FC strength within this network was positively associated with performance on the WCST. These findings suggest that long-term heavy drinking alters cerebro-cerebellar FC, particularly within networks that are involved in executive function.

Cerebellar Degeneration Impairs Strategy Discovery but Not Strategy Recall

The cerebellum is recognized to play a critical role in the automatic and implicit process by which movement errors are used to keep the sensorimotor system precisely calibrated. However, its role in other learning processes frequently engaged during sensorimotor adaptation tasks remains unclear. In the present study, we tested the performance of individuals with cerebellar degeneration on a variant of a visuomotor adaptation task in which learning requires the use of strategic re-aiming, a process that can nullify movement errors in a rapid and volitional manner. Our design allowed us to assess two components of this learning process, the discovery of an appropriate strategy and the recall of a learned strategy. Participants were exposed to a 60° visuomotor rotation twice, with the initial exposure block assessing strategy discovery and the re-exposure block assessing strategy recall. Compared to age-matched controls, individuals with cerebellar degeneration were slower to derive an appropriate aiming strategy in the initial Discovery block but exhibited similar recall of the aiming strategy during the Recall block. This dissociation underscores the multi-faceted contributions of the cerebellum to sensorimotor learning, highlighting one way in which this subcortical structure facilitates volitional action selection.

Silences, spikes and bursts: Three-part knot of the neural code

When a neuron breaks silence, it can emit action potentials in a number of patterns. Some responses are so sudden and intense that electrophysiologists felt the need to single them out, labelling action potentials emitted at a particularly high frequency with a metonym - bursts. Is there more to bursts than a figure of speech? After all, sudden bouts of high-frequency firing are expected to occur whenever inputs surge. The burst coding hypothesis advances that the neural code has three syllables: silences, spikes and bursts. We review evidence supporting this ternary code in terms of devoted mechanisms for burst generation, synaptic transmission and synaptic plasticity. We also review the learning and attention theories for which such a triad is beneficial.

Activity map of a cortico-cerebellar loop underlying motor planning

The neocortex and cerebellum interact to mediate cognitive functions. It remains unknown how the two structures organize into functional networks to mediate specific behaviors. Here we delineate activity supporting motor planning in relation to the mesoscale cortico-cerebellar connectome. In mice planning directional licking based on short-term memory, preparatory activity instructing future movement depends on the anterior lateral motor cortex (ALM) and the cerebellum. Transneuronal tracing revealed divergent and largely open-loop connectivity between the ALM and distributed regions of the cerebellum. A cerebellum-wide survey of neuronal activity revealed enriched preparatory activity in hotspot regions with conjunctive input-output connectivity to the ALM. Perturbation experiments show that the conjunction regions were required for maintaining preparatory activity and correct subsequent movement. Other cerebellar regions contributed little to motor planning despite input or output connectivity to the ALM. These results identify a functional cortico-cerebellar loop and suggest the cerebellar cortex selectively establishes reciprocal cortico-cerebellar communications to orchestrate motor planning.

Altered dynamic functional connectivity of the thalamus subregions in patients with schizophrenia

BACKGROUND

Previous neuroimaging studies indicated that patients with schizophrenia showed impaired thalamus and thalamo-cortical circuits. However, the dynamic functional connectivity (dFC) patterns of the thalamus remain unclear. In this study, we explored the dFC of the thalamus in SZ patients and whether clinical features are correlated with altered dFC.

METHODS

Forty-three patients with schizophrenia and 31 healthy controls underwent 3.0 T rs-fMRI. Based on the human Brainnetome atlas, the thalamus is divided into 8 subregions. Subsequently, we performed flexible least squares method to calculate the dFC of each thalamus subregions.

RESULTS

Compared with healthy controls, patients with schizophrenia exhibited increased dFC between the thalamus and cerebellar, visual-related cortex, sensorimotor-related cortex, and frontal lobe. In addition, we found that the dFC of the thalamus and the right fusiform gyrus was negatively associated with age of onset.

CONCLUSIONS

Our findings demonstrated that the dFC of specific thalamus sub-regions is altered in patients with schizophrenia. Our results further suggested the dysconnectivity of thalamus plays an important role in the pathophysiology of schizophrenia.

Recent data on the cerebellum require new models and theories

The cerebellum has been a popular topic for theoretical studies because its structure was thought to be simple. Since David Marr and James Albus related its function to motor skill learning and proposed the Marr-Albus cerebellar learning model, this theory has guided and inspired cerebellar research. In this review, we summarize the theoretical progress that has been made within this framework of error-based supervised learning. We discuss the experimental progress that demonstrates more complicated molecular and cellular mechanisms in the cerebellum as well as new cell types and recurrent connections. We also cover its involvement in diverse non-motor functions and evidence of other forms of learning. Finally, we highlight the need to explain these new experimental findings into an integrated cerebellar model that can unify its diverse computational functions.

Dynamic organization of cerebellar climbing fiber response and synchrony in multiple functional components reduces dimensions for reinforcement learning

Cerebellar climbing fibers convey diverse signals, but how they are organized in the compartmental structure of the cerebellar cortex during learning remains largely unclear. We analyzed a large amount of coordinate-localized two-photon imaging data from cerebellar Crus II in mice undergoing 'Go/No-go' reinforcement learning. Tensor component analysis revealed that a majority of climbing fiber inputs to Purkinje cells were reduced to only four functional components, corresponding to accurate timing control of motor initiation related to a Go cue, cognitive error-based learning, reward processing, and inhibition of erroneous behaviors after a No-go cue. Changes in neural activities during learning of the first two components were correlated with corresponding changes in timing control and error learning across animals, indirectly suggesting causal relationships. Spatial distribution of these components coincided well with boundaries of Aldolase-C/zebrin II expression in Purkinje cells, whereas several components are mixed in single neurons. Synchronization within individual components was bidirectionally regulated according to specific task contexts and learning stages. These findings suggest that, in close collaborations with other brain regions including the inferior olive nucleus, the cerebellum, based on anatomical compartments, reduces dimensions of the learning space by dynamically organizing multiple functional components, a feature that may inspire new-generation AI designs.

Cerebellar climbing fibers multiplex movement and reward signals during a voluntary movement task in mice

Cerebellar climbing fibers convey sensorimotor information and their errors, which are used for motor control and learning. Furthermore, they represent reward-related information. Despite such functional diversity of climbing fiber signals, it is still unclear whether each climbing fiber conveys the information of single or multiple modalities and how the climbing fibers conveying different information are distributed over the cerebellar cortex. Here we perform two-photon calcium imaging from cerebellar Purkinje cells in mice engaged in a voluntary forelimb lever-pull task and demonstrate that climbing fiber responses in 68% of Purkinje cells can be explained by the combination of multiple behavioral variables such as lever movement, licking, and reward delivery. Neighboring Purkinje cells exhibit similar climbing fiber response properties, form functional clusters, and share noise fluctuations of responses. Taken together, individual climbing fibers convey behavioral information on multiplex variables and are spatially organized into the functional modules of the cerebellar cortex.

Reinforcement learning establishes a minimal metacognitive process to monitor and control motor learning performance

Humans and animals develop learning-to-learn strategies throughout their lives to accelerate learning. One theory suggests that this is achieved by a metacognitive process of controlling and monitoring learning. Although such learning-to-learn is also observed in motor learning, the metacognitive aspect of learning regulation has not been considered in classical theories of motor learning. Here, we formulated a minimal mechanism of this process as reinforcement learning of motor learning properties, which regulates a policy for memory update in response to sensory prediction error while monitoring its performance. This theory was confirmed in human motor learning experiments, in which the subjective sense of learning-outcome association determined the direction of up- and down-regulation of both learning speed and memory retention. Thus, it provides a simple, unifying account for variations in learning speeds, where the reinforcement learning mechanism monitors and controls the motor learning process.

Putting forward a model for the role of the cerebellum in cocaine-induced pavlovian memory

Substance Use Disorder (SUD) involves emotional, cognitive, and motivational dysfunction. Long-lasting molecular and structural changes in brain regions functionally and anatomically linked to the cerebellum, such as the prefrontal cortex, amygdala, hippocampus, basal ganglia, and ventral tegmental area, are characteristic of SUD. Direct and indirect reciprocal connectivity between the cerebellum and these brain regions can explain cerebellar roles in Pavlovian and reinforcement learning, fear memory, and executive functions. It is increasingly clear that the cerebellum modulates brain functions altered in SUD and other neuropsychiatric disorders that exhibit comorbidity with SUD. In the present manuscript, we review and discuss this evidence and present new research exploring the role of the cerebellum in cocaine-induced conditioned memory using chemogenetic tools (designer receptor exclusively activated by designer drug, DREADDs). Our preliminary data showed that inactivation of a region that includes the interposed and lateral deep cerebellar nuclei reduces the facilitating effect of a posterior vermis lesion on cocaine-induced preference conditioning. These findings support our previous research and suggest that posterior vermis damage may increase drug impact on the addiction circuitry by regulating activity in the DCN. However, they raise further questions that will also be discussed.

Exploring the role of task success in implicit motor adaptation

We learn to improve our motor skills using different forms of feedback: sensory-prediction error, task success, and reward/punishment. While implicit motor adaptation is driven by sensory-prediction errors, recent work has shown that task success modulates this process. Task success is often confounded with reward, so we sought to determine if the effects of these two signals on adaptation can be dissociated. To address this question, we conducted five experiments that isolated implicit learning using error-clamp visuomotor reach adaptation paradigms. Task success was manipulated by changing the size and position of the target relative to the cursor providing visual feedback, and reward expectation was established using monetary cues and auditory feedback. We found that neither monetary cues nor auditory feedback affected implicit adaptation, suggesting that task success influences implicit adaptation via mechanisms distinct from conventional reward-related processes. Additionally, we found that changes in target size, which caused the target to either exclude or fully envelop the cursor, only affected implicit adaptation for a narrow range of error sizes, while jumping the target to overlap with the cursor more reliably and robustly affected implicit adaptation. Taken together, our data indicate that, while task success exerts a small effect on implicit adaptation, these effects are susceptible to methodological variations and unlikely to be mediated by reward.

A disynaptic basal ganglia connection to the inferior olive: potential for basal ganglia influence on cerebellar learning

Recent studies have shown that the cerebellum and the basal ganglia are interconnected at subcortical levels. However, a subcortical basal ganglia connection to the inferior olive (IO), being the source of the olivocerebellar climbing fiber system, is not known. We have used classical tracing with CTb, retrograde transneuronal infection with wildtype rabies virus, conditional tracing with genetically modified rabies virus, and examination of material made available by the Allen Brain Institute, to study potential basal ganglia connections to the inferior olive in rats and mice. We show in both species that parvalbumin-positive, and therefore GABAergic, neurons in the entopeduncular nucleus, representing the rodent equivalent of the internal part of the globus pallidus, innervate a group of cells that surrounds the fasciculus retroflexus and that are collectively known as the area parafascicularis prerubralis. As these neurons supply a direct excitatory input to large parts of the inferior olivary complex, we propose that the entopeduncular nucleus, as a main output station of the basal ganglia, provides an inhibitory influence on olivary excitability. As such, this connection may influence olivary involvement in cerebellar learning and/or could be involved in transmission of reward properties that have recently been established for olivocerebellar signaling.

Alterations in gray matter morphology and functional connectivity in adult patients with newly diagnosed, untreated hypothyroidism

BACKGROUND

Untreated adult hypothyroidism may be associated with cognitive and emotional impairment, but the precise underlying neuropathological mechanism is unknown. We investigated the brain morphological and functional abnormalities associated with cognition and emotion in hypothyroidism.

METHODS

This is a cross-sectional observational study. Forty-four newly diagnosed adult hypothyroid patients and 54 well-matched healthy controls (HC) were enrolled. All participants underwent 3D T1WI and resting-state functional brain MRI. Morphological and seed-based functional connectivity (FC) analyses were performed to compare the inter-group differences. Neuropsychological tests, including the Montreal Cognitive Assessment Scale (MoCA), 24-item Hamilton Depression Rating Scale (HAMD-24), and Hamilton Anxiety Rating Scale (HAMA) were administered. Thyroid function test and blood lipid levels were measured. Correlations were computed between neuropsychological and biochemical measures with neuroimaging indices. Sensitive morphological or functional neuroimaging indicators were identified using ROC analysis.

RESULTS

Compared with HC, hypothyroid patients demonstrated lower total and subdomain scores on the MoCA and higher HAMD-24 and HAMA scores. Morphological analysis revealed the hypothyroid patients had significantly reduced gray matter (GM) volumes in the right superior frontal gyrus, superior temporal gyrus, left dorsolateral superior frontal gyrus, middle frontal gyrus, and supplementary motor area as well as significantly increased GM volumes in the bilateral cerebellar CrusⅠand left precentral gyrus. Furthermore, MRIs of hypothyroid patients showed increased FC between the right cerebellar CrusⅠand left precentral gyrus, triangular part of the inferior frontal gyrus, and angular gyrus of the inferior parietal lobe. The language scores of the MoCA were positively correlated with Jacobian values of the left supplementary motor area (r = 0.391, P = 0.046) and precentral gyrus (r = 0.401, P = 0.039). ROC analysis revealed FC value between cerebellar CrusⅠand angular gyrus could differentiate groups with relatively high accuracy (sensitivity: 75%, specificity: 77.8%, AUC: 0.794, 95% CI: 0.701-0.888, P < 0.001).

CONCLUSION

Untreated adult-onset hypothyroidism may be associated with impaired cognition and anxiety or depression. GM morphological alterations and FC of the cerebellum with subregions of the frontal and parietal lobes may represent key neuropathological mechanisms underlying the cognitive deterioration and mood dysregulation observed in hypothyroid adults.

Neural correlates of cocaine-induced conditioned place preference in the posterior cerebellar cortex

Introduction

Addictive drugs are potent neuropharmacological agents capable of inducing long-lasting changes in learning and memory neurocircuitry. With repeated use, contexts and cues associated with consumption can acquire motivational and reinforcing properties of abused drugs, triggering drug craving and relapse. Neuroplasticity underlying drug-induced memories takes place in prefrontal-limbic-striatal networks. Recent evidence suggests that the cerebellum is also involved in the circuitry responsible for drug-induced conditioning. In rodents, preference for cocaine-associated olfactory cues has been shown to correlate with increased activity at the apical part of the granular cell layer in the posterior vermis (lobules VIII and IX). It is important to determine if the cerebellum's role in drug conditioning is a general phenomenon or is limited to a particular sensory modality.

Methods

The present study evaluated the role of the posterior cerebellum (lobules VIII and IX), together with the medial prefrontal cortex (mPFC), ventral tegmental area (VTA), and nucleus accumbens (NAc) using a cocaine-induced conditioned place preference procedure with tactile cues. Cocaine CPP was tested using ascending (3, 6, 12, and 24 mg/kg) doses of cocaine in mice.

Results

Compared to control groups (Unpaired and Saline animals), Paired mice were able to show a preference for the cues associated with cocaine. Increased activation (cFos expression) of the posterior cerebellum was found in cocaine CPP groups and showed a positive correlation with CPP levels. Such increases in cFos activity in the posterior cerebellum significantly correlated with cFos expression in the mPFC.

Discussion

Our data suggest that the dorsal region of the cerebellum could be an important part of the network that mediates cocaine-conditioned behavior.

The human cerebellum in reward anticipation and reward outcome processing: An activation likelihood estimation meta-analysis

The cerebellum generates internal prediction models and actively compares anticipated and actual outcomes in order to reach a desired end state. In this process, reward can serve as a reinforcer that shapes internal prediction models, enabling context-appropriate behavior. While the involvement of the cerebellum in reward processing has been established in animals, there is no detailed account of which cerebellar regions are involved in reward anticipation and reward outcome processing in humans. To this end, an activation likelihood estimation meta-analysis of functional neuroimaging studies was performed to investigate cerebellar functional activity patterns associated with reward anticipation and reward outcome processing in healthy adults. Results showed that reward anticipation (k=31) was associated with regional activity in the bilateral anterior lobe, bilateral lobule VI, left Crus I and the posterior vermis, while reward outcome (k=16) was associated with regional activity in the declive and left lobule VI. The findings of this meta-analysis show distinct involvement of the cerebellum in reward anticipation and reward outcome processing as part of a predictive coding routine.

Cerebellum as a kernel machine: A novel perspective on expansion recoding in granule cell layer

Sensorimotor information provided by mossy fibers (MF) is mapped to high-dimensional space by a huge number of granule cells (GrC) in the cerebellar cortex's input layer. Significant studies have demonstrated the computational advantages and primary contributor of this expansion recoding. Here, we propose a novel perspective on the expansion recoding where each GrC serve as a kernel basis function, thereby the cerebellum can operate like a kernel machine that implicitly use high dimensional (even infinite) feature spaces. We highlight that the generation of kernel basis function is indeed biologically plausible scenario, considering that the key idea of kernel machine is to memorize important input patterns. We present potential regimes for developing kernels under constrained resources and discuss the advantages and disadvantages of each regime using various simulation settings.

Cerebellar Contributions to the Basal Ganglia Influence Motor Coordination, Reward Processing, and Movement Vigor

Both the cerebellum and the basal ganglia are known for their roles in motor control and motivated behavior. These two systems have been classically considered as independent structures that coordinate their contributions to behavior via separate cortico-thalamic loops. However, recent evidence demonstrates the presence of a rich set of direct connections between these two regions. Although there is strong evidence for connections in both directions, for brevity we limit our discussion to the better-characterized connections from the cerebellum to the basal ganglia. We review two sets of such connections: disynaptic projections through the thalamus and direct monosynaptic projections to the midbrain dopaminergic nuclei, the VTA and the SNc. In each case, we review the evidence for these pathways from anatomic tracing and physiological recordings, and discuss their potential functional roles. We present evidence that the disynaptic pathway through the thalamus is involved in motor coordination, and that its dysfunction contributes to motor deficits, such as dystonia. We then discuss how cerebellar projections to the VTA and SNc influence dopamine release in the respective targets of these nuclei: the NAc and the dorsal striatum. We argue that the cerebellar projections to the VTA may play a role in reward-based learning and therefore contribute to addictive behavior, whereas the projection to the SNc may contribute to movement vigor. Finally, we speculate how these projections may explain many of the observations that indicate a role for the cerebellum in mental disorders, such as schizophrenia.

A Liaison Brought to Light: Cerebellum-Hippocampus, Partners for Spatial Cognition

This review focuses on the functional and anatomical links between the cerebellum and the hippocampus and the role of their interplay in goal-directed navigation and spatial cognition. We will describe the interactions between the cerebellum and the hippocampus at different scales: a macroscopic scale revealing the joint activations of these two structures at the level of neuronal circuits, a mesoscopic scale highlighting the synchronization of neuronal oscillations, and finally a cellular scale where we will describe the activity of hippocampal neuronal assemblies following a targeted manipulation of the cerebellar system. We will take advantage of this framework to summarize the different anatomical pathways that may sustain this multiscale interaction. We will finally consider the possible influence of the cerebellum on pathologies traditionally associated with hippocampal dysfunction.

Role of Perineuronal nets in the cerebellar cortex in cocaine-induced conditioned preference, extinction, and reinstatement

Perineuronal nets (PNNs) are cartilage-like structures of extracellular matrix molecules that enwrap in a net-like manner the cell-body and proximal dendrites of special subsets of neurons. PNNs stabilize their incoming connections and restrict plasticity. Consequently, they have been proposed as a candidate mechanism for drug-induced learning and memory. In the cerebellum, PNNs surround Golgi inhibitory interneurons and both inhibitory and excitatory neurons in the deep cerebellar nuclei (DCN). Previous studies from the lab showed that cocaine-induced conditioned memory increased PNN expression in the granule cell layer of the posterior vermis. The present research aimed to investigate the role of cerebellar PNNs in cocaine-induced conditioned preference. For this purpose, we use the enzyme chondroitinase ABC (ChABC) to digest PNNs at different time points of the learning process to ascertain whether their removal can affect drug-induced memory. Our results show that PNN digestion using ChABC in the posterior vermis (Lobule VIII) did not affect the acquisition of cocaine-induced conditioned preference. However, the removal of PNNs in Lobule VIII -but not in the DCN- disrupted short-term memory of conditioned preference. Moreover, although PNN digestion facilitated the formation of extinction, reinstatement of cocaine-induced conditioned preference was encouraged under PNN digestion. The present findings suggests that PNNs around Golgi interneurons are needed to maintain cocaine-induced Pavlovian memory but also to stabilize extinction memory. Conversely, PNN degradation within the DCN did not affect stability of cocaine-induced memories. Therefore, degradation of PNNs in the vermis might be used as a promising tool to manipulate drug-induced memory.