Probabilistic reversal learning is impaired in Parkinson's disease

Ca2+-dependent phosphodiesterase 1 regulates the plasticity of striatal spiny projection neuron glutamatergic synapses

Long-term synaptic plasticity at glutamatergic synapses on striatal spiny projection neurons (SPNs) is central to learning goal-directed behaviors and habits. Our studies reveal that SPNs manifest a heterosynaptic, nitric oxide (NO)-dependent form of long-term postsynaptic depression of glutamatergic SPN synapses (NO-LTD) that is preferentially engaged at quiescent synapses. Plasticity is gated by Ca2+ entry through CaV1.3 Ca2+ channels and phosphodiesterase 1 (PDE1) activation, which blunts intracellular cyclic guanosine monophosphate (cGMP) and NO signaling. Both experimental and simulation studies suggest that this Ca2+-dependent regulation of PDE1 activity allows for local regulation of dendritic cGMP signaling. In a mouse model of Parkinson disease (PD), NO-LTD is absent because of impaired interneuronal NO release; re-balancing intrastriatal neuromodulatory signaling restores NO release and NO-LTD. Taken together, these studies provide important insights into the mechanisms governing NO-LTD in SPNs and its role in psychomotor disorders such as PD.

Dynamic balance and gait impairments in Parkinson's disease: novel cholinergic patterns

The cholinergic system has been implicated in postural deficits, in particular falls, in Parkinson's disease (PD). Falls and freezing of gait typically occur during dynamic and challenging balance and gait conditions, such as when initiating gait, experiencing postural perturbations, or making turns. However, the precise cholinergic neural substrate underlying dynamic postural and gait changes remains poorly understood. The aim of this study was to investigate whether brain vesicular acetylcholine transporter binding, as measured with [18F]-fluoroethoxybenzovesamicol binding PET, correlates with dynamic gait and balance impairments in 125 patients with PD (mean age 66.89 ± 7.71 years) using the abbreviated balance evaluation systems test total and its four functional domain sub-scores (anticipatory postural control, reactive postural control, dynamic gait, and sensory integration). Whole brain false discovery-corrected (P < 0.05) correlations for total abbreviated balance evaluation systems test scores included the following bilateral or asymmetric hemispheric regions: gyrus rectus, orbitofrontal cortex, anterior part of the dorsomedial prefrontal cortex, dorsolateral prefrontal cortex, cingulum, frontotemporal opercula, insula, fimbria, right temporal pole, mesiotemporal, parietal and visual cortices, caudate nucleus, lateral and medial geniculate bodies, thalamus, lingual gyrus, cerebellar hemisphere lobule VI, left cerebellar crus I, superior cerebellar peduncles, flocculus, and nodulus. No significant correlations were found for the putamen or anteroventral putamen. The four domain-specific sub-scores demonstrated overlapping cholinergic topography in the metathalamus, fimbria, thalamus proper, and prefrontal cortices but also showed distinct topographic variations. For example, reactive postural control functions involved the right flocculus but not the upper brainstem regions. The anterior cingulum associated with reactive postural control whereas the posterior cingulum correlated with anticipatory control. The spatial extent of associated cholinergic system changes were least for dynamic gait and sensory orientation functional domains compared to the anticipatory and reactive postural control functions. We conclude that specific aspects of dynamic balance and gait deficits in PD associate with overlapping but also distinct patterns of cerebral cholinergic system changes in numerous brain regions. Our study also presents novel evidence of cholinergic topography involved in dynamic balance and gait in PD that have not been typically associated with mobility disturbances, such as the right anterior temporal pole, right anterior part of the dorsomedial prefrontal cortex, gyrus rectus, fimbria, lingual gyrus, flocculus, nodulus, and right cerebellar hemisphere lobules VI and left crus I.

Characterisation of behaviours relevant to apathy syndrome in the aged male rat

Apathy is a complex psychiatric syndrome characterised by motivational deficit, emotional blunting and cognitive changes. It occurs alongside a broad range of neurological disorders, but also occurs in otherwise healthy ageing. Despite its clinical prevalence, apathy does not yet have a designated treatment strategy. Generation of a translational animal model of apathy syndrome would facilitate the development of novel treatments. Given the multidimensional nature of apathy, a model cannot be achieved with a single behavioural test. Using a battery of behavioural tests we investigated whether aged rats exhibit behavioural deficits across different domains relevant to apathy. Using the effort for reward and progressive ratio tasks we found that aged male rats (21-27 months) show intact reward motivation. Using the novelty supressed feeding test and position-based object exploration we found aged rats showed increased anxiety-like behaviour inconsistent with emotional blunting. The sucrose preference test and reward learning assay showed intact reward sensitivity and reward-related cognition in aged rats. However, using a bowl-digging version of the probabilistic reversal learning task, we found a deficit in cognitive flexibility in aged rats that did not translate across to a touchscreen version of the task. While these data reveal important changes in cognitive flexibility and anxiety associated with ageing, aged rats do not show deficits across other behavioural domains relevant to apathy. This suggests that aged rats are not a suitable model for age-related apathy syndrome. These findings contrast with previous work in mice, revealing important species differences in behaviours relevant to apathy syndrome in ageing.

Artificial neural networks for model identification and parameter estimation in computational cognitive models

Computational cognitive models have been used extensively to formalize cognitive processes. Model parameters offer a simple way to quantify individual differences in how humans process information. Similarly, model comparison allows researchers to identify which theories, embedded in different models, provide the best accounts of the data. Cognitive modeling uses statistical tools to quantitatively relate models to data that often rely on computing/estimating the likelihood of the data under the model. However, this likelihood is computationally intractable for a substantial number of models. These relevant models may embody reasonable theories of cognition, but are often under-explored due to the limited range of tools available to relate them to data. We contribute to filling this gap in a simple way using artificial neural networks (ANNs) to map data directly onto model identity and parameters, bypassing the likelihood estimation. We test our instantiation of an ANN as a cognitive model fitting tool on classes of cognitive models with strong inter-trial dependencies (such as reinforcement learning models), which offer unique challenges to most methods. We show that we can adequately perform both parameter estimation and model identification using our ANN approach, including for models that cannot be fit using traditional likelihood-based methods. We further discuss our work in the context of the ongoing research leveraging simulation-based approaches to parameter estimation and model identification, and how these approaches broaden the class of cognitive models researchers can quantitatively investigate.

Ca 2+ -dependent phosphodiesterase 1 regulates the plasticity of striatal spiny projection neuron glutamatergic synapses

Long-term synaptic plasticity at glutamatergic synapses on striatal spiny projection neurons (SPNs) is central to learning goal-directed behaviors and habits. Although considerable attention has been paid to the mechanisms underlying synaptic strengthening and new learning, little scrutiny has been given to those involved in the attenuation of synaptic strength that attends suppression of a previously learned association. Our studies revealed a novel, non-Hebbian, long-term, postsynaptic depression of glutamatergic SPN synapses induced by interneuronal nitric oxide (NO) signaling (NO-LTD) that was preferentially engaged at quiescent synapses. This form of plasticity was gated by local Ca 2+ influx through CaV1.3 Ca 2+ channels and stimulation of phosphodiesterase 1 (PDE1), which degraded cyclic guanosine monophosphate (cGMP) and blunted NO signaling. Consistent with this model, mice harboring a gain-of-function mutation in the gene coding for the pore-forming subunit of CaV1.3 channels had elevated depolarization-induced dendritic Ca 2+ entry and impaired NO-LTD. Extracellular uncaging of glutamate and intracellular uncaging of cGMP suggested that this Ca 2+ -dependent regulation of PDE1 activity allowed for local regulation of dendritic NO signaling. This inference was supported by simulation of SPN dendritic integration, which revealed that dendritic spikes engaged PDE1 in a branch-specific manner. In a mouse model of Parkinson's disease (PD), NO-LTD was absent not because of a postsynaptic deficit in NO signaling machinery, but rather due to impaired interneuronal NO release. Re-balancing intrastriatal neuromodulatory signaling in the PD model restored NO release and NO-LTD. Taken together, these studies provide novel insights into the mechanisms governing NO-LTD in SPN and its role in psychomotor disorders, like PD.

Artificial neural networks for model identification and parameter estimation in computational cognitive models

Computational cognitive models have been used extensively to formalize cognitive processes. Model parameters offer a simple way to quantify individual differences in how humans process information. Similarly, model comparison allows researchers to identify which theories, embedded in different models, provide the best accounts of the data. Cognitive modeling uses statistical tools to quantitatively relate models to data that often rely on computing/estimating the likelihood of the data under the model. However, this likelihood is computationally intractable for a substantial number of models. These relevant models may embody reasonable theories of cognition, but are often under-explored due to the limited range of tools available to relate them to data. We contribute to filling this gap in a simple way using artificial neural networks (ANNs) to map data directly onto model identity and parameters, bypassing the likelihood estimation. We test our instantiation of an ANN as a cognitive model fitting tool on classes of cognitive models with strong inter-trial dependencies (such as reinforcement learning models), which offer unique challenges to most methods. We show that we can adequately perform both parameter estimation and model identification using our ANN approach, including for models that cannot be fit using traditional likelihood-based methods. We further discuss our work in the context of the ongoing research leveraging simulation-based approaches to parameter estimation and model identification, and how these approaches broaden the class of cognitive models researchers can quantitatively investigate.

Ketamine produces no detectable long-term positive or negative effects on cognitive flexibility or reinforcement learning of male rats

RATIONALE

Patients with major depressive disorder (MDD) often experience abnormalities in behavioral adaptation following environmental changes (i.e., cognitive flexibility) and tend to undervalue positive outcomes but overvalue negative outcomes. The probabilistic reversal learning task (PRL) is used to study these deficits across species and to explore drugs that may have therapeutic value. Selective serotonin-reuptake inhibitors (SSRIs) have limited effectiveness in treating MDD and produce inconsistent effects in non-human versions of the PRL. As such, ketamine, a novel and potentially rapid-acting therapeutic, has begun to be examined using the PRL. Two previous studies examining the effects of ketamine in the PRL have shown conflicting results and only examined short-term effects of ketamine.

OBJECTIVE

This experiment examined PRL performance across a 2-week period following a single exposure to a ketamine dose that varied across groups.

METHODS

After five sessions of PRL training, groups of rats received an injection of either 0, 10, 20 or 30 mg/kg ketamine. One-hour post-injection, rats engaged in the PRL, and subsequently sessions continued daily for 2 weeks. Traditional behavioral and computational reinforcement learning-derived measures were examined.

RESULTS

Results showed that ketamine had acute effects 1-h post-injection, including a significant decrease in the value of the punishment learning rate. Beyond 1 h, ketamine produced no detectable improvements nor decrements in performance across 2 weeks.

CONCLUSION

Overall, the present results suggest that the range of ketamine doses examined do not have long-term positive or negative effects on cognitive flexibility or reward processing in healthy rats as measured by the PRL.

Dopamine across timescales and cell types: Relevance for phenotypes in Parkinson's disease progression

Dopamine neurons in the substantia nigra pars compacta (SNc) synthesize and release dopamine, a critical neurotransmitter for movement and learning. SNc dopamine neurons degenerate in Parkinson's Disease (PD), causing a host of motor and non-motor symptoms. Here, we review recent conceptual advances in our basic understanding of the dopamine system - including our rapidly advancing knowledge of dopamine neuron heterogeneity - with special attention to their importance for understanding PD. In PD patients, dopamine neuron degeneration progresses from lateral SNc to medial SNc, suggesting clinically relevant heterogeneity in dopamine neurons. With technical advances in dopamine system interrogation, we can understand the relevance of this heterogeneity for PD progression and harness it to develop new treatments.

Executive dysfunction and cognitive decline, a non-motor symptom of Parkinson's disease captured in animal models

The non-motor symptoms of Parkinson's disease (PD) have gained increasing attention in recent years due to their significant impact on patients' quality of life. Among these non-motor symptoms, cognitive dysfunction has emerged as an area of particular interest where the clinical aspects are covered in Chapter 2 of this volume. This chapter explores the rationale for investigating the underlying neurobiology of cognitive dysfunction by utilising translational animal models of PD, from rodents to non-human primates. The objective of this chapter is to review the various animal models of cognition that have explored the dysfunction in animal models of Parkinson's disease. Some of the more advanced pharmacological studies aimed at restoring these cognitive deficits are reviewed, although this chapter highlights the lack of systematic approaches in dealing with this non-motor symptom at the pre-clinical stages.

Characterization of striatal dopamine projections across striatal subregions in behavioral flexibility

Behavioural flexibility is key to survival in a dynamic environmentWhile flexible, goal-directed behaviours are initially dependent on dorsomedial striatum, they become dependent on lateral striatum as behaviours become inflexible. Similarly, lesions of dopamine terminals in lateral striatum disrupt the development of inflexible habits. This work suggests that dopamine release in lateral striatum may drive inflexible behaviours, though few studies have investigated a causative role of subpopulations of striatal dopamine terminals in reversal learning, a measure of flexibility. Here, we performed two optogenetic experiments to activate dopamine terminals in dorsomedial (DMS), dorsolateral (DLS) or ventral (nucleus accumbens [NAc]) striatum in DAT-Cre mice that expressed channelrhodopsin-2 via viral injection (Experiment I) or through transgenic breeding with an Ai32 reporter line (Experiment II) to determine how specific dopamine subpopulations impact reversal learning. Mice performed a reversal task in which they self-stimulated DMS, DLS, or NAc dopamine terminals by pressing one of two levers before action-outcome lever contingencies were reversed. Largely consistent with presumed ventromedial/lateral striatal function, we found that mice self-stimulating medial dopamine terminals reversed lever preference following contingency reversal, while mice self-stimulating NAc showed parial flexibility, and DLS self-stimulation resulted in impaired reversal. Impairments in DLS mice were characterized by more regressive errors and reliance on lose-stay strategies following reversal, as well as reduced within-session learning, suggesting reward insensitivity and overreliance on previously learned actions. This study supports a model of striatal function in which DMS and ventral dopamine facilitate goal-directed responding, and DLS dopamine supports more inflexible responding.

How does apathy impact exploration-exploitation decision-making in older patients with neurocognitive disorders?

Apathy is a pervasive clinical syndrome in neurocognitive disorders, characterized by a quantitative reduction in goal-directed behaviors. The brain structures involved in the physiopathology of apathy have also been connected to the brain structures involved in probabilistic reward learning in the exploration-exploitation dilemma. This dilemma in question involves the challenge of selecting between a familiar option with a more predictable outcome, and another option whose outcome is uncertain and may yield potentially greater rewards compared to the known option. The aim of this study was to combine experimental procedures and computational modeling to examine whether, in older adults with mild neurocognitive disorders, apathy affects performance in the exploration-exploitation dilemma. Through using a four-armed bandit reinforcement-learning task, we showed that apathetic older adults explored more and performed worse than non-apathetic subjects. Moreover, the mental flexibility assessed by the Trail-making test-B was negatively associated with the percentage of exploration. These results suggest that apathy is characterized by an increased explorative behavior and inefficient decision-making, possibly due to weak mental flexibility to switch toward the exploitation of the more rewarding options. Apathetic participants also took longer to make a choice and failed more often to respond in the allotted time, which could reflect the difficulties in action initiation and selection. In conclusion, the present results suggest that apathy in participants with neurocognitive disorders is associated with specific disturbances in the exploration-exploitation trade-off and sheds light on the disturbances in reward processing in patients with apathy.

Parallel learning and cognitive flexibility impairments between Fmr1 knockout mice and individuals with fragile X syndrome

Introduction

Fragile X Syndrome (FXS) is a monogenic condition that leads to intellectual disability along with behavioral and learning difficulties. Among behavioral and learning difficulties, cognitive flexibility impairments are among the most commonly reported in FXS, which significantly impacts daily living. Despite the extensive use of the Fmr1 knockout (KO) mouse to understand molecular, synaptic and behavioral alterations related to FXS, there has been limited development of translational paradigms to understand cognitive flexibility that can be employed in both animal models and individuals with FXS to facilitate treatment development.

Methods

To begin addressing this limitation, a parallel set of studies were carried out that investigated probabilistic reversal learning along with other behavioral and cognitive tests in individuals with FXS and Fmr1 KO mice. Fifty-five adolescents and adults with FXS (67% male) and 34 age- and sex-matched typically developing controls (62% male) completed an initial probabilistic learning training task and a probabilistic reversal learning task.

Results

In males with FXS, both initial probabilistic learning and reversal learning deficits were found. However, in females with FXS, we only observed reversal learning deficits. Reversal learning deficits related to more severe psychiatric features in females with FXS, whereas increased sensitivity to negative feedback (lose:shift errors) unexpectedly appear to be adaptive in males with FXS. Male Fmr1 KO mice exhibited both an initial probabilistic learning and reversal learning deficit compared to that of wildtype (WT) mice. Female Fmr1 KO mice were selectively impaired on probabilistic reversal learning. In a prepotent response inhibition test, both male and female Fmr1 KO mice were impaired in learning to choose a non-preferred spatial location to receive a food reward compared to that of WT mice. Neither male nor female Fmr1 KO mice exhibited a change in anxiety compared to that of WT mice.

Discussion

Together, our findings demonstrate strikingly similar sex-dependent learning disturbances across individuals with FXS and Fmr1 KO mice. This suggests the promise of using analogous paradigms of cognitive flexibility across species that may speed treatment development to improve lives of individuals with FXS.

No effect of subthalamic deep brain stimulation on metacognition in Parkinson's disease

Deep brain stimulation of the subthalamic nucleus (STN-DBS) is a powerful treatment in Parkinson's disease (PD), which provides a positive effect on motor symptoms although the way it operates on high cognitive processes such as metacognition remains unclear. To address this issue, we recorded electroencephalogram (EEG) of PD patients treated with STN-DBS that performed a reversal learning (RL) paradigm endowed with metacognitive self-assessment. We considered two stimulation conditions, namely DBS-ON (stimulation on) and DBS-OFF (stimulation off), and focused our EEG-analysis on the frontal brain region due to its involvement on high cognitive processes. We found a trend towards a significant difference in RL ability between stimulation conditions. STN-DBS showed no effect on metacognition, although a significant association between accuracy and decision confidence level held for DBS OFF, but not in the case of DBS ON. In summary, our study revealed no significant effect of STN-DBS on RL or metacognition.

The interpretation of computational model parameters depends on the context

Reinforcement Learning (RL) models have revolutionized the cognitive and brain sciences, promising to explain behavior from simple conditioning to complex problem solving, to shed light on developmental and individual differences, and to anchor cognitive processes in specific brain mechanisms. However, the RL literature increasingly reveals contradictory results, which might cast doubt on these claims. We hypothesized that many contradictions arise from two commonly-held assumptions about computational model parameters that are actually often invalid: That parameters generalize between contexts (e.g. tasks, models) and that they capture interpretable (i.e. unique, distinctive) neurocognitive processes. To test this, we asked 291 participants aged 8-30 years to complete three learning tasks in one experimental session, and fitted RL models to each. We found that some parameters (exploration / decision noise) showed significant generalization: they followed similar developmental trajectories, and were reciprocally predictive between tasks. Still, generalization was significantly below the methodological ceiling. Furthermore, other parameters (learning rates, forgetting) did not show evidence of generalization, and sometimes even opposite developmental trajectories. Interpretability was low for all parameters. We conclude that the systematic study of context factors (e.g. reward stochasticity; task volatility) will be necessary to enhance the generalizability and interpretability of computational cognitive models.

Reinforcement learning and Bayesian inference provide complementary models for the unique advantage of adolescents in stochastic reversal

During adolescence, youth venture out, explore the wider world, and are challenged to learn how to navigate novel and uncertain environments. We investigated how performance changes across adolescent development in a stochastic, volatile reversal-learning task that uniquely taxes the balance of persistence and flexibility. In a sample of 291 participants aged 8-30, we found that in the mid-teen years, adolescents outperformed both younger and older participants. We developed two independent cognitive models, based on Reinforcement learning (RL) and Bayesian inference (BI). The RL parameter for learning from negative outcomes and the BI parameters specifying participants' mental models were closest to optimal in mid-teen adolescents, suggesting a central role in adolescent cognitive processing. By contrast, persistence and noise parameters improved monotonically with age. We distilled the insights of RL and BI using principal component analysis and found that three shared components interacted to form the adolescent performance peak: adult-like behavioral quality, child-like time scales, and developmentally-unique processing of positive feedback. This research highlights adolescence as a neurodevelopmental window that can create performance advantages in volatile and uncertain environments. It also shows how detailed insights can be gleaned by using cognitive models in new ways.

Phasic Dopamine Changes and Hebbian Mechanisms during Probabilistic Reversal Learning in Striatal Circuits: A Computational Study

Cognitive flexibility is essential to modify our behavior in a non-stationary environment and is often explored by reversal learning tasks. The basal ganglia (BG) dopaminergic system, under a top-down control of the pre-frontal cortex, is known to be involved in flexible action selection through reinforcement learning. However, how adaptive dopamine changes regulate this process and learning mechanisms for training the striatal synapses remain open questions. The current study uses a neurocomputational model of the BG, based on dopamine-dependent direct (Go) and indirect (NoGo) pathways, to investigate reinforcement learning in a probabilistic environment through a task that associates different stimuli to different actions. Here, we investigated: the efficacy of several versions of the Hebb rule, based on covariance between pre- and post-synaptic neurons, as well as the required control in phasic dopamine changes crucial to achieving a proper reversal learning. Furthermore, an original mechanism for modulating the phasic dopamine changes is proposed, assuming that the expected reward probability is coded by the activity of the winner Go neuron before a reward/punishment takes place. Simulations show that this original formulation for an automatic phasic dopamine control allows the achievement of a good flexible reversal even in difficult conditions. The current outcomes may contribute to understanding the mechanisms for active control of dopamine changes during flexible behavior. In perspective, it may be applied in neuropsychiatric or neurological disorders, such as Parkinson’s or schizophrenia, in which reinforcement learning is impaired.

Cholinergic neurons in the pedunculopontine nucleus guide reversal learning by signaling the changing reward contingency

Cognitive flexibility enables effective switching between mental processes to generate appropriate responses. Cholinergic neurons (CNs) within the pedunculopontine nucleus (PPN) are associated with many functions, but their contribution to cognitive flexibility remains poorly understood. Here we measure PPN cholinergic activities using calcium indicators during the attentional set-shifting task. We find that PPN CNs exhibit increasing activities correlated with rewards during each stage and error trials in reversal stages, indicating sensitivity to rule switching. Inhibition of PPN cholinergic activity selectively impairs reversal learning, which improves with PPN CN activation. Activation of PPN CNs projecting to the substantia nigra pars compacta, mediodorsal thalamus, and parafascicular nucleus in a time-locked manner with reward improves reversal learning. Therefore, PPN CNs may encode not only reward signals but also the information of changing reward contingency that contributes to guiding reversal learning through output projections to multiple nuclei that participate in flexibility.

Interplay Between Inhibitory Control and Behavioural Flexibility: Impact of Dorsomedial Striatal Dopamine Denervation in Mice

In Parkinson's disease, nigrostriatal dopamine (DA) degeneration is commonly associated with motor symptomatology. However, non-motor symptoms affecting cognitive function, such as behavioural flexibility and inhibitory control may also appear early in the disease. Here we addressed the role of DA innervation of the dorsomedial striatum (DMS) in mediating these functions in 6-hydroxydopamine (6-OHDA)-lesioned mice using instrumental conditioning in various tasks. Behavioural flexibility was studied in a simple reversal task (nose-poke discrimination) or in reversal of a two-step sequence of actions (central followed by lateral nose-poke). Our results show that mild DA lesions of the DMS induces behavioural flexibility deficits in the sequential reversal learning only. In the first sessions following reversal of contingency, lesioned mice enhanced perseverative sequence of actions to the initial rewarded side then produced premature responses directly to the correct side omitting the central response, thus disrupting the two-step sequence of actions. These deficits may be linked to increased impulsivity as 6-OHDA-lesioned mice were unable to inhibit a previously learned motor response in a cued response inhibition task assessing proactive inhibitory control. Our findings show that partial DA denervation restricted to DMS impairs behavioural flexibility and proactive response inhibition in mice. Such striatal DA lesion may thus represent a valuable animal model for exploring deficits in executive control documented in early stage of Parkinson's disease.

Divergent pallidal pathways underlying distinct Parkinsonian behavioral deficits

The basal ganglia regulate a wide range of behaviors, including motor control and cognitive functions, and are profoundly affected in Parkinson's disease (PD). However, the functional organization of different basal ganglia nuclei has not been fully elucidated at the circuit level. In this study, we investigated the functional roles of distinct parvalbumin-expressing neuronal populations in the external globus pallidus (GPe-PV) and their contributions to different PD-related behaviors. We demonstrate that substantia nigra pars reticulata (SNr)-projecting GPe-PV neurons and parafascicular thalamus (PF)-projecting GPe-PV neurons are associated with locomotion and reversal learning, respectively. In a mouse model of PD, we found that selective manipulation of the SNr-projecting GPe-PV neurons alleviated locomotor deficit, whereas manipulation of the PF-projecting GPe-PV neurons rescued the impaired reversal learning. Our findings establish the behavioral importance of two distinct GPe-PV neuronal populations and, thereby, provide a new framework for understanding the circuit basis of different behavioral deficits in the Parkinsonian state.

The neural network basis of altered decision-making in patients with amyotrophic lateral sclerosis

OBJECTIVE

Amyotrophic lateral sclerosis (ALS) is a multisystem disorder associated with motor impairment and behavioral/cognitive involvement. We examined decision-making features and changes in the neural hub network in patients with ALS using a probabilistic reversal learning task and resting-state network analysis, respectively.

METHODS

Ninety ALS patients and 127 cognitively normal participants performed this task. Data from 62 ALS patients and 63 control participants were fitted to a Q-learning model.

RESULTS

ALS patients had anomalous decision-making features with little shift in choice until they thought the value of the two alternatives had become equal. The quantified parameters (Pαβ) calculated by logistic regression analysis with learning rate and inverse temperature well represented the unique choice pattern of ALS patients. Resting-state network analysis demonstrated a strong correlation between Pαβ and decreased degree centrality in the anterior cingulate gyrus and frontal pole.

INTERPRETATION

Altered decision-making in ALS patients may be related to the decreased hub function of medial prefrontal areas.

Subcortical Dopamine and Cognition in Schizophrenia: Looking Beyond Psychosis in Preclinical Models

Schizophrenia is characterized by positive, negative and cognitive symptoms. All current antipsychotic treatments feature dopamine-receptor antagonism that is relatively effective at addressing the psychotic (positive) symptoms of schizophrenia. However, there is no clear evidence that these medications improve the negative or cognitive symptoms, which are the greatest predictors of functional outcomes. One of the most robust pathophysiological observations in patients with schizophrenia is increased subcortical dopamine neurotransmission, primarily in the associative striatum. This brain area has an important role in a range of cognitive processes. Dopamine is also known to play a major part in regulating a number of cognitive functions impaired in schizophrenia but much of this research has been focused on cortical dopamine. Emerging research highlights the strong influence subcortical dopamine has on a range of cognitive domains, including attention, reward learning, goal-directed action and decision-making. Nonetheless, the precise role of the associative striatum in the cognitive impairments observed in schizophrenia remains poorly understood, presenting an opportunity to revisit its contribution to schizophrenia. Without a better understanding of the mechanisms underlying cognitive dysfunction, treatment development remains at a standstill. For this reason, improved preclinical animal models are needed if we are to understand the complex relationship between subcortical dopamine and cognition. A range of new techniques are facillitating the discrete manipulation of dopaminergic neurotransmission and measurements of cognitive performance, which can be investigated using a variety of sensitive translatable tasks. This has the potential to aid the successful incorporation of recent clinical research to address the lack of treatment strategies for cognitive symptoms in schizophrenia. This review will give an overview on the current state of research focused on subcortical dopamine and cognition in the context of schizophrenia research. We also discuss future strategies and approaches aimed at improving the translational outcomes for the treatment of cognitive deficits in schizophrenia.

Maladaptive avoidance patterns in Parkinson's disease are exacerbated by symptoms of depression

Parkinson's disease (PD) is a chronic, progressive neurodegenerative disorder, characterized by a loss of dopaminergic neurons in the substantia nigra pars compacta. Given that dopamine is critically involved in learning and other cognitive processes, such as working memory, dopamine loss in PD has been linked both to learning abnormalities and to cognitive dysfunction more generally in the disease. It is unclear, however, whether avoidance behavior is impacted in PD. This is significant, as this type of instrumental behavior plays an important role in both decision-making and emotional (dys) function. Consequently, the aim of the present study was to examine avoidance learning and operant extinction in PD using a computer-based task. On this task, participants control a spaceship and attempt to shoot an enemy spaceship to gain points. They also learn to hide in safe areas to protect from (i.e., avoid) aversive events (on-screen explosions and point loss). The results showed that patients with PD (N = 25) acquired an avoidance response during aversive periods to the same extent as healthy age-matched controls (N = 19); however, patients demonstrated greater hiding during safe periods not associated with aversive events, which could represent maladaptive generalization of the avoidance response. Furthermore, this impairment was more pronounced during the extinction phase, and in patients who reported higher levels of depression. These results demonstrate for the first time that PD is associated with maladaptive avoidance patterns, which could possibly contribute to the emergence of depression in the disease.

Basal ganglia role in learning rewarded actions and executing previously learned choices: Healthy and diseased states

The basal ganglia (BG) is a collection of nuclei located deep beneath the cerebral cortex that is involved in learning and selection of rewarded actions. Here, we analyzed BG mechanisms that enable these functions. We implemented a rate model of a BG-thalamo-cortical loop and simulated its performance in a standard action selection task. We have shown that potentiation of corticostriatal synapses enables learning of a rewarded option. However, these synapses became redundant later as direct connections between prefrontal and premotor cortices (PFC-PMC) were potentiated by Hebbian learning. After we switched the reward to the previously unrewarded option (reversal), the BG was again responsible for switching to the new option. Due to the potentiated direct cortical connections, the system was biased to the previously rewarded choice, and establishing the new choice required a greater number of trials. Guided by physiological research, we then modified our model to reproduce pathological states of mild Parkinson's and Huntington's diseases. We found that in the Parkinsonian state PMC activity levels become extremely variable, which is caused by oscillations arising in the BG-thalamo-cortical loop. The model reproduced severe impairment of learning and predicted that this is caused by these oscillations as well as a reduced reward prediction signal. In the Huntington state, the potentiation of the PFC-PMC connections produced better learning, but altered BG output disrupted expression of the rewarded choices. This resulted in random switching between rewarded and unrewarded choices resembling an exploratory phase that never ended. Along with other computational studies, our results further reconcile the apparent contradiction between the critical involvement of the BG in execution of previously learned actions and yet no impairment of these actions after BG output is ablated by lesions or deep brain stimulation. We predict that the cortico-BG-thalamo-cortical loop conforms to previously learned choice in healthy conditions, but impedes those choices in disease states.

Maladaptive striatal plasticity and abnormal reward-learning in cervical dystonia

In monogenetic generalized forms of dystonia, in vitro neurophysiological recordings have demonstrated direct evidence for abnormal plasticity at the level of the cortico-striatal synapse. It is unclear whether similar abnormalities contribute to the pathophysiology of cervical dystonia, the most common type of focal dystonia. We investigated whether abnormal cortico-striatal synaptic plasticity contributes to abnormal reward-learning behavior in patients with focal dystonia. Forty patients and 40 controls performed a reward gain and loss avoidance reversal learning task. Participant's behavior was fitted to a computational model of the basal ganglia incorporating detailed cortico-striatal synaptic learning rules. Model comparisons were performed to assess the ability of four hypothesized receptor specific abnormalities of cortico-striatal long-term potentiation (LTP) and long-term depression (LTD): increased or decreased D1:LTP/LTD and increased or decreased D2: LTP/LTD to explain abnormal behavior in patients. Patients were selectively impaired in the post-reversal phase of the reward task. Individual learning rates in the reward reversal task correlated with the severity of the patient's motor symptoms. A model of the striatum with decreased D2:LTP/ LTD best explained the patient's behavior, suggesting excessive D2 cortico-striatal synaptic depotentiation could underpin biased reward-learning in patients with cervical dystonia. Reversal learning impairment in cervical dystonia may be a behavioral correlate of D2-specific abnormalities in cortico-striatal synaptic plasticity. Reinforcement learning tasks with computational modeling could allow the identification of molecular targets for novel treatments based on their ability to restore normal reward-learning behavior in these patients.

Assessment of Metacognition and Reversal Learning in Parkinson's Disease: Preliminary Results

Reversal learning (RL) has been widely used for assessment of behavioral adaptation, impulsivity, obsession, and compulsion in healthy controls as well as people suffering from psychiatric and neurological disorders such as Parkinson’s disease (PD). Nevertheless, studies addressing high cognitive functions such as metacognition in PD are scarce. Here, we address for the first time the effect of levodopa and PD on metacognition within the framework of a RL paradigm. In agreement with previous reports, PD patients exhibited reversal shifting impairment with respect to healthy controls (CTRL) regardless of medication condition (MED-ON and MED-OFF), which was supported by a well-known model of learning conditioning (Rescorla–Wagner). In spite that we found a significant association between accuracy and decision confidence level for MED-OFF and CTRL, analysis of metacognitive sensitivity assessed by type 2 signal detection theory (SDT) revealed only a significant underperformance for patients without medication (MED-OFF). This finding points toward a non-compromising positive effect of dopaminergic medication on metacognition for PD.

Locomotor differences in mice expressing wild-type human α-synuclein

Parkinson's disease manifests as a progressive movement disorder with underlying degeneration of dopaminergic neurons in the substantia nigra, consequent depletion of dopamine levels, and the accumulation of Lewy bodies in the brain. Because α-synuclein (α-Syn) protein is the major component of Lewy bodies, mouse models expressing wild-type or mutant SNCA/α-Syn genes provide a useful tool to investigate canonical characteristics of the disease. We evaluated a mouse model (denoted M20) that expresses human wild-type SNCA gene. The M20 mice showed abnormal locomotor behavior and reduced species-specific home cage activity. However, the direction of behavioral changes was task specific. In comparison with their control littermates, the M20 mice exhibited shorter grip endurance, and longer times to traverse elevated beams, but they descended the vertical pole faster and stayed longer on the accelerated rod than the control mice. The M20 mice were also impaired in burrowing and nest building activities. These results indicate a possible role of α-Syn in motor coordination and the motivation to perform species-specific behaviors in the presymptomatic model of synucleinopathy.

Roles of centromedian parafascicular nuclei of thalamus and cholinergic interneurons in the dorsal striatum in associative learning of environmental events

The thalamus provides a massive input to the striatum, but despite accumulating evidence, the functions of this system remain unclear. It is known, however, that the centromedian (CM) and parafascicular (Pf) nuclei of the thalamus can strongly influence particular striatal neuron subtypes, notably including the cholinergic interneurons of the striatum (CINs), key regulators of striatal function. Here, we highlight the thalamostriatal system through the CM-Pf to striatal CINs. We consider how, by virtue of the direct synaptic connections of the CM and PF, their neural activity contributes to the activity of CINs and striatal projection neurons (SPNs). CM-Pf neurons are strongly activated at sudden changes in behavioral context, such as switches in action-outcome contingency or sequence of behavioral requirements, suggesting that their activity may represent change of context operationalized as associability. Striatal CINs, on the other hand, acquire and loose responses to external events associated with particular contexts. In light of this physiological evidence, we propose a hypothesis of the CM-Pf-CINs system, suggesting that it augments associative learning by generating an associability signal and promotes reinforcement learning guided by reward prediction error signals from dopamine-containing neurons. We discuss neuronal circuit and synaptic organizations based on in vivo/in vitro studies that we suppose to underlie our hypothesis. Possible implications of CM-Pf-CINs dysfunction (or degeneration) in brain diseases are also discussed by focusing on Parkinson's disease.

Effects of lesions of the subthalamic nucleus/zona incerta area and dorsomedial striatum on attentional set-shifting in the rat

Patients with Parkinson's disease (PD) show cognitive impairments, including difficulty in shifting attention between perceptual dimensions of complex stimuli. Inactivation of the subthalamic nucleus (STN) has been shown to be effective in ameliorating the motor abnormalities associated with striatal dopamine (DA) depletion, but it is possible that STN inactivation might result in additional, perhaps attentional, deficits. This study examined the effects of: DA depletion from the dorsomedial striatum (DMS); lesions of the STN area; and the effects of the two lesions together, on the ability to shift attentional set in the rat. In a single session, rats performed the intradimensional/extradimensional (ID/ED) test of attentional set-shifting. This comprises a series of seven, two-choice discriminations, including acquisitions of novel discriminations in which the relevant stimulus is either in the currently attended dimension (ID) or the currently unattended dimension (ED shift) and reversals (REVs) following each acquisition stage. Bilateral lesions were made by injection of 6-hydroxydopamine (6-OHDA) into the DMS, resulting in a selective impairment in reversal learning. Large bilateral ibotenic acid lesions centered on the STN resulted in an increase in trials to criterion in the initial stages, but learning rate improved within the session. There was no evidence of a 'cost' of set-shifting - the ED stage was completed in fewer trials than the ID stage - and neither was there a cost of reversal learning. Strikingly, combined lesions of both regions did not resemble the effects of either lesion alone and resulted in no apparent deficits.

Partial lesion of the nigrostriatal dopamine pathway in rats impairs egocentric learning but not spatial learning or behavioral flexibility

Degeneration of the nigrostriatal dopaminergic system in Parkinson's disease (PD) causes motor dysfunction and cognitive impairment, but the etiology of the cognitive deficits remains unclear. The present study investigated the behavioral effects of partial lesions of the nigrostriatal dopamine (DA) pathway. Rats received bilateral infusions of either 6-hydroxydopamine (6-OHDA) or vehicle into the dorsolateral striatum and were tested in spatial and procedural learning tasks. Compared with intact rats, DA-depleted rats were impaired when the first task they learned required egocentric responses. Intact rats that received prior training on a spatial task were impaired while learning a subsequent body-turn task, suggesting that prior spatial training may compete with egocentric learning in intact but not DA-depleted rats. Spatial discrimination, reversal learning, and switching between allocentric and egocentric strategies were similar in both groups. The results suggest that DA loss that is not associated with gross motor pathology temporarily impairs egocentric, but not allocentric, learning or subsequent behavioral flexibility. (PsycINFO Database Record

Ketamine decreases sensitivity of male rats to misleading negative feedback in a probabilistic reversal-learning task

RATIONALE

Depression is characterized by an excessive attribution of value to negative feedback. This imbalance in feedback sensitivity can be measured using the probabilistic reversal-learning (PRL) task. This task was initially designed for clinical research, but introduction of its rodent version provides a new and much needed translational paradigm to evaluate potential novel antidepressants.

OBJECTIVES

In the present study, we aimed at evaluating the effects of a compound showing clear antidepressant properties-ketamine (KET)-on the sensitivity of rats to positive and negative feedback in the PRL paradigm.

METHODS

We trained healthy rats in an operant version of the PRL task. For successful completion of the task, subjects had to learn to ignore infrequent and misleading feedback, arising from the probabilistic (80:20) nature of the discrimination. Subsequently, we evaluated the effect of KET (5, 10, and 20 mg/kg) on feedback sensitivity 1, 24, and 48 h after administration.

RESULTS

We report that acute administration of the highest dose of KET (20 mg/kg) rapidly and persistently decreases the proportion of lose-shift responses made by rats after receiving negative feedback.

CONCLUSION

Present results suggest that KET decreases negative feedback sensitivity and that changes in this basic neurocognitive function might be one of the factors responsible for its antidepressant action.

Changes in functional organization and white matter integrity in the connectome in Parkinson's disease

Parkinson's disease (PD) leads to dysfunction in multiple cortico-striatal circuits. The neurodegeneration has also been associated with impaired white matter integrity. This structural and functional “disconnection” in PD needs further characterization.

We investigated the structural and functional organization of the PD whole brain connectome consisting of 200 nodes using diffusion tensor imaging and resting-state functional MRI, respectively. Data from 20 non-demented PD patients on dopaminergic medication and 20 matched controls were analyzed using graph theory-based methods. We focused on node strength, clustering coefficient, and local efficiency as measures of local network properties; and network modularity as a measure of information flow.

PD patients showed reduced white matter connectivity in frontoparietal-striatal nodes compared to controls, but no change in modular organization of the white matter tracts. PD group also showed reduction in functional local network metrics in many nodes distributed across the connectome. There was also decreased functional modularity in the core cognitive networks including the default mode and dorsal attention networks, and sensorimotor network, as well as a lack of modular distinction in the orbitofrontal and basal ganglia nodes in the PD group compared to controls.

Our results suggest that despite subtle white matter connectivity changes, the overall structural organization of the PD connectome remains robust at relatively early disease stages. However, there is a breakdown in the functional modular organization of the PD connectome.

•

DTI and rs-fMRI investigation of the connectome in Parkinson's disease (PD)

•

Local network properties and modularity were examined using graph theory.

•

White matter connectivity was decreased in frontoparietal-striatal nodes in PD.

•

Functional modularity of cognitive and motor networks was reduced in PD.

Dopaminergic Medication Modulates Learning from Feedback and Error-Related Negativity in Parkinson's Disease: A Pilot Study

Dopamine systems mediate key aspects of reward learning. Parkinson’s disease (PD) represents a valuable model to study reward mechanisms because both the disease process and the anti-Parkinson medications influence dopamine neurotransmission. The aim of this pilot study was to investigate whether the level of levodopa differently modulates learning from positive and negative feedback and its electrophysiological correlate, the error related negativity (ERN), in PD. Ten PD patients and ten healthy participants performed a two-stage reinforcement learning task. In the Learning Phase, they had to learn the correct stimulus within a stimulus pair on the basis of a probabilistic positive or negative feedback. Three sets of stimulus pairs were used. In the Testing Phase, the participants were tested with novel combinations of the stimuli previously experienced to evaluate whether they learned more from positive or negative feedback. PD patients performed the task both ON- and OFF-levodopa in two separate sessions while they remained on stable therapy with dopamine agonists. The electroencephalogram (EEG) was recorded during the task. PD patients were less accurate in negative than positive learning both OFF- and ON-levodopa. In the OFF-levodopa state they were less accurate than controls in negative learning. PD patients had a smaller ERN amplitude OFF- than ON-levodopa only in negative learning. In the OFF-levodopa state they had a smaller ERN amplitude than controls in negative learning. We hypothesize that high tonic dopaminergic stimulation due to the dopamine agonist medication, combined to the low level of phasic dopamine due to the OFF-levodopa state, could prevent phasic “dopamine dips” indicated by the ERN needed for learning from negative feedback.

Reward sensitivity deficits modulated by dopamine are associated with apathy in Parkinson's disease

Apathy is a debilitating and under-recognized condition that has a significant impact in many neurodegenerative disorders. In Parkinson's disease, it is now known to contribute to worse outcomes and a reduced quality of life for patients and carers, adding to health costs and extending disease burden. However, despite its clinical importance, there remains limited understanding of mechanisms underlying apathy. Here we investigated if insensitivity to reward might be a contributory factor and examined how this relates to severity of clinical symptoms. To do this we created novel ocular measures that indexed motivation level using pupillary and saccadic response to monetary incentives, allowing reward sensitivity to be evaluated objectively. This approach was tested in 40 patients with Parkinson's disease, 31 elderly age-matched control participants and 20 young healthy volunteers. Thirty patients were examined ON and OFF their dopaminergic medication in two counterbalanced sessions, so that the effect of dopamine on reward sensitivity could be assessed. Pupillary dilation to increasing levels of monetary reward on offer provided quantifiable metrics of motivation in healthy subjects as well as patients. Moreover, pupillary reward sensitivity declined with age. In Parkinson's disease, reduced pupillary modulation by incentives was predictive of apathy severity, and independent of motor impairment and autonomic dysfunction as assessed using overnight heart rate variability measures. Reward sensitivity was further modulated by dopaminergic state, with blunted sensitivity when patients were OFF dopaminergic drugs, both in pupillary response and saccadic peak velocity response to reward. These findings suggest that reward insensitivity may be a contributory mechanism to apathy and provide potential new clinical measures for improved diagnosis and monitoring of apathy.media-1vid110.1093/brain/aww188_video_abstractaww188_video_abstract.

Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity

The brain enables animals to behaviorally adapt in order to survive in a complex and dynamic environment, but how reward-oriented behaviors are achieved and computed by its underlying neural circuitry is an open question. To address this concern, we have developed a spiking model of the basal ganglia (BG) that learns to dis-inhibit the action leading to a reward despite ongoing changes in the reward schedule. The architecture of the network features the two pathways commonly described in BG, the direct (denoted D1) and the indirect (denoted D2) pathway, as well as a loop involving striatum and the dopaminergic system. The activity of these dopaminergic neurons conveys the reward prediction error (RPE), which determines the magnitude of synaptic plasticity within the different pathways. All plastic connections implement a versatile four-factor learning rule derived from Bayesian inference that depends upon pre- and post-synaptic activity, receptor type, and dopamine level. Synaptic weight updates occur in the D1 or D2 pathways depending on the sign of the RPE, and an efference copy informs upstream nuclei about the action selected. We demonstrate successful performance of the system in a multiple-choice learning task with a transiently changing reward schedule. We simulate lesioning of the various pathways and show that a condition without the D2 pathway fares worse than one without D1. Additionally, we simulate the degeneration observed in Parkinson's disease (PD) by decreasing the number of dopaminergic neurons during learning. The results suggest that the D1 pathway impairment in PD might have been overlooked. Furthermore, an analysis of the alterations in the synaptic weights shows that using the absolute reward value instead of the RPE leads to a larger change in D1.

Pedunculopontine tegmental nucleus lesions impair probabilistic reversal learning by reducing sensitivity to positive reward feedback

Recent findings indicate that pedunculopontine tegmental nucleus (PPTg) neurons encode reward-related information that is context-dependent. This information is critical for behavioral flexibility when reward outcomes change signaling a shift in response patterns should occur. The present experiment investigated whether NMDA lesions of the PPTg affects the acquisition and/or reversal learning of a spatial discrimination using probabilistic reinforcement. Male Long-Evans rats received a bilateral infusion of NMDA (30nmoles/side) or saline into the PPTg. Subsequently, rats were tested in a spatial discrimination test using a probabilistic learning procedure. One spatial location was rewarded with an 80% probability and the other spatial location rewarded with a 20% probability. After reaching acquisition criterion of 10 consecutive correct trials, the spatial location - reward contingencies were reversed in the following test session. Bilateral and unilateral PPTg-lesioned rats acquired the spatial discrimination test comparable to that as sham controls. In contrast, bilateral PPTg lesions, but not unilateral PPTg lesions, impaired reversal learning. The reversal learning deficit occurred because of increased regressions to the previously 'correct' spatial location after initially selecting the new, 'correct' choice. PPTg lesions also reduced the frequency of win-stay behavior early in the reversal learning session, but did not modify the frequency of lose-shift behavior during reversal learning. The present results suggest that the PPTg contributes to behavioral flexibility under conditions in which outcomes are uncertain, e.g. probabilistic reinforcement, by facilitating sensitivity to positive reward outcomes that allows the reliable execution of a new choice pattern.

Decreased synaptic plasticity in the medial prefrontal cortex underlies short-term memory deficits in 6-OHDA-lesioned rats

Parkinson's disease (PD) is characterized by motor dysfunction associated with dopaminergic degeneration in the dorsolateral striatum (DLS). However, motor symptoms in PD are often preceded by short-term memory deficits, which have been argued to involve deregulation of medial prefrontal cortex (mPFC). We now used a 6-hydroxydopamine (6-OHDA) rat PD model to explore if alterations of synaptic plasticity in DLS and mPFC underlie short-term memory impairments in PD prodrome. The bilateral injection of 6-OHDA (20μg/hemisphere) in the DLS caused a marked loss of dopaminergic neurons in the substantia nigra (>80%) and decreased monoamine levels in the striatum and PFC, accompanied by motor deficits evaluated after 21 days in the open field and accelerated rotarod. A lower dose of 6-OHDA (10μg/hemisphere) only induced a partial degeneration (about 60%) of dopaminergic neurons in the substantia nigra with no gross motor impairments, thus mimicking an early premotor stage of PD. Notably, 6-OHDA (10μg)-lesioned rats displayed decreased monoamine levels in the PFC as well as short-term memory deficits evaluated in the novel object discrimination and in the modified Y-maze tasks; this was accompanied by a selective decrease in the amplitude of long-term potentiation in the mPFC, but not in DLS, without changes of synaptic transmission in either brain regions. These results indicate that the short-term memory dysfunction predating the motor alterations in the 6-OHDA model of PD is associated with selective changes of information processing in PFC circuits, typified by persistent changes of synaptic plasticity.

Altered brain activation in a reversal learning task unmasks adaptive changes in cognitive control in writer's cramp

Previous receptor binding studies suggest dopamine function is altered in the basal ganglia circuitry in task-specific dystonia, a condition characterized by contraction of agonist and antagonist muscles while performing specific tasks. Dopamine plays a role in reward-based learning. Using fMRI, this study compared 31 right-handed writer's cramp patients to 35 controls in reward-based learning of a probabilistic reversal-learning task. All subjects chose between two stimuli and indicated their response with their left or right index finger. One stimulus response was rewarded 80%, the other 20%. After contingencies reversal, the second stimulus response was rewarded in 80%. We further linked the DRD2/ANKK1-TaqIa polymorphism, which is associated with 30% reduction of the striatal dopamine receptor density with reward-based learning and assumed impaired reversal learning in A + subjects. Feedback learning in patients was normal. Blood-oxygen level dependent (BOLD) signal in controls increased with negative feedback in the insula, rostral cingulate cortex, middle frontal gyrus and parietal cortex (pFWE < 0.05). In comparison to controls, patients showed greater increase in BOLD activity following negative feedback in the dorsal anterior cingulate cortex (BA32). The genetic status was not correlated with the BOLD activity. The Brodmann area 32 (BA32) is part of the dorsal anterior cingulate cortex (dACC) that plays an important role in coordinating and integrating information to guide behavior and in reward-based learning. The dACC is connected with the basal ganglia-thalamo-loop modulated by dopaminergic signaling. This finding suggests disturbed integration of reinforcement history in decision making and implicate that the reward system might contribute to the pathogenesis in writer's cramp.

The initial stage of reversal learning is impaired in mice hemizygous for the vesicular glutamate transporter (VGluT1)

Behavioral flexibility is a complex cognitive function that is necessary for survival in changeable environments. Patients with schizophrenia or Parkinson's disease often suffer from cognitive rigidity, reducing their capacity to function in society. Patients and rodent models with focal lesions in the prefrontal cortex (PFC) show similar rigidity, owing to the loss of PFC regulation of subcortical reward circuits involved in behavioral flexibility. The vesicular glutamate transporter (VGluT1) is preferentially expressed at modulatory synapses, including PFC neurons that project to components of the reward circuit (such as the nucleus accumbens, NAc). VGluT1(+/-) mice display behavioral phenotypes matching many symptoms of schizophrenia, and VGluT1 expression is reduced in the PFC of patients with schizophrenia and Parkinson's disease. Thus, it appears likely that VGluT1-expressing synapses from PFC play a key role in behavioral flexibility. To examine this hypothesis, we studied behavioral flexibility in VGluT1(+/-) mice by testing reversal learning in a visual discrimination task. Here, we show that VGluT1(+/-) mice acquired the initial visual discrimination at the same rate as controls. However, they failed to suppress responses to the previously rewarded stimulus following reversal of reward contingencies. Thus, our genetic disruption of modulatory glutamatergic signaling, including that arising from PFC, appears to have impaired the first stage of reversal learning (extinguishing responses to previously rewarded stimuli). Our data show that this deficit stems from a preservative phenotype. These findings suggest that glutamatergic regulation from the cortex is important for behavioral flexibility and the disruption of this pathway may be relevant in diseases such as schizophrenia.

Baseline frontostriatal-limbic connectivity predicts reward-based memory formation

Reward mediates the acquisition and long-term retention of procedural skills in humans. Yet, learning under rewarded conditions is highly variable across individuals and the mechanisms that determine interindividual variability in rewarded learning are not known. We postulated that baseline functional connectivity in a large-scale frontostriatal-limbic network could predict subsequent interindividual variability in rewarded learning. Resting-state functional MRI was acquired in two groups of subjects (n = 30) who then trained on a visuomotor procedural learning task with or without reward feedback. We then tested whether baseline functional connectivity within the frontostriatal-limbic network predicted memory strength measured immediately, 24 h and 1 month after training in both groups. We found that connectivity in the frontostriatal-limbic network predicted interindividual variability in the rewarded but not in the unrewarded learning group. Prediction was strongest for long-term memory. Similar links between connectivity and reward-based memory were absent in two control networks, a fronto-parieto-temporal language network and the dorsal attention network. The results indicate that baseline functional connectivity within the frontostriatal-limbic network successfully predicts long-term retention of rewarded learning.

Preferential involvement by nucleus accumbens shell in mediating probabilistic learning and reversal shifts

Different subregions of nucleus accumbens (NAc) have been implicated in reward seeking, promoting flexible approach responses, suppressing nonrewarded actions, and facilitating shifts between different discrimination strategies. Interestingly, the NAc does not appear to mediate shifting between stimulus-reward associations (i.e., reversal learning) when reinforcement is predictable. How these nuclei may facilitate flexible response strategies when reward delivery is uncertain remains unclear. We investigated the effects of inactivation of the NAc shell and core on probabilistic reversal learning using an operant task wherein a "correct" response delivered reward on 80% of trials, and an "incorrect" response was reinforced on 20% of trials. Reinforcement contingencies were reversed repeatedly within a session. In well-trained rats, shell inactivation reduced the number of reversals completed and selectively reduced win-stay behavior. This impairment was apparent during the first discrimination, indicating a more general deficit in the use of probabilistic reward feedback to guide action selection. Shell inactivation also impaired reversal performance on a similar task where correct/incorrect choices always/never delivered reward. However, this impairment only emerged after both levers had been associated with reward. Inactivation of NAc core did not impair reversal performance but increased latencies to approach the response levers. These results suggest the NAc shell and core facilitate reward seeking in a distinct yet complementary manner when the relationship between specific actions and reward is uncertain or ambiguous and cognitive flexibility is required. The core promotes approach toward reward-associated stimuli, whereas the shell refines response selection to those specific actions more likely to yield reward.

Prospective Optimization

Human performance approaches that of an ideal observer and optimal actor in some perceptual and motor tasks. These optimal abilities depend on the capacity of the cerebral cortex to store an immense amount of information and to flexibly make rapid decisions. However, behavior only approaches these limits after a long period of learning while the cerebral cortex interacts with the basal ganglia, an ancient part of the vertebrate brain that is responsible for learning sequences of actions directed toward achieving goals. Progress has been made in understanding the algorithms used by the brain during reinforcement learning, which is an online approximation of dynamic programming. Humans also make plans that depend on past experience by simulating different scenarios, which is called prospective optimization. The same brain structures in the cortex and basal ganglia that are active online during optimal behavior are also active offline during prospective optimization. The emergence of general principles and algorithms for goal-directed behavior has consequences for the development of autonomous devices in engineering applications.

Toward a semi-self-paced EEG brain computer interface: decoding initiation state from non-initiation state in dedicated time slots

Brain computer interfaces (BCIs) offer a broad class of neurologically impaired individuals an alternative means to interact with the environment. Many BCIs are "synchronous" systems, in which the system sets the timing of the interaction and tries to infer what control command the subject is issuing at each prompting. In contrast, in "asynchronous" BCIs subjects pace the interaction and the system must determine when the subject's control command occurs. In this paper we propose a new idea for BCI which draws upon the strengths of both approaches. The subjects are externally paced and the BCI is able to determine when control commands are issued by decoding the subject's intention for initiating control in dedicated time slots. A single task with randomly interleaved trials was designed to test whether it can be used as stimulus for inducing initiation and non-initiation states when the sensory and motor requirements for the two types of trials are very nearly identical. Further, the essential problem on the discrimination between initiation state and non-initiation state was studied. We tested the ability of EEG spectral power to distinguish between these two states. Among the four standard EEG frequency bands, beta band power recorded over parietal-occipital cortices provided the best performance, achieving an average accuracy of 86% for the correct classification of initiation and non-initiation states. Moreover, delta band power recorded over parietal and motor areas yielded a good performance and thus could also be used as an alternative feature to discriminate these two mental states. The results demonstrate the viability of our proposed idea for a BCI design based on conventional EEG features. Our proposal offers the potential to mitigate the signal detection challenges of fully asynchronous BCIs, while providing greater flexibility to the subject than traditional synchronous BCIs.

Transgenic rodent models to study alpha-synuclein pathogenesis, with a focus on cognitive deficits

The aggregation of alpha-synuclein (aSyn) has been implicated in a number of degenerative diseases collectively termed synucleinopathies. Although most cases of synucleinopathies are idiopathic in nature, there are familial cases of these diseases that are due to mutations or multiplications of the gene coding for aSyn. Two of the most common synucleinopathies are Parkinson's disease (PD) and dementia with Lewy bodies (DLB). Both of these diseases present with cognitive deficits, though with different clinical and temporal features. In PD, cognitive deficits are subtle, may occur before the onset of the classical motor symptoms, and only occasionally lead to dementia in the later stages of the disease. In contrast, dementia is the dominating feature of DLB from the disease onset. The impact of aSyn pathology on the development of neurobiological and behavioral impairments can be investigated using rodent models. There are currently several lines of transgenic mice overexpressing wild-type or mutated aSyn under various promoters. This review will provide an updated synopsis of the mouse lines available, summarize their cognitive deficits, and reflect on how deficits observed in these mice relate to the disease process in humans. In addition, we will review mouse lines where knockout strategies have been applied to study the effects of aSyn on various cognitive tasks and comment on how these lines have been used in combination with other transgenic strains, or with human aSyn overexpression by viral vectors. Finally, we will discuss the recent advent of bacterial artificial chromosome (BAC) transgenic models of PD and their effectiveness in modeling cognitive decline in PD.

Facilitative effects of bi-hemispheric tDCS in cognitive deficits of Parkinson disease patients

Parkinson's disease (PD) is a progressive neurodegenerative disorder, primarily characterized by motor symptoms such as tremor, rigidity, bradykinesia, stiffness, slowness and impaired equilibrium. Although the motor symptoms have been the focus in PD, slight cognitive deficits are commonly found in non-demented and non-depressed PD patients, even in early stages of the disease, which have been linked to the subsequent development of pathological dementia. Thus, strongly reducing the quality of life (QoL). Both levodopa therapy and deep brain stimulation (DBS) have yield controversial results concerning the cognitive symptoms amelioration in PD patients. That does not seems to be the case with transcranial direct current stimulation (tDCS), although better stimulation parameters are needed. Therefore we hypothesize that simultaneously delivering cathodal tDCS (or ctDCS), over the right prefrontal cortex delivered with anodal tDCS (or atDCS) to left prefrontal cortex could be potentially beneficial for PD patients, either by mechanisms of homeostatic plasticity and by increases in the extracellular dopamine levels over the striatum.