Role of [corrected] nigrostriatal dopamine system in learning to perform sequential motor tasks in a predictive manner

Dopaminergic Pathways in Neuroplasticity After Stroke and Vagus Nerve Stimulation

Stroke remains a significant cause of disability worldwide. In addition to multidisciplinary rehabilitation approaches, various forms of technology, including vagus nerve stimulation, have emerged to facilitate neuroplasticity and, thereby, improve functional status after stroke. Vagus nerve stimulation was recently approved by the Food and Drug Administration, but questions remain regarding its mechanism of action. Here, a potential role for dopaminergic signaling is considered. This review first examines evidence that dopamine is important to neuroplasticity after stroke. Next, 2 different dopaminergic pathways are considered potential mechanisms underlying vagus nerve stimulation-related benefits after stroke, direct modulation of brain dopaminergic pathways, and engagement of systemic dopaminergic pathways such as those found in the gut-brain axis. A contribution of dopamine signaling to vagus nerve stimulation efficacy could have therapeutic implications that extend to a precision medicine approach to stroke rehabilitation.

Alterations in amygdala subregions-Default mode network connectivity after treatment in patients with schizophrenia

Disrupted connectivity in the default mode network (DMN) during resting-state functional MRI (rs-fMRI) is well-documented in schizophrenia (SCZ). The amygdala, a key component in the neurobiology of SCZ, comprises distinct subregions that may exert varying effects on the disorder. This study aimed to investigate variations in functional connectivity (FC) between distinct amygdala subregions and the DMN in SCZ individuals and explore the effects of treatment on these connections. Fifty-six SCZ patients and 51 healthy controls underwent FC analysis and questionnaire surveys during resting state. The amygdala was selected as the region of interest (ROI) and subdivided into four parts. Changes in FC were examined, and correlations between questionnaire scores and brain activity were explored. Pre-treatment, SCZ patients exhibited reduced FC between the amygdala and DMN compared to HCs. After treatment, significant differences persisted in the right medial amygdala, while other regions did not differ significantly from controls. In addition, PANSS scores positively correlated with FC between the Right Medial Amygdala and the left SMFC (r = .347, p = .009), while RBANS5A scores showed a positive correlation with FC between the Left Lateral Amygdala and the right MTG (rho = -.347, p = .009). The rsFC between the amygdala and the DMN plays a crucial role in the treatment mechanisms of SCZ. This could provide a promising predictive indicator for understanding the neural mechanisms behind treatment and symptomatic improvement.

Alteration of Postural Reactions in Rats with Different Levels of Dopamine Depletion

Dopamine (DA) is the critical neurotransmitter involved in the unconscious control of muscle tone and body posture. We evaluated the general motor capacities and muscle responses to postural disturbance in three conditions: normal DA level (wild-type rats, WT), mild DA deficiency (WT after administration of α-methyl-p-tyrosine-AMPT, that blocks DA synthesis), and severe DA depletion (DAT-KO rats after AMPT). The horizontal displacements in WT rats elicited a multi-component EMG corrective response in the flexor and extensor muscles. Similar to the gradual progression of DA-related diseases, we observed different degrees of bradykinesia, rigidity, and postural instability after AMPT. The mild DA deficiency impaired the initiation pattern of corrective responses, specifically delaying the extensor muscles' activity ipsilaterally to displacement direction and earlier extensor activity from the opposite side. DA depletion in DAT-KO rats after AMPT elicited tremors, general stiffness, and akinesia, and caused earlier response to horizontal displacements in the coactivated flexor and extensor muscles bilaterally. The data obtained show the specific role of DA in postural reactions and suggest that this experimental approach can be used to investigate sensorimotor control in different dopamine-deficient states and to model DA-related diseases.

Hyperconnectivity of Two Separate Long-Range Cholinergic Systems Contributes to the Reorganization of the Brain Functional Connectivity during Nicotine Withdrawal in Male Mice

Chronic nicotine results in dependence with withdrawal symptoms on discontinuation of use, through desensitization of nicotinic acetylcholine receptors and altered cholinergic neurotransmission. Nicotine withdrawal is associated with increased whole-brain functional connectivity and decreased network modularity; however, the role of cholinergic neurons in those changes is unknown. To identify the contribution of nicotinic receptors and cholinergic regions to changes in the functional network, we analyzed the contribution of the main cholinergic regions to brain-wide activation of the immediate early-gene Fos during withdrawal in male mice and correlated these changes with the expression of nicotinic receptor mRNA throughout the brain. We show that the main functional connectivity modules included the main long-range cholinergic regions, which were highly synchronized with the rest of the brain. However, despite this hyperconnectivity, they were organized into two anticorrelated networks that were separated into basal forebrain-projecting and brainstem-thalamic-projecting cholinergic regions, validating a long-standing hypothesis of the organization of the brain cholinergic systems. Moreover, baseline (without nicotine) expression of Chrna2, Chrna3, Chrna10, and Chrnd mRNA of each brain region correlated with withdrawal-induced changes in Fos expression. Finally, by mining the Allen Brain mRNA expression database, we were able to identify 1755 gene candidates and three pathways (Sox2-Oct4-Nanog, JAK-STAT, and MeCP2-GABA) that may contribute to nicotine withdrawal-induced Fos expression. These results identify the dual contribution of the basal forebrain and brainstem-thalamic cholinergic systems to whole-brain functional connectivity during withdrawal; and identify nicotinic receptors and novel cellular pathways that may be critical for the transition to nicotine dependence.

Hyperconnectivity of two separate long-range cholinergic systems contributes to the reorganization of the brain functional connectivity during nicotine withdrawal in male mice

Chronic nicotine results in dependence with withdrawal symptoms upon discontinuation of use, through desensitization of nicotinic acetylcholine receptors and altered cholinergic neurotransmission. Nicotine withdrawal is associated with increased whole-brain functional connectivity and decreased network modularity, however, the role of cholinergic neurons in those changes is unknown. To identify the contribution of nicotinic receptors and cholinergic regions to changes in the functional network, we analyzed the contribution of the main cholinergic regions to brain-wide activation of the immediate early-gene FOS during withdrawal in male mice and correlated these changes with the expression of nicotinic receptor mRNA throughout the brain. We show that the main functional connectivity modules included the main long-range cholinergic regions, which were highly synchronized with the rest of the brain. However, despite this hyperconnectivity they were organized into two anticorrelated networks that were separated into basal forebrain projecting and brainstem-thalamic projecting cholinergic regions, validating a long-standing hypothesis of the organization of the brain cholinergic systems. Moreover, baseline (without nicotine) expression of Chrna2 , Chrna3 , Chrna10 , and Chrnd mRNA of each brain region correlated with withdrawal-induced changes in FOS expression. Finally, by mining the Allen Brain mRNA expression database, we were able to identify 1755 gene candidates and three pathways (Sox2-Oct4-Nanog, JAK-STAT, and MeCP2-GABA) that may contribute to nicotine withdrawal-induced FOS expression. These results identify the dual contribution of the basal forebrain and brainstem-thalamic cholinergic systems to whole-brain functional connectivity during withdrawal; and identify nicotinic receptors and novel cellular pathways that may be critical for the transition to nicotine dependence.

Significance Statement

Discontinuation of nicotine use in dependent users is associated with increased whole-brain activation and functional connectivity and leads to withdrawal symptoms. Here we investigated the contribution of the nicotinic cholinergic receptors and main cholinergic projecting brain areas in the whole-brain changes associated with withdrawal. This not only allowed us to visualize and confirm the previously described duality of the cholinergic brain system using this novel methodology, but also identify nicotinic receptors together with 1751 other genes that contribute, and could thus be targets for treatments against, nicotine withdrawal and dependence.

Neurological soft signs and structural network changes: a longitudinal analysis in first-episode schizophrenia

BACKGROUND

Neurological soft signs (NSS) are often reported in patients with schizophrenia and may vary with psychopathological symptoms during the course of disease. Many cross-sectional neuroimaging studies have shown that NSS are associated with disturbed network connectivity in schizophrenia. However, it remains unclear how these associations change over time during the course of disorder.

METHODS

In present study, 20 patients with first-episode schizophrenia and 20 controls underwent baseline structural magnetic resonance imaging (MRI) scan and at one-year follow-up. Structural network characteristics of patients and controls were analyzed using graph theoretical approach based on MRI data. NSS were assessed using the Heidelberg scale.

RESULTS

At baseline, patients demonstrated significant changes of the local network properties mainly involving regions of the cortical-subcortical-cerebellar circuits compared to healthy controls. For further analysis, the whole patient group was dichotomized into a NSS-persisting and NSS-decreasing subgroup. After one-year follow-up, the NSS-persisting subgroup showed decreased betweenness in right inferior opercular frontal cortex, left superior medial frontal cortex, left superior temporal cortex, right putamen and cerebellum vermis and increased betweenness in right lingual cortex. However, the NSS-decreasing subgroup exhibited only localized changes in right middle temporal cortex, right insula and right fusiform with decreased betweenness, and in left lingual cortex with increased betweenness.

CONCLUSIONS

These findings provide evidence for brain network reorganization subsequent to clinical disease manifestation in patients with first-episode schizophrenia, and support the hypothesis that persisting NSS refer to progressive brain network abnormalities in patients with schizophrenia. Therefore, NSS could help to establish a better prognosis in first-episode schizophrenia patients.

Determination of the Unilaterally Damaged Region May Depend on the Asymmetry of Carotid Blood Flow Velocity in Hemiparkinsonian Monkey: A Pilot Study

The hemiparkinsonian nonhuman primate model induced by unilateral injection of 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine (MPTP) into the carotid artery is used to study Parkinson's disease. However, there have been no studies that the contralateral distribution of MPTP via the cerebral collateral circulation is provided by both the circle of Willis (CoW) and connections of the carotid artery. To investigate whether MPTP-induced unilaterally damaged regions were determined by asymmetrical cerebral blood flow, the differential asymmetric damage of striatal subregions, and examined structural asymmetries in a circle of Willis, and blood flow velocity of the common carotid artery were observed in three monkeys that were infused with MPTP through the left internal carotid artery. Lower flow velocity in the ipsilateral common carotid artery and a higher ratio of ipsilateral middle cerebral artery diameter to anterior cerebral artery diameter resulted in unilateral damage. Additionally, the unilateral damaged monkey observed the apomorphine-induced contralateral rotation behavior and the temporary increase of plasma RANTES. Contrastively, higher flow velocity in the ipsilateral common carotid artery was observed in the bilateral damaged monkey. It is suggested that asymmetry of blood flow velocity and structural asymmetry of the circle of Willis should be taken into consideration when establishing more efficient hemiparkinsonian nonhuman primate models.

Learning-induced changes in the neural circuits underlying motor sequence execution

As the old adage goes: practice makes perfect. Yet, the neural mechanisms by which rote repetition transforms a halting behavior into a fluid, effortless, and "automatic" action are not well understood. Here we consider the possibility that well-practiced motor sequences, which initially rely on higher-level decision-making circuits, become wholly specified in lower-level control circuits. We review studies informing this idea, discuss the constraints on such shift in control, and suggest approaches to pinpoint circuit-level changes associated with motor sequence learning.

Hierarchical Reinforcement Learning, Sequential Behavior, and the Dorsal Frontostriatal System

To effectively behave within ever-changing environments, biological agents must learn and act at varying hierarchical levels such that a complex task may be broken down into more tractable subtasks. Hierarchical reinforcement learning (HRL) is a computational framework that provides an understanding of this process by combining sequential actions into one temporally extended unit called an option. However, there are still open questions within the HRL framework, including how options are formed and how HRL mechanisms might be realized within the brain. In this review, we propose that the existing human motor sequence literature can aid in understanding both of these questions. We give specific emphasis to visuomotor sequence learning tasks such as the discrete sequence production task and the M × N (M steps × N sets) task to understand how hierarchical learning and behavior manifest across sequential action tasks as well as how the dorsal cortical-subcortical circuitry could support this kind of behavior. This review highlights how motor chunks within a motor sequence can function as HRL options. Furthermore, we aim to merge findings from motor sequence literature with reinforcement learning perspectives to inform experimental design in each respective subfield.

Novelty exposure induces stronger sensorimotor representations during a manual adaptation task

Active exploration of novel spatial environments enhances memory for subsequently presented explicit, declarative information in humans. These effects have been attributed to novelty promoting dopamine release via mesolimbic dopaminergic pathways in the brain. As procedural motor learning has been linked to dopamine as well, we predict that novelty effects extend to this domain. To test this hypothesis, the present study examined whether spatial novelty exploration benefits subsequent sensorimotor adaptation. Participants explored either two different virtual environments (i.e., novelty condition; n = 210) or two identical environments (i.e., familiar condition; n = 253). They then performed a manual adaptation task in which they had to adapt joystick movements to a visual perturbation. We assessed the rate of adaptation following the introduction of this perturbation, and the rate of deadaptation following its removal. While results showed reliable adaptation patterns and similar adaptation rates across both conditions, individuals in the novelty condition showed slower deadaptation. This suggests that exposure to spatial novelty induced stronger sensorimotor representations during adaptation, potentially through novelty-induced dopaminergic effects in mesocortical and/or nigrostriatal pathways. Novelty exposure may be employed to promote motor learning on tasks that require precision movements in altered sensory contexts, for example, in astronauts moving in microgravity or patients with impaired motor processing.

Nigrostriatal dopamine signals sequence-specific action-outcome prediction errors

Dopamine has been suggested to encode cue-reward prediction errors during Pavlovian conditioning, signaling discrepancies between actual versus expected reward predicted by the cues.^1-5 While this theory has been widely applied to reinforcement learning concerning instrumental actions, whether dopamine represents action-outcome prediction errors and how it controls sequential behavior remain largely unknown. The vast majority of previous studies examining dopamine responses primarily have used discrete reward-predictive stimuli,^1-15 whether Pavlovian conditioned stimuli for which no action is required to earn reward or explicit discriminative stimuli that essentially instruct an animal how and when to respond for reward. Here, by training mice to perform optogenetic intracranial self-stimulation, we examined how self-initiated goal-directed behavior influences nigrostriatal dopamine transmission during single and sequential instrumental actions, in behavioral contexts with minimal overt changes in the animal's external environment. We found that dopamine release evoked by direct optogenetic stimulation was dramatically reduced when delivered as the consequence of the animal's own action, relative to non-contingent passive stimulation. This dopamine suppression generalized to food rewards was specific to the reinforced action, was temporally restricted to counteract the expected outcome, and exhibited sequence-selectivity consistent with hierarchical control of sequential behavior. These findings demonstrate that nigrostriatal dopamine signals sequence-specific prediction errors in action-outcome associations, with fundamental implications for reinforcement learning and instrumental behavior in health and disease.

Diverse midbrain dopaminergic neuron subtypes and implications for complex clinical symptoms of Parkinson's disease

Parkinson’s disease (PD), the most common degenerative movement disorder, is clinically manifested with various motor and non-motor symptoms. Degeneration of midbrain substantia nigra pas compacta (SNc) dopaminergic neurons (DANs) is generally attributed to the motor syndrome. The underlying neuronal mechanisms of non-motor syndrome are largely unexplored. Besides SNc, midbrain ventral tegmental area (VTA) DANs also produce and release dopamine and modulate movement, reward, motivation, and memory. Degeneration of VTA DANs also occurs in postmortem brains of PD patients, implying an involvement of VTA DANs in PD-associated non-motor symptoms. However, it remains to be established that there is a distinct segregation of different SNc and VTA DAN subtypes in regulating different motor and non-motor functions, and that different DAN subpopulations are differentially affected by normal ageing or PD. Traditionally, the distinction among different DAN subtypes was mainly based on the location of cell bodies and axon terminals. With the recent advance of single cell RNA sequencing technology, DANs can be readily classified based on unique gene expression profiles. A combination of specific anatomic and molecular markers shows great promise to facilitate the identification of DAN subpopulations corresponding to different behavior modules under normal and disease conditions. In this review, we first summarize the recent progress in characterizing genetically, anatomically, and functionally diverse midbrain DAN subtypes. Then, we provide perspectives on how the preclinical research on the connectivity and functionality of DAN subpopulations improves our current understanding of cell-type and circuit specific mechanisms of the disease, which could be critically informative for designing new mechanistic treatments.

Motor Sequence Learning Deficits in Idiopathic Parkinson's Disease Are Associated With Increased Substantia Nigra Activity

Previous studies have shown that persons with Parkinson’s disease (pwPD) share specific deficits in learning new sequential movements, but the neural substrates of this impairment remain unclear. In addition, the degree to which striatal dopaminergic denervation in PD affects the cortico-striato-thalamo-cerebellar motor learning network remains unknown. We aimed to answer these questions using fMRI in 16 pwPD and 16 healthy age-matched control subjects while they performed an implicit motor sequence learning task. While learning was absent in both pwPD and controls assessed with reaction time differences between sequential and random trials, larger error-rates during the latter suggest that at least some of the complex sequence was encoded. Moreover, we found that while healthy controls could improve general task performance indexed by decreased reaction times across both sequence and random blocks, pwPD could not, suggesting disease-specific deficits in learning of stimulus-response associations. Using fMRI, we found that this effect in pwPD was correlated with decreased activity in the hippocampus over time. Importantly, activity in the substantia nigra (SN) and adjacent bilateral midbrain was specifically increased during sequence learning in pwPD compared to healthy controls, and significantly correlated with sequence-specific learning deficits. As increased SN activity was also associated (on trend) with higher doses of dopaminergic medication as well as disease duration, the results suggest that learning deficits in PD are associated with disease progression, indexing an increased drive to recruit dopaminergic neurons in the SN, however, unsuccessfully. Finally, there were no differences between pwPD and controls in task modulation of the cortico-striato-thalamo-cerebellar network. However, a restricted nigral-striatal model showed that negative modulation of SN to putamen connection was larger in pwPD compared to controls during random trials, while no differences between the groups were found during sequence learning. We speculate that learning-specific SN recruitment leads to a relative increase in SN- > putamen connectivity, which returns to a pathological reduced state when no learning takes place.

Imaging of Functional Brain Circuits during Acquisition and Memory Retrieval in an Aversive Feedback Learning Task: Single Photon Emission Computed Tomography of Regional Cerebral Blood Flow in Freely Behaving Rats

Active avoidance learning is a complex form of aversive feedback learning that in humans and other animals is essential for actively coping with unpleasant, aversive, or dangerous situations. Since the functional circuits involved in two-way avoidance (TWA) learning have not yet been entirely identified, the aim of this study was to obtain an overall picture of the brain circuits that are involved in active avoidance learning. In order to obtain a longitudinal assessment of activation patterns in the brain of freely behaving rats during different stages of learning, we applied single-photon emission computed tomography (SPECT). We were able to identify distinct prefrontal cortical, sensory, and limbic circuits that were specifically recruited during the acquisition and retrieval phases of the two-way avoidance learning task.

New roles for dopamine in motor skill acquisition: lessons from primates, rodents, and songbirds

Motor learning is a core aspect of human life and appears to be ubiquitous throughout the animal kingdom. Dopamine, a neuromodulator with a multifaceted role in synaptic plasticity, may be a key signaling molecule for motor skill learning. Though typically studied in the context of reward-based associative learning, dopamine appears to be necessary for some types of motor learning. Mesencephalic dopamine structures are highly conserved among vertebrates, as are some of their primary targets within the basal ganglia, a subcortical circuit important for motor learning and motor control. With a focus on the benefits of cross-species comparisons, this review examines how "model-free" and "model-based" computational frameworks for understanding dopamine's role in associative learning may be applied to motor learning. The hypotheses that dopamine could drive motor learning either by functioning as a reward prediction error, through passive facilitating of normal basal ganglia activity, or through other mechanisms are examined in light of new studies using humans, rodents, and songbirds. Additionally, new paradigms that could enhance our understanding of dopamine's role in motor learning by bridging the gap between the theoretical literature on motor learning in humans and other species are discussed.

Movement Decomposition in the Primary Motor Cortex

A complex action can be described as the composition of a set of elementary movements. While both kinematic and dynamic elements have been proposed to compose complex actions, the structure of movement decomposition and its neural representation remain unknown. Here, we examined movement decomposition by modeling the temporal dynamics of neural populations in the primary motor cortex of macaque monkeys performing forelimb reaching movements. Using a hidden Markov model, we found that global transitions in the neural population activity are associated with a consistent segmentation of the behavioral output into acceleration and deceleration epochs with directional selectivity. Single cells exhibited modulation of firing rates between the kinematic epochs, with abrupt changes in spiking activity timed with the identified transitions. These results reveal distinct encoding of acceleration and deceleration phases at the level of M1, and point to a specific pattern of movement decomposition that arises from the underlying neural activity. A similar approach can be used to probe the structure of movement decomposition in different brain regions, possibly controlling different temporal scales, to reveal the hierarchical structure of movement composition.

Lesion of striatal patches disrupts habitual behaviors and increases behavioral variability

Habits are automated behaviors that are insensitive to changes in behavioral outcomes. Habitual responding is thought to be mediated by the striatum, with medial striatum guiding goal-directed action and lateral striatum promoting habits. However, interspersed throughout the striatum are neurochemically differing subcompartments known as patches, which are characterized by distinct molecular profiles relative to the surrounding matrix tissue. These structures have been thoroughly characterized neurochemically and anatomically, but little is known regarding their function. Patches have been shown to be selectively activated during inflexible motor stereotypies elicited by stimulants, suggesting that patches may subserve habitual behaviors. To explore this possibility, we utilized transgenic mice (Sepw1 NP67) preferentially expressing Cre recombinase in striatal patch neurons to target these neurons for ablation with a virus driving Cre-dependent expression of caspase 3. Mice were then trained to press a lever for sucrose rewards on a variable interval schedule to elicit habitual responding. Mice were not impaired on the acquisition of this task, but lesioning striatal patches disrupted behavioral stability across training, and lesioned mice utilized a more goal-directed behavioral strategy during training. Similarly, when mice were forced to omit responses to receive sucrose rewards, habitual responding was impaired in lesioned mice. To rule out effects of lesion on motor behaviors, mice were then tested for impairments in motor learning on a rotarod and locomotion in an open field. We found that patch lesions partially impaired initial performance on the rotarod without modifying locomotor behaviors in open field. This work indicates that patches promote behavioral stability and habitual responding, adding to a growing literature implicating striatal patches in stimulus-response behaviors.

Hierarchical Action Control: Adaptive Collaboration Between Actions and Habits

It is now commonly accepted that instrumental actions can reflect goal-directed control; i.e., they can show sensitivity to changes in the relationship to and the value of their consequences. With overtraining, stress, neurodegeneration, psychiatric conditions, or after exposure to various drugs of abuse, goal-directed control declines and instrumental actions are performed independently of their consequences. Although this latter insensitivity has been argued to reflect the development of habitual control, the lack of a positive definition of habits has rendered this conclusion controversial. Here we consider various alternative definitions of habit, including recent suggestions they reflect chunked action sequences, to derive criteria with which to categorize responses as habitual. We consider various theories regarding the interaction between goal-directed and habitual controllers and propose a collaborative model based on their hierarchical integration. We argue that this model is consistent with the available data, can be instantiated both at an associative level and computationally and generates interesting predictions regarding the influence of this collaborative integration on behavior.

Contributions of the basal ganglia to action sequence learning and performance

Animals engage in intricately woven and choreographed action sequences that are constructed from trial-and-error learning. The mechanisms by which the brain links together individual actions which are later recalled as fluid chains of behavior are not fully understood, but there is broad consensus that the basal ganglia play a crucial role in this process. This paper presents a comprehensive review of the role of the basal ganglia in action sequencing, with a focus on whether the computational framework of reinforcement learning can capture key behavioral features of sequencing and the neural mechanisms that underlie them. While a simple neurocomputational model of reinforcement learning can capture key features of action sequence learning, this model is not sufficient to capture goal-directed control of sequences or their hierarchical representation. The hierarchical structure of action sequences, in particular, poses a challenge for building better models of action sequencing, and it is in this regard that further investigations into basal ganglia information processing may be informative.

STIL: a multi-function protein required for dopaminergic neural proliferation, protection, and regeneration

Degeneration of dopaminergic (DA) neurons in the brain is the major cause for Parkinson’s disease (PD). While genetic loci and cellular pathways involved in DA neuron proliferation have been well documented, the genetic and molecular and cellular basis of DA cell survival remains to be elucidated. Recently, studies aimed to uncover the mechanisms of DA neural protection and regeneration have been reported. One of the most recent discoveries, i.e., multi-function of human oncogene SCL/TAL interrupting locus (Stil) in DA cell proliferation, neural protection, and regeneration, created a new field for studying DA cells and possible treatment of PD. In DA neurons, Stil functions through the Sonic hedgehog (Shh) pathway by releasing the inhibition of SUFU to GLI1, and thereby enhances Shh-target gene transcription required for neural proliferation, protection, and regeneration. In this review article, we will highlight some of the new findings from researches relate to Stil in DA cells using zebrafish models and cultured mammalian PC12 cells. The findings may provide the proof-of-concept for the development of Stil as a tool for diagnosis and/or treatment of human diseases, particularly those caused by DA neural degeneration.

Dynamic cortical participation during bilateral, cyclical ankle movements: Effects of Parkinson's disease

Parkinson’s disease (PD) is known to increase asymmetry and variability of bilateral movements. However, the mechanisms of such abnormalities are not fully understood. Here, we aimed to investigate whether kinematic abnormalities are related to cortical participation during bilateral, cyclical ankle movements, which required i) maintenance of a specific frequency and ii) bilateral coordination of the lower limbs in an anti-phasic manner. We analyzed electroencephalographic and electromyographic signals from nine men with PD and nine aged-matched healthy men while they sat and cyclically dorsi- and plantarflexed their feet. This movement was performed at a similar cadence to normal walking under two conditions: i) self-paced and ii) externally paced by a metronome. Participants with PD exhibited reduced range of motion and more variable bilateral coordination. However, participants with and without PD did not differ in the magnitude of corticomuscular coherence between the midline cortical areas and tibialis anterior and medial gastrocnemius muscles. This finding suggests that either the kinematic abnormalities were related to processes outside linear corticomuscular communication or PD-related changes in neural correlates maintained corticomuscular communication but not motor performance.

Basal ganglia mechanisms in action selection, plasticity, and dystonia

Basal ganglia circuits are organized to selected desired actions and to inhibit potentially competing unwanted actions. This is accomplished through a complex circuitry that is modified through development and learning. Mechanisms of neural plasticity underlying these modifications are increasingly understood, but new mechanisms continue to be discovered. Dystonia, a movement disorder characterized by involuntary muscle contractions that cause abnormal postures and movements. Emerging evidence points to important links between mechanisms of plasticity and the manifestations of dystonia. Investigation of these mechanisms has improved understanding of the action of currently used medication and is informing the development of new treatments.

Basal Ganglia Neuromodulation Over Multiple Temporal and Structural Scales-Simulations of Direct Pathway MSNs Investigate the Fast Onset of Dopaminergic Effects and Predict the Role of Kv4.2

The basal ganglia are involved in the motivational and habitual control of motor and cognitive behaviors. Striatum, the largest basal ganglia input stage, integrates cortical and thalamic inputs in functionally segregated cortico-basal ganglia-thalamic loops, and in addition the basal ganglia output nuclei control targets in the brainstem. Striatal function depends on the balance between the direct pathway medium spiny neurons (D1-MSNs) that express D1 dopamine receptors and the indirect pathway MSNs that express D2 dopamine receptors. The striatal microstructure is also divided into striosomes and matrix compartments, based on the differential expression of several proteins. Dopaminergic afferents from the midbrain and local cholinergic interneurons play crucial roles for basal ganglia function, and striatal signaling via the striosomes in turn regulates the midbrain dopaminergic system directly and via the lateral habenula. Consequently, abnormal functions of the basal ganglia neuromodulatory system underlie many neurological and psychiatric disorders. Neuromodulation acts on multiple structural levels, ranging from the subcellular level to behavior, both in health and disease. For example, neuromodulation affects membrane excitability and controls synaptic plasticity and thus learning in the basal ganglia. However, it is not clear on what time scales these different effects are implemented. Phosphorylation of ion channels and the resulting membrane effects are typically studied over minutes while it has been shown that neuromodulation can affect behavior within a few hundred milliseconds. So how do these seemingly contradictory effects fit together? Here we first briefly review neuromodulation of the basal ganglia, with a focus on dopamine. We furthermore use biophysically detailed multi-compartmental models to integrate experimental data regarding dopaminergic effects on individual membrane conductances with the aim to explain the resulting cellular level dopaminergic effects. In particular we predict dopaminergic effects on Kv4.2 in D1-MSNs. Finally, we also explore dynamical aspects of the onset of neuromodulation effects in multi-scale computational models combining biochemical signaling cascades and multi-compartmental neuron models.

Action-semantic and syntactic deficits in subjects at risk for Huntington's disease

Frontostriatal networks play critical roles in grounding action semantics and syntactic skills. Indeed, their atrophy distinctively disrupts both domains, as observed in patients with Huntington's disease (HD) and Parkinson's disease, even during early disease stages. However, frontostriatal degeneration in these conditions may begin up to 15 years before the onset of clinical symptoms, opening avenues for pre-clinical detection via sensitive tasks. Such a mission is particularly critical in HD, given that patients' children have 50% chances of inheriting the disease. Against this background, we assessed whether deficits in the above-mentioned domains emerge in subjects at risk to develop HD. We administered tasks tapping action semantics, object semantics, and two forms of syntactic processing to 18 patients with HD, 19 asymptomatic first-degree relatives, and sociodemographically matched controls for each group. The patients evinced significant deficits in all tasks, but only those in the two target domains were independent of overall cognitive state. More crucially, relative to controls, the asymptomatic relatives were selectively impaired in action semantics and in the more complex syntactic task, with both patterns emerging irrespective of the subjects' overall cognitive state. Our findings highlight the relevance of these dysfunctions as potential prodromal biomarkers of HD. Moreover, they offer theoretical insights into the differential contributions of frontostriatal hubs to both domains while paving the way for innovations in diagnostic procedures.

Correlations between physical activity and neurocognitive domain functions in patients with schizophrenia: a cross-sectional study

BACKGROUND

Neurocognitive dysfunction is a critical target symptom of schizophrenia treatment. A positive correlation between physical activity level and neurocognitive function has been reported in healthy individuals, but it is unclear whether such a correlation exists in patients with schizophrenia and whether the relationship is different according to inpatients or outpatients. This study aimed to examine the differences in the correlations between physical activity and multiple neurocognitive domains in inpatients and outpatients with schizophrenia and obtain suggestions for further study to facilitate this field.

METHODS

Twenty-nine patients with schizophrenia were examined (16 inpatients and 13 outpatients, 56.0 ± 11.4 years of age). Current symptoms were assessed using the Positive and Negative Symptom Scale and neurocognitive functions using Cognitrax, which yields a composite neurocognitive index (NCI) and 11 domain scores. After testing, participants wore an HJA-750C accelerometer for one week to measure physical activity levels and durations. Partial correlation analyses were performed between exercise and cognitive parameters.

RESULTS

In the outpatient group, higher physical activity was associated with faster Motor and Psychomotor Speeds in outpatients. However, higher physical activity was associated with lower overall NCI, Attention score, and Memory scores in inpatients.

CONCLUSION

Although higher physical activity was associated with better neurocognitive functions of outpatients, in inpatients with non-remitted schizophrenia, higher physical activity was associated with worsening of several cognitive domains. In a future study examining the relationship between physical activity and neurocognitive function for facilitating this research field, separation between inpatients and outpatients are needed because the relationship is different between inpatients and outpatients.

Chemical anatomy of pallidal afferents in primates

Neurons of the globus pallidus receive massive inputs from the striatum and the subthalamic nucleus, but their activity, as well as those of their striatal and subthalamic inputs, are modulated by brainstem afferents. These include serotonin (5-HT) projections from the dorsal raphe nucleus, cholinergic (ACh) inputs from the pedunculopontine tegmental nucleus, and dopamine (DA) afferents from the substantia nigra pars compacta. This review summarizes our recent findings on the distribution, quantitative and ultrastructural aspects of pallidal 5-HT, ACh and DA innervations. These results have led to the elaboration of a new model of the pallidal neuron based on a precise knowledge of the hierarchy and chemical features of the various synaptic inputs. The dense 5-HT, ACh and DA innervations disclosed in the associative and limbic pallidal territories suggest that these brainstem inputs contribute principally to the planification of motor behaviors and the regulation of attention and mood. Although 5-HT, ACh and DA inputs were found to modulate pallidal neurons and their afferents mainly through asynaptic (volume) transmission, genuine synaptic contacts occur between these chemospecific axon varicosities and pallidal dendrites, revealing that these brainstem projections have a direct access to pallidal neurons, in addition to their indirect input through the striatum and subthalamic nucleus. Altogether, these findings reveal that the brainstem 5-HT, ACh and DA pallidal afferents act in concert with the more robust GABAergic inhibitory striatopallidal and glutamatergic excitatory subthalamopallidal inputs. We hypothesize that a fragile equilibrium between forebrain and brainstem pallidal afferents plays a key role in the functional organization of the primate basal ganglia, in both health and disease.

Drugs of abuse and Parkinson's disease

The term "drug of abuse" is highly contextual. What constitutes a drug of abuse for one population of patients does not for another. It is therefore important to examine the needs of the patient population to properly assess the status of drugs of abuse. The focus of this article is on the bidirectional relationship between patients and drug abuse. In this paper we will introduce the dopaminergic systems of the brain in Parkinson's and the influence of antiparkinsonian drugs upon them before discussing this synergy of condition and medication as fertile ground for drug abuse. We will then examine the relationship between drugs of abuse and Parkinson's, both beneficial and deleterious. In summary we will draw the different strands together and speculate on the future merit of current drugs of abuse as treatments for Parkinson's disease.

Online feedback enhances early consolidation of motor sequence learning and reverses recall deficit from transcranial stimulation of motor cortex

Feedback and monetary reward can enhance motor skill learning, suggesting reward system involvement. Continuous theta burst (cTBS) transcranial magnetic stimulation (TMS) of the primary motor area (M1) disrupts processing, reduces excitability and impairs motor learning. To see whether feedback and reward can overcome the learning impairment associated with M1 cTBS, we delivered real or sham stimulation to two groups of participants before they performed a motor sequence learning task with and without feedback. Participants were trained on two intermixed sequences, one occurring 85% of the time (the "probable" sequence) and the other 15% of the time (the "improbable" sequence). We measured sequence learning as the difference in reaction time (RT) and error rate between probable and improbable trials (RT and error difference scores). Participants were also tested for sequence recall with the same indices of learning 60 min after cTBS. Real stimulation impaired initial sequence learning and sequence knowledge recall as measured by error difference scores and impaired sequence knowledge recall as measured by RT difference score. Relative to non-feedback learning, the introduction of feedback during sequence learning improved subsequent sequence knowledge recall indexed by RT difference score, in both real and sham stimulation groups and feedback reversed the RT difference score based sequence knowledge recall impairment from real cTBS that we observed in the non-feedback learning condition. Only the real cTBS group in the non-feedback condition showed no evidence of explicit sequence knowledge when tested at the end of the study. Feedback improves recall of implicit and explicit motor sequence knowledge and can protect sequence knowledge against the effect of M1 inhibition. Adding feedback and monetary reward/punishment to motor skill learning may help overcome retention impairments or accelerate training in clinical and other settings.

Morphological evidence for dopamine interactions with pallidal neurons in primates

The external (GPe) and internal (GPi) segments of the primate globus pallidus receive dopamine (DA) axonal projections arising mainly from the substantia nigra pars compacta and this innervation is here described based on tyrosine hydroxylase (TH) immunohistochemical observations gathered in the squirrel monkey (Saimiri sciureus). At the light microscopic level, unbiased stereological quantification of TH positive (+) axon varicosities reveals a similar density of innervation in the GPe (0.19 ± 0.02 × 10⁶ axon varicosities/mm³ of tissue) and GPi (0.17 ± 0.01 × 10⁶), but regional variations occur in the anteroposterior and dorsoventral axes in both GPe and GPi and along the mediolateral plane in the GPe. Estimation of the neuronal population in the GPe (3.47 ± 0.15 × 10³ neurons/mm³) and GPi (2.69 ± 0.18 × 10³) yields a mean ratio of, respectively, 28 ± 3 and 68 ± 15 TH+ axon varicosities/pallidal neuron. At the electron microscopic level, TH+ axon varicosities in the GPe appear significantly smaller than those in the GPi and very few TH+ axon varicosities are engaged in synaptic contacts in the GPe (17 ± 3%) and the GPi (15 ± 4%) compared to their unlabeled counterparts (77 ± 6 and 50 ± 12%, respectively). Genuine synaptic contacts made by TH+ axon varicosities in the GPe and GPi are of the symmetrical and asymmetrical type. Such synaptic contacts together with the presence of numerous synaptic vesicles in all TH+ axon varicosities observed in the GPe and GPi support the functionality of the DA pallidal innervation. By virtue of its predominantly volumic mode of action, DA appears to exert a key modulatory effect upon pallidal neurons in concert with the more direct GABAergic inhibitory and glutamatergic excitatory actions of the striatum and subthalamic nucleus. We argue that the DA pallidal innervation plays a major role in the functional organization of the primate basal ganglia under both normal and pathological conditions.

Habits as action sequences: hierarchical action control and changes in outcome value

Goal-directed action involves making high-level choices that are implemented using previously acquired action sequences to attain desired goals. Such a hierarchical schema is necessary for goal-directed actions to be scalable to real-life situations, but results in decision-making that is less flexible than when action sequences are unfolded and the decision-maker deliberates step-by-step over the outcome of each individual action. In particular, from this perspective, the offline revaluation of any outcomes that fall within action sequence boundaries will be invisible to the high-level planner resulting in decisions that are insensitive to such changes. Here, within the context of a two-stage decision-making task, we demonstrate that this property can explain the emergence of habits. Next, we show how this hierarchical account explains the insensitivity of over-trained actions to changes in outcome value. Finally, we provide new data that show that, under extended extinction conditions, habitual behaviour can revert to goal-directed control, presumably as a consequence of decomposing action sequences into single actions. This hierarchical view suggests that the development of action sequences and the insensitivity of actions to changes in outcome value are essentially two sides of the same coin, explaining why these two aspects of automatic behaviour involve a shared neural structure.

Dopamine and the development of executive dysfunction in autism spectrum disorders

Persons with autism regularly exhibit executive dysfunction (ED), including problems with deliberate goal-directed behavior, planning, and flexible responding in changing environments. Indeed, this array of deficits is sufficiently prominent to have prompted a theory that executive dysfunction is at the heart of these disorders. A more detailed examination of these behaviors reveals, however, that some aspects of executive function remain developmentaly appropriate. In particular, while people with autism often have difficulty with tasks requiring cognitive flexibility, their fundamental cognitive control capabilities, such as those involved in inhibiting an inappropriate but relatively automatic response, show no significant impairment on many tasks. In this article, an existing computational model of the prefrontal cortex and its role in executive control is shown to explain this dichotomous pattern of behavior by positing abnormalities in the dopamine-based modulation of frontal systems in individuals with autism. This model offers excellent qualitative and quantitative fits to performance on standard tests of cognitive control and cognitive flexibility in this clinical population. By simulating the development of the prefrontal cortex, the computational model also offers a potential explanation for an observed lack of executive dysfunction early in life.

Cognitive correlates of psychosis in patients with Parkinson's disease

INTRODUCTION

Psychosis and hallucinations occur in 20-30% of patients with Parkinson's disease (PD). In the current study, we investigate cognitive functions in relation to the occurrence of psychosis in PD patients.

METHODS

We tested three groups of subjects - PD with psychosis, PD without psychosis and healthy controls - on working memory, learning and transitive inference tasks, which are known to assess prefrontal, basal ganglia and hippocampal functions.

RESULTS

In the working memory task, results show that patients with and without psychosis were more impaired than the healthy control group. In the transitive inference task, we did not find any difference among the groups in the learning phase performance. Importantly, PD patients with psychosis were more impaired than both PD patients without psychosis and controls at transitive inference. We also found that the severity of psychotic symptoms in PD patients [as measured by the Unified Parkinson Disease Rating Scale Thought Disorder (UPDRS TD) item] is directly associated with the severity of cognitive impairment [as measured by the mini-mental status exam (MMSE)], sleep disturbance [as measured by the Scales for Outcome in Parkinson Disease (SCOPA) sleep scale] and transitive inference (although the latter did not reach significance).

CONCLUSIONS

Although hypothetical, our data may suggest that the hippocampus is a neural substrate underlying the occurrence of psychosis, sleep disturbance and cognitive impairment in PD patients.

The role of prediction and outcomes in adaptive cognitive control

Humans adaptively perform actions to achieve their goals. This flexible behaviour requires two core abilities: the ability to anticipate the outcomes of candidate actions and the ability to select and implement actions in a goal-directed manner. The ability to predict outcomes has been extensively researched in reinforcement learning paradigms, but this work has often focused on simple actions that are not embedded in hierarchical and sequential structures that are characteristic of goal-directed human behaviour. On the other hand, the ability to select actions in accordance with high-level task goals, particularly in the presence of alternative responses and salient distractors, has been widely researched in cognitive control paradigms. Cognitive control research, however, has often paid less attention to the role of action outcomes. The present review attempts to bridge these accounts by proposing an outcome-guided mechanism for selection of extended actions. Our proposal builds on constructs from the hierarchical reinforcement learning literature, which emphasises the concept of reaching and evaluating informative states, i.e., states that constitute subgoals in complex actions. We develop an account of the neural mechanisms that allow outcome-guided action selection to be achieved in a network that relies on projections from cortical areas to the basal ganglia and back-projections from the basal ganglia to the cortex. These cortico-basal ganglia-thalamo-cortical 'loops' allow convergence - and thus integration - of information from non-adjacent cortical areas (for example between sensory and motor representations). This integration is essential in action sequences, for which achieving an anticipated sensory state signals the successful completion of an action. We further describe how projection pathways within the basal ganglia allow selection between representations, which may pertain to movements, actions, or extended action plans. The model lastly envisages a role for hierarchical projections from the striatum to dopaminergic midbrain areas that enable more rostral frontal areas to bias the selection of inputs from more posterior frontal areas via their respective representations in the basal ganglia.

TJ, Lo SE, Ghosh PT, Howard Jr. JH, Howard DV. Implicit sequence learning in people with Parkinson's disease

Implicit sequence learning involves learning about dependencies in sequences of events without intent to learn or awareness of what has been learned. Sequence learning is related to striatal dopamine levels, striatal activation, and integrity of white matter connections. People with Parkinson's disease (PD) have degeneration of dopamine-producing neurons, leading to dopamine deficiency and therefore striatal deficits, and they have difficulties with sequencing, including complex language comprehension and postural stability. Most research on implicit sequence learning in PD has used motor-based tasks. However, because PD presents with motor deficits, it is difficult to assess whether learning itself is impaired in these tasks. The present study used an implicit sequence learning task with a reduced motor component, the Triplets Learning Task (TLT). People with PD and age- and education-matched healthy older adults completed three sessions (each consisting of 10 blocks of 50 trials) of the TLT. Results revealed that the PD group was able to learn the sequence, however, when learning was examined using a Half Blocks analysis (Nemeth et al., 2013), which compared learning in the 1st 25/50 trials of all blocks to that in the 2nd 25/50 trials, the PD group showed significantly less learning than Controls in the 2nd Half Blocks, but not in the 1st. Nemeth et al. (2013) hypothesized that the 1st Half Blocks involve recall and reactivation of the sequence learned, thus reflecting hippocampal-dependent learning, while the 2nd Half Blocks involve proceduralized behavior of learned sequences, reflecting striatal-based learning. The present results suggest that the PD group had intact hippocampal-dependent implicit sequence learning, but impaired striatal-dependent learning. Thus, sequencing deficits in PD are likely due to striatal impairments, but other brain systems, such as the hippocampus, may be able to partially compensate for striatal decline to improve performance.

Annual research review: The neurobehavioral development of multiple memory systems--implications for childhood and adolescent psychiatric disorders

Extensive evidence indicates that mammalian memory is organized into multiple brains systems, including a 'cognitive' memory system that depends on the hippocampus and a stimulus-response 'habit' memory system that depends on the dorsolateral striatum. Dorsal striatal-dependent habit memory may in part influence the development and expression of some human psychopathologies, particularly those characterized by strong habit-like behavioral features. The present review considers this hypothesis as it pertains to psychopathologies that typically emerge during childhood and adolescence. These disorders include Tourette syndrome, attention-deficit/hyperactivity disorder, obsessive-compulsive disorder, eating disorders, and autism spectrum disorders. Human and nonhuman animal research shows that the typical development of memory systems comprises the early maturation of striatal-dependent habit memory and the relatively late maturation of hippocampal-dependent cognitive memory. We speculate that the differing rates of development of these memory systems may in part contribute to the early emergence of habit-like symptoms in childhood and adolescence. In addition, abnormalities in hippocampal and striatal brain regions have been observed consistently in youth with these disorders, suggesting that the aberrant development of memory systems may also contribute to the emergence of habit-like symptoms as core pathological features of these illnesses. Considering these disorders within the context of multiple memory systems may help elucidate the pathogenesis of habit-like symptoms in childhood and adolescence, and lead to novel treatments that lessen the habit-like behavioral features of these disorders.

Investigating habits: strategies, technologies and models

Understanding habits at a biological level requires a combination of behavioral observations and measures of ongoing neural activity. Theoretical frameworks as well as definitions of habitual behaviors emerging from classic behavioral research have been enriched by new approaches taking account of the identification of brain regions and circuits related to habitual behavior. Together, this combination of experimental and theoretical work has provided key insights into how brain circuits underlying action-learning and action-selection are organized, and how a balance between behavioral flexibility and fixity is achieved. New methods to monitor and manipulate neural activity in real time are allowing us to have a first look “under the hood” of a habit as it is formed and expressed. Here we discuss ideas emerging from such approaches. We pay special attention to the unexpected findings that have arisen from our own experiments suggesting that habitual behaviors likely require the simultaneous activity of multiple distinct components, or operators, seen as responsible for the contrasting dynamics of neural activity in both cortico-limbic and sensorimotor circuits recorded concurrently during different stages of habit learning. The neural dynamics identified thus far do not fully meet expectations derived from traditional models of the structure of habits, and the behavioral measures of habits that we have made also are not fully aligned with these models. We explore these new clues as opportunities to refine an understanding of habits.

Mouse models of neurodevelopmental disease of the basal ganglia and associated circuits

This chapter focuses on neurodevelopmental diseases that are tightly linked to abnormal function of the striatum and connected structures. We begin with an overview of three representative diseases in which striatal dysfunction plays a key role--Tourette syndrome and obsessive-compulsive disorder, Rett's syndrome, and primary dystonia. These diseases highlight distinct etiologies that disrupt striatal integrity and function during development, and showcase the varied clinical manifestations of striatal dysfunction. We then review striatal organization and function, including evidence for striatal roles in online motor control/action selection, reinforcement learning, habit formation, and action sequencing. A key barrier to progress has been the relative lack of animal models of these diseases, though recently there has been considerable progress. We review these efforts, including their relative merits providing insight into disease pathogenesis, disease symptomatology, and basal ganglia function.

Complex motor stereotypies: an evolving neurobiological concept

Complex motor stereotypies are repetitive arm and/or hand flapping, waving and wiggling movements that begin before the age of 3 years, occur repeatedly throughout the day and stop with distraction. These movements are commonly seen in children with autism, but also appear in otherwise normally developing individuals labelled as primary. Although proposed to have a psychological and neurobiological mechanism, evidence suggests that there is an abnormality within the corticostriatal–thalamocortical circuitry or its connecting structures. Animal models include both drug-induced (i.e., via stimulants or cocaine) and spontaneously appearing prototypes. Neurochemical investigations, primarily in rodents, have identified a variety of neurotransmitter alterations, with an emphasis on dopamine or glutamate; however, findings are inconsistent. We hypothesize that, based on its various roles in controlling and modulating movements, the frontal cortex will ultimately be shown to be the prime site of abnormality in this disorder. Future studies investigating both humans and animal models are essential for attaining a greater understanding of the pathobiology underlying motor stereotypies.

Genetic variation in the human brain dopamine system influences motor learning and its modulation by L-Dopa

Dopamine is important to learning and plasticity. Dopaminergic drugs are the focus of many therapies targeting the motor system, where high inter-individual differences in response are common. The current study examined the hypothesis that genetic variation in the dopamine system is associated with significant differences in motor learning, brain plasticity, and the effects of the dopamine precursor L-Dopa. Skilled motor learning and motor cortex plasticity were assessed using a randomized, double-blind, placebo-controlled, crossover design in 50 healthy adults during two study weeks, one with placebo and one with L-Dopa. The influence of five polymorphisms with established effects on dopamine neurotransmission was summed using a gene score, with higher scores corresponding to higher dopaminergic neurotransmission. Secondary hypotheses examined each polymorphism individually. While training on placebo, higher gene scores were associated with greater motor learning (p = .03). The effect of L-Dopa on learning varied with the gene score (gene score*drug interaction, p = .008): participants with lower gene scores, and thus lower endogenous dopaminergic neurotransmission, showed the largest learning improvement with L-Dopa relative to placebo (p<.0001), while L-Dopa had a detrimental effect in participants with higher gene scores (p = .01). Motor cortex plasticity, assessed via transcranial magnetic stimulation (TMS), also showed a gene score*drug interaction (p = .02). Individually, DRD2/ANKK1 genotype was significantly associated with motor learning (p = .02) and its modulation by L-Dopa (p<.0001), but not with any TMS measures. However, none of the individual polymorphisms explained the full constellation of findings associated with the gene score. These results suggest that genetic variation in the dopamine system influences learning and its modulation by L-Dopa. A polygene score explains differences in L-Dopa effects on learning and plasticity most robustly, thus identifying distinct biological phenotypes with respect to L-Dopa effects on learning and plasticity. These findings may have clinical applications in post-stroke rehabilitation or the treatment of Parkinson's disease.

Habits, action sequences and reinforcement learning

It is now widely accepted that instrumental actions can be either goal-directed or habitual; whereas the former are rapidly acquired and regulated by their outcome, the latter are reflexive, elicited by antecedent stimuli rather than their consequences. Model-based reinforcement learning (RL) provides an elegant description of goal-directed action. Through exposure to states, actions and rewards, the agent rapidly constructs a model of the world and can choose an appropriate action based on quite abstract changes in environmental and evaluative demands. This model is powerful but has a problem explaining the development of habitual actions. To account for habits, theorists have argued that another action controller is required, called model-free RL, that does not form a model of the world but rather caches action values within states allowing a state to select an action based on its reward history rather than its consequences. Nevertheless, there are persistent problems with important predictions from the model; most notably the failure of model-free RL correctly to predict the insensitivity of habitual actions to changes in the action-reward contingency. Here, we suggest that introducing model-free RL in instrumental conditioning is unnecessary, and demonstrate that reconceptualizing habits as action sequences allows model-based RL to be applied to both goal-directed and habitual actions in a manner consistent with what real animals do. This approach has significant implications for the way habits are currently investigated and generates new experimental predictions.

The mysterious motivational functions of mesolimbic dopamine

Nucleus accumbens dopamine is known to play a role in motivational processes, and dysfunctions of mesolimbic dopamine may contribute to motivational symptoms of depression and other disorders, as well as features of substance abuse. Although it has become traditional to label dopamine neurons as "reward" neurons, this is an overgeneralization, and it is important to distinguish between aspects of motivation that are differentially affected by dopaminergic manipulations. For example, accumbens dopamine does not mediate primary food motivation or appetite, but is involved in appetitive and aversive motivational processes including behavioral activation, exertion of effort, approach behavior, sustained task engagement, Pavlovian processes, and instrumental learning. In this review, we discuss the complex roles of dopamine in behavioral functions related to motivation.

Response inhibition signals and miscoding of direction in dorsomedial striatum

The ability to inhibit action is critical for everyday behavior and is affected by a variety of disorders. Behavioral control and response inhibition is thought to depend on a neural circuit that includes the dorsal striatum, yet the neural signals that lead to response inhibition and its failure are unclear. To address this issue, we recorded from neurons in rat dorsomedial striatum (mDS) in a novel task in which rats responded to a spatial cue that signaled that reward would be delivered either to the left or to the right. On 80% of trials rats were instructed to respond in the direction cued by the light (GO). On 20% of trials a second light illuminated instructing the rat to refrain from making the cued movement and move in the opposite direction (STOP). Many neurons in mDS encoded direction, firing more or less strongly for GO movements made ipsilateral or contralateral to the recording electrode. Neurons that fired more strongly for contralateral GO responses were more active when rats were faster, showed reduced activity on STOP trials, and miscoded direction on errors, suggesting that when these neurons were overly active, response inhibition failed. Neurons that decreased firing for contralateral movement were excited during trials in which the rat was required to stop the ipsilateral movement. For these neurons activity was reduced when errors were made and was negatively correlated with movement time suggesting that when these neurons were less active on STOP trials, response inhibition failed. Finally, the activity of a significant number of neurons represented a global inhibitory signal, firing more strongly during response inhibition regardless of response direction. Breakdown by cell type suggests that putative medium spiny neurons (MSNs) tended to fire more strongly under STOP trials, whereas putative interneurons exhibited both activity patterns.

Action selection and action value in frontal-striatal circuits

The role that frontal-striatal circuits play in normal behavior remains unclear. Two of the leading hypotheses suggest that these circuits are important for action selection or reinforcement learning. To examine these hypotheses, we carried out an experiment in which monkeys had to select actions in two different task conditions. In the first (random) condition, actions were selected on the basis of perceptual inference. In the second (fixed) condition, the animals used reinforcement from previous trials to select actions. Examination of neural activity showed that the representation of the selected action was stronger in lateral prefrontal cortex (lPFC), and occurred earlier in the lPFC than it did in the dorsal striatum (dSTR). In contrast to this, the representation of action values, in both the random and fixed conditions, was stronger in the dSTR. Thus, the dSTR contains an enriched representation of action value, but it followed frontal cortex in action selection.

A hypothesis for basal ganglia-dependent reinforcement learning in the songbird

Most of our motor skills are not innately programmed, but are learned by a combination of motor exploration and performance evaluation, suggesting that they proceed through a reinforcement learning (RL) mechanism. Songbirds have emerged as a model system to study how a complex behavioral sequence can be learned through an RL-like strategy. Interestingly, like motor sequence learning in mammals, song learning in birds requires a basal ganglia (BG)-thalamocortical loop, suggesting common neural mechanisms. Here, we outline a specific working hypothesis for how BG-forebrain circuits could utilize an internally computed reinforcement signal to direct song learning. Our model includes a number of general concepts borrowed from the mammalian BG literature, including a dopaminergic reward prediction error and dopamine-mediated plasticity at corticostriatal synapses. We also invoke a number of conceptual advances arising from recent observations in the songbird. Specifically, there is evidence for a specialized cortical circuit that adds trial-to-trial variability to stereotyped cortical motor programs, and a role for the BG in "biasing" this variability to improve behavioral performance. This BG-dependent "premotor bias" may in turn guide plasticity in downstream cortical synapses to consolidate recently learned song changes. Given the similarity between mammalian and songbird BG-thalamocortical circuits, our model for the role of the BG in this process may have broader relevance to mammalian BG function.

Parallel contributions of cerebellar, striatal and M1 mechanisms to motor sequence learning

When learning a new motor sequence, we must execute the correct order of movements while simultaneously optimizing sensorimotor parameters such as trajectory, timing, velocity and force. Neurophysiological studies in animals and humans have identified the major brain regions involved in sequence learning, including the motor cortex (M1), basal ganglia (BG) and cerebellum. Current models link these regions to different stages of learning (early vs. late) or different components of performance (spatial vs. sensorimotor). At the same time, research in motor control has given rise to the concept that internal models at different levels of the motor system may contribute to learning. The goal of this review is to develop a new framework for motor sequence learning that combines stage and component models within the context of internal models. To do this, we review behavioral and neuroimaging studies in humans and neurophysiological studies in animals. Based on this evidence, we present a model proposing that sequence learning is underwritten by parallel, interacting processes, including internal model formation and sequence representation, that are instantiated in specific cerebellar, BG or M1 mechanisms depending on task demands and the stage of learning. The striatal system learns predictive stimulus-response associations and is critical for motor chunking. The role of the cerebellum is to acquire the optimal internal model for sequence performance in a particular context, and to contribute to error correction and control of on-going movement. M1 acts to store the representation of a learned sequence, likely as part of a distributed network including the parietal lobe and premotor cortex.

Neural substrates for the motivational regulation of motor recovery after spinal-cord injury

It is believed that depression impedes and motivation enhances functional recovery after neuronal damage such as spinal-cord injury and stroke. However, the neuronal substrate underlying such psychological effects on functional recovery remains unclear. A longitudinal study of brain activation in the non-human primate model of partial spinal-cord injury using positron emission tomography (PET) revealed a contribution of the primary motor cortex (M1) to the recovery of finger dexterity through the rehabilitative training. Here, we show that activity of the ventral striatum, including the nucleus accumbens (NAc), which plays a critical role in processing of motivation, increased and its functional connectivity with M1 emerged and was progressively strengthened during the recovery. In addition, functional connectivities among M1, the ventral striatum and other structures belonging to neural circuits for processing motivation, such as the orbitofrontal cortex, anterior cingulate cortex and pedunculopontine tegmental nucleus were also strengthened during the recovery. These results give clues to the neuronal substrate for motivational regulation of motor learning required for functional recovery after spinal-cord injury.

Importance of the temporal structure of movement sequences on the ability of monkeys to use serial order information

The capacity to acquire motor skills through repeated practice of a sequence of movements underlies many everyday activities. Extensive research in humans has dealt with the importance of spatial and temporal factors on motor sequence learning, standing in contrast to the few studies available in animals, particularly in nonhuman primates. In the present experiments, we studied the effect of the serial order of stimuli and associated movements in macaque monkeys overtrained to make arm-reaching movements in response to spatially distinct visual targets. Under different conditions, the temporal structure of the motor sequence was varied by changing the duration of the interval between successive target stimuli or by adding a cue that reliably signaled the onset time of the forthcoming target stimulus. In each condition, the extent to which the monkeys are sensitive to the spatial regularities was assessed by comparing performance when stimulus locations follow a repeating sequence, as opposed to a random sequence. We observed no improvement in task performance on repeated sequence blocks, compared to random sequence blocks, when target stimuli are relatively distant from each other in time. On the other hand, the shortening of the time interval between successive target stimuli or, more efficiently, the addition of a temporal cue before the target stimulus yielded a performance advantage under repeated sequence, reflected in a decrease in the latency of arm and saccadic eye movements accompanied by an increased tendency for eye movements to occur in an anticipatory manner. Contrary to the effects on movement initiation, the serial order of stimuli and movements did not markedly affect the execution of movement. Moreover, the location of a given target in the random sequence influenced task performance based on the location of the preceding target, monkeys being faster in responding as a result of familiarity caused by extensive practice with some target transitions also used in the repeated sequence. This performance advantage was most prominently detectable when temporal prediction of forthcoming target stimuli was optimized. Taken together, the present findings demonstrate that the monkey's capacity to make use of serial order information to speed task performance was dependent on the temporal structure of the motor sequence.

Age-related striatal dopaminergic denervation and severity of a slip perturbation

BACKGROUND

Striatal dopamine activity declines with normal aging. Age-related striatal dopaminergic denervation (SDD) has been implicated in standing balance and unperturbed gait. The goal of this study was to analyze the association between the degree of SDD and the magnitude of an unexpected slip perturbation induced during gait.

METHODS

Fifty healthy participants aged 20-86 years old underwent dopamine transporter positron emission tomography to classify SDD severity as mild, moderate, or severe. Participants also walked on a floor that was unexpectedly contaminated with a glycerol solution for gait testing. The magnitude of a slip was quantified using the peak slip velocity (PSV), measured at the slipping foot. Data were analyzed for both fast (greater than 1.2 m/s) and slow walkers as gait speed correlated with slip severity. All data analyses were age adjusted.

RESULTS

Greater severity of dopaminergic denervation in the caudate nucleus was correlated with higher PSV (p < .01) but only in the fast speed walking group. The relationship between SDD in the putamen and slip severity was not statistically significant in fast and slow walkers.

CONCLUSIONS

Age-related SDD may impact the ability to recover from large perturbations during walking in individuals who typically walk fast. This effect, prominent in the caudate nucleus, may implicate a role of cognitive frontostriatal pathways in the executive control of gait when balance is challenged by large perturbations. Finally, a cautious gait behavior present in slow walkers may explain the apparent lack of involvement of striatal dopaminergic pathways in postural responses to slips.