Building action repertoires: memory and learning functions of the basal ganglia

Face-name association learning and brain structural substrates in alcoholism

BACKGROUND

Associative learning is required for face-name association and is impaired in alcoholism, but the cognitive processes and brain structural components underlying this deficit remain unclear. It is also unknown whether prompting alcoholics to implement a deep level of processing during face-name encoding would enhance performance.

METHODS

Abstinent alcoholics and controls performed a levels-of-processing face-name learning task. Participants indicated whether the face was that of an honest person (deep encoding) or that of a man (shallow encoding). Retrieval was examined using an associative (face-name) recognition task and a single-item (face or name only) recognition task. Participants also underwent 3T structural MRI.

RESULTS

Compared with controls, alcoholics had poorer associative and single-item learning and performed at similar levels. Level of processing at encoding had little effect on recognition performance but affected reaction time (RT). Correlations with brain volumes were generally modest and based primarily on RT in alcoholics, where the deeper the processing at encoding, the more restricted the correlations with brain volumes. In alcoholics, longer control task RTs correlated modestly with smaller tissue volumes across several anterior to posterior brain regions; shallow encoding correlated with calcarine and striatal volumes; deep encoding correlated with precuneus and parietal volumes; and associative recognition RT correlated with cerebellar volumes. In controls, poorer associative recognition with deep encoding correlated significantly with smaller volumes of frontal and striatal structures.

CONCLUSIONS

Despite prompting, alcoholics did not take advantage of encoding memoranda at a deep level to enhance face-name recognition accuracy. Nonetheless, conditions of deeper encoding resulted in faster RTs and more specific relations with regional brain volumes than did shallow encoding. The normal relation between associative recognition and corticostriatal volumes was not present in alcoholics. Rather, their speeded RTs occurred at the expense of accuracy and were related most robustly to cerebellar volumes.

Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis

Growing evidence suggests that the prefrontal cortex (PFC) is organized hierarchically, with more anterior regions having increasingly abstract representations. How does this organization support hierarchical cognitive control and the rapid discovery of abstract action rules? We present computational models at different levels of description. A neural circuit model simulates interacting corticostriatal circuits organized hierarchically. In each circuit, the basal ganglia gate frontal actions, with some striatal units gating the inputs to PFC and others gating the outputs to influence response selection. Learning at all of these levels is accomplished via dopaminergic reward prediction error signals in each corticostriatal circuit. This functionality allows the system to exhibit conditional if-then hypothesis testing and to learn rapidly in environments with hierarchical structure. We also develop a hybrid Bayesian-reinforcement learning mixture of experts (MoE) model, which can estimate the most likely hypothesis state of individual participants based on their observed sequence of choices and rewards. This model yields accurate probabilistic estimates about which hypotheses are attended by manipulating attentional states in the generative neural model and recovering them with the MoE model. This 2-pronged modeling approach leads to multiple quantitative predictions that are tested with functional magnetic resonance imaging in the companion paper.

A modular neural model of motor synergies

Animals such as reptiles, amphibians and mammals (including humans) are mechanically extremely complex. It has been estimated that the human body has between 500 and 1400 degrees of freedom! And yet, these animals can generate an infinite variety of very precise, complicated and goal-directed movements in continuously changing and uncertain environments. Understanding how this is achieved is of great interest to both biologists and engineers. There are essentially two questions that must be addressed: (1) What type of control strategy is used to handle the large number of degrees of freedom involved? and (2) How is this strategy instantiated in the substrate of neural and musculoskeletal elements comprising the animal bodies? The first question has been studied intensively for several decades, providing strong indications that, rather than using standard feedback control based on continuous tracking of desired trajectories, animals' movements emerge from the controlled combination of pre-configured movement primitives or synergies. These synergies represent coordinated activity patterns over groups of muscles, and can be triggered as a whole with controlled amplitude and temporal offset. Complex movements can thus be constructed from the appropriate combination of a relatively small number of synergies, greatly simplifying the control problem. Although experimental studies on animal movements have confirmed the existence of motor synergies, and their utility has been demonstrated in the control of fairly complex robots, their neural basis remains poorly understood. In this paper, we introduce a simple but plausible and general neural model for motor synergies based on the principle that these functional modules reflect the structural modularity of the underlying physical system. Using this model, we show that a small set of synergies selected through a redundancy-reduction principle can generate a rich motor repertoire in a model two-jointed arm system. We investigate the synergies generated by this model systematically with respect to various parameters, and compare them to those observed in experiments.

Reduced caudate volume and enhanced striatal-DMN integration in chess experts

The superior capability of chess experts largely depends on quick automatic processing skills which are considered to be mediated by the caudate nucleus. We asked whether continued practice or rehearsal of the skill over a long period of time can lead to structural changes in this region. We found that, comparing to novice controls, grandmaster and master level Chinese chess players (GM/Ms), who had a mean period of over 10years of tournament and training practice, exhibited significant smaller gray-matter volume in the bilateral caudate nuclei. When these regions were used as seeds in functional connectivity analysis in resting-state fMRI, significantly enhanced integration was found in GM/Ms between the caudate and the default mode network (DMN), a constellation of brain areas important for goal-directed cognitive performance and theory of mind. These findings demonstrate the structural changes in the caudate nucleus in response to its extensive engagement in chess problem solving, and its enhanced functional integration with widely distributed circuitry to better support high-level cognitive control of behavior.

Dopaminergic modulation of the striatal microcircuit: receptor-specific configuration of cell assemblies

Selection and inhibition of motor behaviors are related to the coordinated activity and compositional capabilities of striatal cell assemblies. Striatal network activity represents a main step in basal ganglia processing. The dopaminergic system differentially regulates distinct populations of striatal medium spiny neurons (MSNs) through the activation of D(1)- or D(2)-type receptors. Although postsynaptic and presynaptic actions of these receptors are clearly different in MSNs during cell-focused studies, their activation during network activity has shown inconsistent responses. Therefore, using electrophysiological techniques, functional multicell calcium imaging, and neuronal population analysis in rat corticostriatal slices, we describe the effect of selective dopaminergic receptor activation in the striatal network by observing cell assembly configurations. At the microcircuit level, during striatal network activity, the selective activation of either D(1)- or D(2)-type receptors is reflected as overall increases in neuronal synchronization. However, graph theory techniques applied to the transitions between network states revealed receptor-specific configurations of striatal cell assemblies: D(1) receptor activation generated closed trajectories with high recurrence and few alternate routes favoring the selection of specific sequences, whereas D(2) receptor activation created trajectories with low recurrence and more alternate pathways while promoting diverse transitions among neuronal pools. At the single-cell level, the activation of dopaminergic receptors enhanced the negative-slope conductance region (NSCR) in D(1)-type-responsive cells, whereas in neurons expressing D(2)-type receptors, the NSCR was decreased. Consequently, receptor-specific network dynamics most probably result from the interplay of postsynaptic and presynaptic dopaminergic actions.

Where is my reward and how do I get it? Interaction between the hippocampus and the basal ganglia during spatial learning

Spatial learning has been recognized over the years to be under the control of the hippocampus and related temporal lobe structures. Hippocampal damage often causes severe impairments in the ability to learn and remember a location in space defined by distal visual cues. Recent experimental evidence in rodents demonstrates, however, that other brain areas might also be involved in the acquisition of spatial information. Amongst these, the cortex--basal ganglia loop is known to be involved in reinforcement learning and has been identified as an important contributor to spatial learning. In particular, it has been shown that altered activity of the basal ganglia striatal complex can impair the ability to perform spatial learning tasks. Until recently, little was known about how the basal ganglia and the hippocampus interact and how their activities evolve during learning. The present review, focusing on rodent studies, provides a glimpse of the findings obtained over the past decade that support a dialog between these two structures during spatial learning. Based on these studies, we propose a new functional spatial decision network with three separate loops encompassing hippocampus and specific basal ganglia regions. Each of the three loops serves a different aspect of spatial decision making and all three are linked by their mutual connections and are under the control of the dopaminergic learning signal.

Preservation of function in Parkinson's disease: what's learning got to do with it?

Dopamine denervation gives rise to abnormal corticostriatal plasticity; however, its role in the symptoms and progression of Parkinson's disease (PD) has not been articulated or incorporated into current clinical models. The 'integrative selective gain' framework proposed here integrates dopaminergic mechanisms known to modulate basal ganglia throughput into a single conceptual framework: (1) synaptic weights, the neural instantiation of accumulated experience and skill modulated by dopamine-dependent plasticity and (2) system gain, the operating parameters of the basal ganglia, modulated by dopamine's on-line effects on cell excitability, glutamatergic transmission and the balance between facilitatory and inhibitory pathways. Within this framework and based on recent work, a hypothesis is presented that prior synaptic weights and established skills can facilitate motor performance and preserve function despite diminished dopamine; however, dopamine denervation induces aberrant corticostriatal plasticity that degrades established synaptic weights and replaces them with inappropriate, inhibitory learning that inverts the function of the basal ganglia resulting in 'anti-optimization' of motor performance. Consequently, mitigating aberrant corticostriatal plasticity represents an important therapeutic objective, as reflected in the long-duration response to levodopa, reinterpreted here as the correction of aberrant learning. It is proposed that viewing aberrant corticostriatal plasticity and learning as a provisional endophenotype of PD would facilitate investigation of this hypothesis.

Remembering forward: neural correlates of memory and prediction in human motor adaptation

We used functional MR imaging (FMRI), a robotic manipulandum and systems identification techniques to examine neural correlates of predictive compensation for spring-like loads during goal-directed wrist movements in neurologically-intact humans. Although load changed unpredictably from one trial to the next, subjects nevertheless used sensorimotor memories from recent movements to predict and compensate upcoming loads. Prediction enabled subjects to adapt performance so that the task was accomplished with minimum effort. Population analyses of functional images revealed a distributed, bilateral network of cortical and subcortical activity supporting predictive load compensation during visual target capture. Cortical regions--including prefrontal, parietal and hippocampal cortices--exhibited trial-by-trial fluctuations in BOLD signal consistent with the storage and recall of sensorimotor memories or "states" important for spatial working memory. Bilateral activations in associative regions of the striatum demonstrated temporal correlation with the magnitude of kinematic performance error (a signal that could drive reward-optimizing reinforcement learning and the prospective scaling of previously learned motor programs). BOLD signal correlations with load prediction were observed in the cerebellar cortex and red nuclei (consistent with the idea that these structures generate adaptive fusimotor signals facilitating cancelation of expected proprioceptive feedback, as required for conditional feedback adjustments to ongoing motor commands and feedback error learning). Analysis of single subject images revealed that predictive activity was at least as likely to be observed in more than one of these neural systems as in just one. We conclude therefore that motor adaptation is mediated by predictive compensations supported by multiple, distributed, cortical and subcortical structures.

Spike frequency adaptation is developmentally regulated in substantia nigra pars compacta dopaminergic neurons

Dopaminergic neurons of the substantia nigra pars compacta play a key role in the modulation of basal ganglia and provide a reward-related teaching signal essential for adaptative motor control. They are generally considered as a homogenous population despite several chemical and electrophysiological heterogeneities, which could underlie different preferential patterns of activity and/or different roles. Using whole-cell patch-clamp recordings in juvenile rat brain slices, we observed that the evoked activity of dopaminergic neurons displays variable spike frequency adaptation patterns. The intensity of spike frequency adaptation decreased during post-natal development. The adaptation was associated with an increase in the initial firing frequency due to faster kinetics of the afterhyperpolarization component of the spike. Adaptation was enhanced when small conductance calcium-activated potassium (SK) channels were blocked with bath application of apamine. Lastly, spike frequency adaptation of the evoked discharge was associated with more irregularity in the spontaneous firing pattern. Altogether these results show a developmental heterogeneity and electrophysiological maturation of substantia nigra dopaminergic neurons.

Motor Skill Learning, Retention, and Control Deficits in Parkinson's Disease

Parkinson's disease, which affects the basal ganglia, is known to lead to various impairments of motor control. Since the basal ganglia have also been shown to be involved in learning processes, motor learning has frequently been investigated in this group of patients. However, results are still inconsistent, mainly due to skill levels and time scales of testing. To bridge across the time scale problem, the present study examined de novo skill learning over a long series of practice sessions that comprised early and late learning stages as well as retention. 19 non-demented, medicated, mild to moderate patients with Parkinson's disease and 19 healthy age and gender matched participants practiced a novel throwing task over five days in a virtual environment where timing of release was a critical element. Six patients and seven control participants came to an additional long-term retention testing after seven to nine months. Changes in task performance were analyzed by a method that differentiates between three components of motor learning prominent in different stages of learning: Tolerance, Noise and Covariation. In addition, kinematic analysis related the influence of skill levels as affected by the specific motor control deficits in Parkinson patients to the process of learning. As a result, patients showed similar learning in early and late stages compared to the control subjects. Differences occurred in short-term retention tests; patients' performance constantly decreased after breaks arising from poorer release timing. However, patients were able to overcome the initial timing problems within the course of each practice session and could further improve their throwing performance. Thus, results demonstrate the intact ability to learn a novel motor skill in non-demented, medicated patients with Parkinson's disease and indicate confounding effects of motor control deficits on retention performance.

Parallel associative processing in the dorsal striatum: segregation of stimulus-response and cognitive control subregions

Although evidence suggests that the dorsal striatum contributes to multiple learning and memory functions, there nevertheless remains considerable disagreement on the specific associative roles of different neuroanatomical subregions. We review evidence indicating that the dorsolateral striatum (DLS) is a substrate for stimulus-response habit formation - incremental strengthening of simple S-R bonds - via input from sensorimotor neocortex while the dorsomedial striatum (DMS) contributes to behavioral flexibility - the cognitive control of behavior - via prefrontal and limbic circuits engaged in relational and spatial information processing. The parallel circuits through dorsal striatum interact with incentive/affective motivational processing in the ventral striatum and portions of the prefrontal cortex leading to overt responding under specific testing conditions. Converging evidence obtained through a detailed task analysis and neurobehavioral assessment is beginning to illuminate striatal subregional interactions and relations to the rest of the mammalian brain.

Impaired L1 and executive control after left basal ganglia damage in a bilingual Basque-Spanish person with aphasia

Bilinguals must focus their attention to control competing languages. In bilingual aphasia, damage to the fronto-subcortical loop may lead to pathological language switching and mixing and the attrition of the more automatic language (usually L1). We present the case of JZ, a bilingual Basque-Spanish 53-year-old man who, after haematoma in the left basal ganglia, presented with executive deficits and aphasia, characterised by more impaired language processing in Basque, his L1. Assessment with the Bilingual Aphasia Test revealed impaired spontaneous and automatic speech production and speech rate in L1, as well as impaired L2-to-L1 sentence translation. Later observation led to the assessment of verbal and non-verbal executive control, which allowed JZ's impaired performance on language tasks to be related to executive dysfunction. In line with previous research, we report the significant attrition of L1 following damage to the left basal ganglia, reported for the first time in a Basque-Spanish bilingual. Implications for models of declarative and procedural memory are discussed.

Ovariectomy alters energy metabolism in rat striatum: effect of supplementation with soy diet rich in isoflavones

In the present study we investigated the effect of ovariectomy on some parameters of energy metabolism, namely Na(+),K(+)-ATPase and pyruvate kinase activities, as well as the mitochondrial respiratory chain enzymes activities succinate dehydrogenase, complex II and cytochrome c oxidase in rat striatum. The influence of soy diet rich in isoflavones on the effects elicited by ovariectomy on enzyme activities was also evaluated. Female adult Wistar rats were assigned to one of the following groups: sham (submitted to surgery without removal of the ovaries) and ovariectomized. Seven days after surgery animals were fed for 30 days on a special diet with soy protein or a standard diet with casein (control). Rats were sacrificed after treatment and the striatum was dissected. Results showed that rats subjected to ovariectomy presented a significant increase in Na(+),K(+)-ATPase, succinate dehydrogenase and complex II activities. Treatment with isoflavones-rich soy diet was able to reverse the increase of Na(+),K(+)-ATPase activity, but was not effective in reversing the changes caused by ovariectomy on succinate dehydrogenase and complex II activities. Since ovariectomy mimics postmenopausal changes, our findings suggest that dysfunction of brain energy metabolism may be related to neurological symptoms observed in some postmenopausal women.

Phasic reward responses in the monkey striatum as detected by voltammetry with diamond microelectrodes

Reward-induced burst firing of dopaminergic neurons has mainly been studied in the primate midbrain. Voltammetry allows high-speed detection of dopamine release in the projection area. Although voltammetry has revealed presynaptic modulation of dopamine release in the striatum, to date, reward-induced release in awakened brains has been recorded only in rodents. To make such recordings, it is possible to use conventional carbon fibres in monkey brains but the use of these fibres is limited by their physical fragility. In this study, constant-potential amperometry was applied to novel diamond microelectrodes for high-speed detection of dopamine. In primate brains during Pavlovian cue-reward trials, a sharp response to a reward cue was detected in the caudate of Japanese monkeys. Overall, this method allows measurements of monoamine release in specific target areas of large brains, the findings from which will expand the knowledge of reward responses obtained by unit recordings.

Changes in striatal procedural memory coding correlate with learning deficits in a mouse model of Huntington disease

In hereditary neurodegenerative Huntington disease (HD), early cognitive impairments before motor deficits have been hypothesized to result from dysfunction in the striatum and cortex before degeneration. To test this hypothesis, we examined the firing properties of single cells and local field activity in the striatum and cortex of pre-motor-symptomatic R6/1 transgenic mice while they were engaged in a procedural learning task, the performance on which typically depends on the integrity of striatum and basal ganglia. Here, we report that a dramatically diminished recruitment of the vulnerable striatal projection cells, but not local interneurons, of R6/1 mice in coding for the task, compared with WT littermates, is associated with severe deficits in procedural learning. In addition, both the striatum and cortex in these mice showed a unique oscillation at high γ-frequency. These data provide crucial information on the in vivo cellular processes in the corticostriatal pathway through which the HD mutation exerts its effects on cognitive abilities in early HD.

White matter integrity and lack of insight in schizophrenia and schizoaffective disorder

OBJECTIVE

Poor insight into illness is commonly associated with schizophrenia and has implications for the clinical outcome of the disease. A better understanding of the neurobiology of these insight deficits may help the development of new treatments targeting insight. Despite the importance of this issue, the neural correlates of insight deficits in schizophrenia remain poorly understood.

METHOD

Thirty-six individuals diagnosed with schizophrenia or schizoaffective disorder underwent diffusion tensor imaging (DTI). The subjects were assessed on two dimensions of insight (symptom awareness and attribution of symptoms) using the Scale to Assess Unawareness of Mental Disorder (SUMD). Level of psychosis was assessed with the Positive and Negative Syndrome Scale (PANSS).

RESULTS

White matter abnormalities in the right superior frontal gyrus, left middle frontal gyrus, bilateral parahippocampal gyrus, adjacent to the right caudate head, right thalamus, left insula, left lentiform nucleus, left fusiform gyrus, bilateral posterior cingulate, left anterior cingulate, right cingulate gyrus, left lingual gyrus, and bilateral claustrum were associated with symptom unawareness. Misattribution of symptoms was related to deficits in the white matter adjacent to the right lentiform nucleus, left middle temporal gyrus, and the right precuneus.

CONCLUSIONS

Impaired insight in schizophrenia implicates a complex neural circuitry: white matter deficits in fronto-temporo brain regions are linked to symptom unawareness; compromised temporal and parietal white matter regions are involved in the misattribution of symptoms. These findings suggest the multidimensional construct of insight has multiple neural determinants.

Computational models of motivated action selection in corticostriatal circuits

Computational models of the basal ganglia have matured and received increasing attention over the last decade. This article reviews some of the theoretical advances offered by these models, focusing on motor and cognitive action selection, learning, and the interaction between multiple corticostriatal circuits in selection and learning.

Perseveration on a reversal-learning task correlates with rates of self-directed behavior in nonhuman primates

In humans and several nonhuman animals, repetitive behavior is associated with deficits on executive function tasks involving response inhibition. We tested for this relationship in nonhuman primates by correlating rates of normative behavior to performance on a reversal-learning task in which animals were required to inhibit a previously learned rule. We focused on rates of self-directed behavior (scratch, autogroom, self touch and manipulation) because these responses are known indicators of arousal or anxiety in primates, however, we also examined rates of other categories of behavior (e.g., locomotion). Behavior rates were obtained from 14 animals representing three nonhuman primate species (Macaca silenus, Saimiri sciureus, Cebus apella) living in separate social groups. The same animals were tested on a reversal-learning task in which they were presented with a black and a grey square on a touch screen and were trained to touch the black square. Once animals learned to select the black square, reward contingencies were reversed and animals were rewarded for selecting the grey square. Performance on the reversal-learning task was positively correlated to self-directed behavior in that animals that exhibited higher rates of self-directed behavior required more trials to achieve reversal. Reversal learning was not correlated to rates of any other category of behavior. Results indicate that rates of behavior associated with anxiety and arousal provide an indicator of executive function in nonhuman primates. The relationship suggests continuity between nonhuman primates and humans in the link between executive functioning and repetitive behavior.

Selective serotonin reuptake inhibitor antidepressants potentiate methylphenidate (Ritalin)-induced gene regulation in the adolescent striatum

The psychostimulant methylphenidate (Ritalin) is used in conjunction with selective serotonin reuptake inhibitors (SSRIs) in the treatment of medical conditions such as attention-deficit hyperactivity disorder with anxiety/depression comorbidity and major depression. Co-exposure also occurs in patients on SSRIs who use psychostimulant 'cognitive enhancers'. Methylphenidate is a dopamine/norepinephrine reuptake inhibitor that produces altered gene expression in the forebrain; these effects partly mimic gene regulation by cocaine (dopamine/norepinephrine/serotonin reuptake inhibitor). We investigated whether the addition of SSRIs (fluoxetine or citalopram; 5 mg/kg) modified gene regulation by methylphenidate (2-5 mg/kg) in the striatum and cortex of adolescent rats. Our results show that SSRIs potentiate methylphenidate-induced expression of the transcription factor genes zif268 and c-fos in the striatum, rendering these molecular changes more cocaine-like. Present throughout most of the striatum, this potentiation was most robust in its sensorimotor parts. The methylphenidate + SSRI combination also enhanced behavioral stereotypies, consistent with dysfunction in sensorimotor striatal circuits. In so far as such gene regulation is implicated in psychostimulant addiction, our findings suggest that SSRIs may enhance the addiction potential of methylphenidate.

Modulation of Neuronal Activity in the Monkey Putamen Associated With Changes in the Habitual Order of Sequential Movements

The striatum, especially its dorsolateral part, plays a major role in motor skill learning and habit formation, but it is still unclear how this contribution might be mediated at the neuronal level. We recorded single neurons in the posterior putamen of two monkeys performing an overlearned sequence of arm reaching movements to examine whether task-related activities are sensitive to manipulations of the serial order of stimulus-target locations. The monkeys' capacity to learn sequential regularities was assessed by comparing arm movement latencies and saccadic ocular reactions when a fixed repeating sequence was replaced with a random sequence. We examined neurons classified as phasically active projection neurons (PANs) and tonically active presumed cholinergic interneurons (TANs). About one-third of the PANs (35/106, 33%) activated during specific parts of a trial displayed modulations of their level of activation when the sequential structure was changed. This differential activity consisted of either decreases or increases in activity without altering the time period during which task-related activations occurred. In addition, half of the TANs (41/80, 51%) changed their responses to task stimuli with the sequence switch, indicating that the response selectivity of TANs reflects the detection of the context that requires adaptation to changes in the serial order of stimulus presentations. Our findings suggest that task-related changes in activity of projection neurons may be an important factor contributing to the production and adjustment of sequential behavior executed in an automatic fashion, whereas putative interneurons may provide a signal for performance monitoring in specific contexts.

Internalizing versus externalizing comorbidity: neural circuit hypothesis

The aim of the present study was to review and update historical controversies in relation to the limbic system in light of advances in the neuroanatmy and neurobiology of cortico-striatal-thalamic-cortical (CSTC) neural circuits. It is hypothesized that observed comorbidities between internalizing and externalizing childhood syndromes may be better understood in terms of excess communication between subcortical brain circuits. A neuroanatomical understanding of reciprocal and non-reciprocal connections between modular circuits provides a basis for understanding previously puzzling aspects of internalizing, externalizing comorbidity. Function depends on the degree of cooperative integration of goal orientation, Pavlovian stimulus response, and operant processing in cortical-subcortical circuits, and interventions may be targeted at any of these functions.

Dopamine D2 receptor density in the limbic striatum is related to implicit but not explicit movement sequence learning

A large body of literature suggests that motor sequence learning involves dopamine-modulated plastic processes in the basal ganglia. Sequence learning can occur both implicitly, without conscious awareness and intention to learn, and explicitly, i.e., under conscious control. Here, we investigated whether individual differences in implicit and explicit sequence learning of movement sequences in a group of 15 healthy participants are related to dopamine D2 receptor densities in functional subregions of the striatum. Sequence learning was assessed using the serial reaction time task, and measures of implicit and explicit knowledge were estimated using a process dissociation procedure. Correlation analyses were performed between these measures and D2 receptor densities, which had been measured previously with positron emission tomography. Striatal D2 densities were negatively related to measures of sequence learning. In the limbic subregion, D2 densities were specifically related to implicit but not explicit learning. These findings suggest that individual differences in striatal DA function underlie differences in sequence learning ability and support that implicit and explicit sequence learning depend on partly distinct neural circuitry. The findings are also in line with the general view that implicit learning systems are evolutionarily primitive and tend to rely more on phylogenetically old neural circuitry than does explicit learning and cognition.

Neurocomputational models of motor and cognitive deficits in Parkinson's disease

We review the contributions of biologically constrained computational models to our understanding of motor and cognitive deficits in Parkinson's disease (PD). The loss of dopaminergic neurons innervating the striatum in PD, and the well-established role of dopamine (DA) in reinforcement learning (RL), enable neural network models of the basal ganglia (BG) to derive concrete and testable predictions. We focus in this review on one simple underlying principle - the notion that reduced DA increases activity and causes long-term potentiation in the indirect pathway of the BG. We show how this theory can provide a unified account of diverse and seemingly unrelated phenomena in PD including progressive motor degeneration as well as cognitive deficits in RL, decision making and working memory. DA replacement therapy and deep brain stimulation can alleviate some aspects of these impairments, but can actually introduce negative effects such as motor dyskinesias and cognitive impulsivity. We discuss these treatment effects in terms of modulation of specific mechanisms within the computational framework. In addition, we review neurocomputational interpretations of increased impulsivity in the face of response conflict in patients with deep-brain-stimulation.

Brief subthreshold events can act as Hebbian signals for long-term plasticity

BACKGROUND

Action potentials are thought to be determinant for the induction of long-term synaptic plasticity, the cellular basis of learning and memory. However, neuronal activity does not lead systematically to an action potential but also, in many cases, to synaptic depolarizing subthreshold events. This is particularly exemplified in corticostriatal information processing. Indeed, the striatum integrates information from the whole cerebral cortex and, due to the membrane properties of striatal medium spiny neurons, cortical inputs do not systematically trigger an action potential but a wide range of subthreshold postsynaptic depolarizations. Accordingly, we have addressed the following question: does a brief subthreshold event act as a Hebbian signal and induce long-term synaptic efficacy changes?

METHODOLOGY/PRINCIPAL FINDINGS

Here, using perforated patch-clamp recordings on rat brain corticostriatal slices, we demonstrate, that brief (30 ms) subthreshold depolarizing events in quasi-coincidence with presynaptic activity can act as Hebbian signals and are sufficient to induce long-term synaptic plasticity at corticostriatal synapses. This "subthreshold-depolarization dependent plasticity" (SDDP) induces strong, significant and bidirectional long-term synaptic efficacy changes at a very high occurrence (81%) for time intervals between pre- and postsynaptic stimulations (Deltat) of -110<Deltat<+110 ms. Such subthreshold depolarizations are able to induce robust long-term depression (cannabinoid type-1 receptor-activation dependent) as well as long-term potentiation (NMDA receptor-activation dependent).

CONCLUSION/SIGNIFICANCE

Our data show the existence of a robust, reliable and timing-dependent bidirectional long-term plasticity induced by brief subthreshold events paired with presynaptic activity. The existence of a subthreshold-depolarization dependent plasticity extends considerably, beyond the action potential, the neuron's capabilities to express long-term synaptic efficacy changes.

Complete 3D visualization of primate striosomes by KChIP1 immunostaining

High-resolution 3D reconstruction and morphometric analysis of striosomes was carried out in macaque monkeys by using immunocytochemistry for the Kv4 potassium channel subunit potassium channel interacting protein 1 (KChIP1), a novel marker. The striosomes form a connected reticulum made up of two distinct planar sheets spanning several millimeters in the putamen, and long finger-like branches in the caudate nucleus and putamen. Although their spatial organization is variable, morphometric analysis of the striosomes, utilizing skeletonizations, reveals several quantitative invariant measures of striosome organization, including the following findings: 1) individual bifurcation-free striosome branches are 355 +/- 108.5 microm in diameter and 1,013 +/- 751 microm in length, and are both lognormally distributed; and 2) striosome branches exhibit three pronounced orientation preferences that are approximately orthogonal. The former finding suggests a fundamental anatomical and functional component of the striatum, whereas the latter indicates that striosomes are more lattice-like than their spatial variability suggests. The perceived variable spatial organization of the striosomes in primates belies many invariant features that may reflect striatal function, development, and pathophysiology.

Dopamine release in the human striatum: motor and cognitive tasks revisited

Striatal dopamine (DA) release has been shown during behavioural tasks, but the relative contribution of motor, reward, and cognitive components is unclear. Dopamine release was quantified using [(11)C]-raclopride in two studies using a triple-scan approach, comprising active task, motor control, and rest. In the first, bolus radiotracer was delivered during a sequential motor learning paradigm; in the second, a spatial planning task, bolus plus constant infusion was applied. [(11)C]-raclopride binding potentials (BP(ND)s) in striatal functional subdivisions were compared across conditions. [(11)C]-raclopride BP(ND) was significantly reduced in active task compared with rest in both the sensorimotor and associative striatum in both studies, because of differences between rest and motor control conditions. In both regions, the motor control BP(ND) fell between the rest and active task in the planning study, but the difference between motor control and active task conditions was not significant. No such changes were observed in the limbic striatum. Using rigorous methodology, this study validates earlier evidence that striatal DA release occurs during behavioural challenges. Increased DA release during movement was reliably detected in the sensorimotor and associative striatum, supporting use of the functional subdivision model in humans. No additional DA release was observed specific to the cognitive component of either task.

Deficient sensorimotor gating induced by selective breeding in rats is improved by entopeduncular nucleus lesions

Deficient prepulse inhibition (PPI) of startle reflects disturbed sensorimotor gating found in certain neuropsychiatric disorders, such as Tourette's syndrome, ADHD, Huntington's and schizophrenia. We here tested, whether lesions of the entopeduncular nucleus (EPN) would improve a PPI-deficit induced by selective breeding. Rats with breeding induced high and low expression of PPI were stereotaxically microinjected with ibotenate (0.2 microg in 0.3 microl phosphate buffered saline) or vehicle into the EPN and two weeks later tested for PPI of the acoustic startle response (ASR) and motor activity. Lesions of the EPN counteracted the breeding-induced PPI-deficit and reduced ASR in the PPI low group without affecting their motor activity. In the PPI high group EPN lesions did not affect PPI, ASR, and motor activity. This work indicates an important role of the EPN in the modulation of sensorimotor gating. Additionally, PPI low rats may provide a non-pharmacological model that can be used to develop new therapeutic strategies for neuropsychiatric disorders.

Activation of the Cholinergic System Endows Compositional Properties to Striatal Cell Assemblies

Striatal cell assemblies are thought to encode network states related to associative learning, procedural memory, and the sequential organization of behavior. Cholinergic neurotransmission modulates memory processes in the striatum and other brain structures. This work asks if the activity of striatal microcircuits observed in living nervous tissue, with attributes similar to cell assemblies, exhibit some of the properties proposed to be necessary to compose memory traces. Accordingly, we used whole cell and calcium-imaging techniques to investigate the cholinergic modulation of striatal neuron pools that have been reported to exhibit several properties expected from cell assemblies such as synchronous states of activity and the alternation of this activity among different neuron pools. We analyzed the cholinergic modulation of the activity of neuron pools with multidimensional reduction techniques and vectorization of network dynamics. It was found that the activation of the cholinergic system enables striatal cell assemblies with properties that have been posited for recurrent neural artificial networks with memory storage capabilities. Graph theory techniques applied to striatal network states revealed sequences of vectors with a recursive dynamics similar to closed reverberating cycles. The cycles exhibited a modular architecture and a hierarchical organization. It is then concluded that, under certain conditions, the cholinergic system enables the striatal microcircuit with the ability to compose complex sequences of activity. Neuronal recurrent networks with the characteristics encountered in the present experiments are proposed to allow repeated sequences of activity to become memories and repeated memories to compose learned motor procedures.

Integrating memories in the human brain: hippocampal-midbrain encoding of overlapping events

Decisions are often guided by generalizing from past experiences. Fundamental questions remain regarding the cognitive and neural mechanisms by which generalization takes place. Prior data suggest that generalization may stem from inference-based processes at the time of generalization. By contrast, generalization may emerge from mnemonic processes occurring while premise events are encoded. Here, participants engaged in a two-phase learning and generalization task, wherein they learned a series of overlapping associations and subsequently generalized what they learned to novel stimulus combinations. Functional MRI revealed that successful generalization was associated with coupled changes in learning-phase activity in the hippocampus and midbrain (ventral tegmental area/substantia nigra). These findings provide evidence for generalization based on integrative encoding, whereby overlapping past events are integrated into a linked mnemonic representation. Hippocampal-midbrain interactions support the dynamic integration of experiences, providing a powerful mechanism for building a rich associative history that extends beyond individual events.

A dopaminergic axon lattice in the striatum and its relationship with cortical and thalamic terminals

Interactions between glutamatergic corticostriatal afferents and dopaminergic nigrostriatal afferents are central to basal ganglia function. The thalamostriatal projection provides a glutamatergic innervation of similar magnitude to the corticostriatal projection. We tested the hypotheses that (1) thalamostriatal synapses have similar spatial relationships with dopaminergic axons as corticostriatal synapses do and (2) the spatial relationships between excitatory synapses and dopaminergic axons are selective associations. We examined at the electron microscopic level rat striatum immunolabeled to reveal vesicular glutamate transporters (VGluTs) 1 and 2, markers of corticostriatal and thalamostriatal terminals, respectively, together with tyrosine hydroxylase (TH) to reveal dopaminergic axons. Over 80% of VGluT-positive synapses were within 1 microm of a TH-positive axon and >40% were within 1 microm of a TH-positive synapse. Of structures postsynaptic to VGluT1- or VGluT2-positive terminals, 21 and 27%, respectively, were apposed by a TH-positive axon and about half of these made synaptic contact. When structures postsynaptic to VGluT-positive terminals and VGluT-positive terminals themselves were normalized for length of plasma membrane, the probability of them being apposed by, or in synaptic contact with, a TH-positive axon was similar to that of randomly selected structures. Extrapolation of the experimental data to more closely reflect the distribution in 3D reveals that all structures in the striatum are within approximately 1 microm of a TH-positive synapse. We conclude that (1) thalamostriatal synapses are in a position to be influenced by released dopamine to a similar degree as corticostriatal synapses are and (2) these associations arise from a nonselective dopaminergic axon lattice.

D-aspartate prevents corticostriatal long-term depression and attenuates schizophrenia-like symptoms induced by amphetamine and MK-801

Since their discovery in the mammalian CNS, D-aspartate and D-serine have aroused a strong interest with regard to their role as putative neuromodulatory molecules. Whereas the functional role of D-serine as an endogenous coagonist of NMDA receptors (NMDARs) has been elucidated, the biological significance of D-aspartate in the brain is still mostly unclear. In the present study, we demonstrated that nonphysiological high levels of D-aspartate (1) increased in vivo NMDAR activity, (2) attenuated prepulse inhibition deficits induced by amphetamine and MK-801 [(+)-5-methyl-10,11-dihydro-5H-dibenzo[a,d]-cyclohepten-5,10-imine hydrogen maleate], (3) produced striatal adaptations of glutamate synapses resembling those observed after chronic haloperidol treatment, and (4) enhanced hippocampal NMDAR-dependent memory. This evidence was obtained using two different experimental strategies that produced an abnormal increase of endogenous D-aspartate levels in the mouse: a genetic approach based on the targeted deletion of the D-aspartate oxidase gene and a pharmacological approach based on oral administration of D-aspartate. This work provides in vivo evidence of a neuromodulatory role exerted by D-aspartate on NMDAR signaling and raises the intriguing hypothesis that also this D-amino acid, like D-serine, could be used as a therapeutic agent in the treatment of schizophrenia-related symptoms.

Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective

Research on human and animal behavior has long emphasized its hierarchical structure-the divisibility of ongoing behavior into discrete tasks, which are comprised of subtask sequences, which in turn are built of simple actions. The hierarchical structure of behavior has also been of enduring interest within neuroscience, where it has been widely considered to reflect prefrontal cortical functions. In this paper, we reexamine behavioral hierarchy and its neural substrates from the point of view of recent developments in computational reinforcement learning. Specifically, we consider a set of approaches known collectively as hierarchical reinforcement learning, which extend the reinforcement learning paradigm by allowing the learning agent to aggregate actions into reusable subroutines or skills. A close look at the components of hierarchical reinforcement learning suggests how they might map onto neural structures, in particular regions within the dorsolateral and orbital prefrontal cortex. It also suggests specific ways in which hierarchical reinforcement learning might provide a complement to existing psychological models of hierarchically structured behavior. A particularly important question that hierarchical reinforcement learning brings to the fore is that of how learning identifies new action routines that are likely to provide useful building blocks in solving a wide range of future problems. Here and at many other points, hierarchical reinforcement learning offers an appealing framework for investigating the computational and neural underpinnings of hierarchically structured behavior.

Muscarinic M(1) modulation of N and L types of calcium channels is mediated by protein kinase C in neostriatal neurons

In some neurons, muscarinic M(1)-class receptors control L-type (Ca(V)1) Ca(2+)-channels via protein kinase C (PKC) or calcineurin (phosphatase 2B; PP-2B) signaling pathways. Both PKC and PP-2B pathways start with phospholipase C (PLC) activation. In contrast, P/Q- and N-type (Ca(V)2.1, 2.2, respectively) Ca(2+)-channels are controlled by M(2)-class receptors via G proteins that may act, directly, to modulate these channels. The hypothesis of this work is that this description is not enough to explain muscarinic modulation of Ca(2+) channels in rat neostriatal projection neurons. Thus, we took advantage of the specific muscarinic toxin 3 (MT-3) to block M(4)-type receptors in neostriatal neurons, and leave in isolation the M(1)-type receptors to study them separately. We then asked what Ca(2+) channels are modulated by M(1)-type receptors only. We found that M(1)-receptors do modulate L, N and P/Q-types Ca(2+) channels. This modulation is blocked by the M(1)-class receptor antagonist (muscarinic toxin 7, MT-7) and is voltage-independent. Thereafter, we asked what signaling pathways, activated by M(1)-receptors would control these channels. We found that inactivation of PLC abolishes the modulation of all three channel types. PKC activators (phorbol esters) mimic muscarinic actions, whereas reduction of intracellular calcium virtually abolishes all modulation. As expected, PKC inhibitors prevented the muscarinic reduction of the afterhyperpolarizing potential (AHP), an event known to be dependent on Ca(2+) entry via N- and P/Q-type Ca(2+) channels. However, PKC inhibitors (bisindolylmaleimide I and PKC-1936) only block modulation of currents through N and L types Ca(2+) channels; while the modulation of P/Q-type Ca(2+) channels remains unaffected. These results show that different branches of the same signaling cascade can be used to modulate different Ca(2+) channels. Finally, we found no evidence of calcineurin modulating these Ca(2+) channels during M(1)-receptor activation, although, in the same cells, we demonstrate functional PP-2B by activating dopaminergic D(2)-receptor modulation.

A central circuit of the mind

The methodologies of cognitive architectures and functional magnetic resonance imaging can mutually inform each other. For example, four modules of the ACT-R (adaptive control of thought - rational) cognitive architecture have been associated with four brain regions that are active in complex tasks. Activity in a lateral inferior prefrontal region reflects retrieval of information in a declarative module; activity in a posterior parietal region reflects changes to problem representations in an imaginal module; activity in the anterior cingulate cortex reflects the updates of control information in a goal module; and activity in the caudate nucleus reflects execution of productions in a procedural module. Differential patterns of activation in such central regions can reveal the time course of different components of complex cognition.

Task-dependent encoding of space and events by striatal neurons is dependent on neural subtype

The dorsal striatum plays a critical role in procedural learning and memory. Current models of basal ganglia assume that striatal neurons and circuitry are critical for the execution of overlearned, habitual sequences of action. However, less is known about how the striatum encodes task information that guides the performance of actions in procedural tasks. To explore the striatal encoding of task information, we compared the behavioral correlates of striatal neurons tested in two tasks: a multiple T-maze task in which reward delivery was entirely predictable based on spatial cues (the Multiple-T task), and a task in which rats ran on a rectangular track, but food delivery depended on the distance traveled on the track and was not dependent solely on spatial location (the Take-5 task). Striatal cells recorded on these tasks were divisible into three cell types: phasic-firing neurons (PFNs), tonically firing neurons (TFNs), and high-firing neurons (HFNs) and similar proportions of each cell type were found in each task. However, the behavioral correlates of each cell type were differentially sensitive to the type of task rats were performing. PFNs were responsive to specific task-parameters on each task. TFNs showed reliable burst-and-pause responses following food delivery and other events that were consistent with tonically active neurons (TANs) on the Take-5 (non-spatial) task but not on the Multiple-T (spatial) task. HFNs showed spatial oscillations on the Multiple-T (spatial) task but not the Take-5 (non-spatial) task. Reconstruction of the rats' position on the maze was highly accurate when using striatal ensembles recorded on the Multiple-T (spatial) task, but not when using ensembles recorded on the Take-5 (non-spatial) task. In contrast, reconstruction of time following food delivery was successful in both tasks. The results indicated a strong task dependency of the quality of the spatial, but not the reward-related, striatal representations on these tasks. These results suggest that striatal spatial representations depend on the degree to which spatial task-parameters can be unambiguously associated with goals.

Encoding Network States by Striatal Cell Assemblies

Correlated activity in cortico-basal ganglia circuits plays a key role in the encoding of movement, associative learning and procedural memory. How correlated activity is assembled by striatal microcircuits is not understood. Calcium imaging of striatal neuronal populations, with single-cell resolution, reveals sporadic and asynchronous activity under control conditions. However, N-methyl-d-aspartate (NMDA) application induces bistability and correlated activity in striatal neurons. Widespread neurons within the field of observation present burst firing. Sets of neurons exhibit episodes of recurrent and synchronized bursting. Dimensionality reduction of network dynamics reveals functional states defined by cell assemblies that alternate their activity and display spatiotemporal pattern generation. Recurrent synchronous activity travels from one cell assembly to the other often returning to the original assembly; suggesting a robust structure. An initial search into the factors that sustain correlated activity of neuronal assemblies showed a critical dependence on both intrinsic and synaptic mechanisms: blockage of fast glutamatergic transmission annihilates all correlated firing, whereas blockage of GABAergic transmission locked the network into a single dominant state that eliminates assembly diversity. Reduction of L-type Ca2+-current restrains synchronization. Each cell assembly comprised different cells, but a small set of neurons was shared by different assemblies. A great proportion of the shared neurons was local interneurons with pacemaking properties. The network dynamics set into action by NMDA in the striatal network may reveal important properties of striatal microcircuits under normal and pathological conditions.

Connexin31.1 deficiency in the mouse impairs object memory and modulates open-field exploration, acetylcholine esterase levels in the striatum, and cAMP response element-binding protein levels in the striatum and piriform cortex

Neuronal gap junctions in the brain, providing intercellular electrotonic signal transfer, have been implicated in physiological and behavioral correlates of learning and memory. In connexin31.1 (Cx31.1) knockout (KO) mice the coding region of the Cx31.1 gene was replaced by a LacZ reporter gene. We investigated the impact of Cx31.1 deficiency on open-field exploration, the behavioral response to an odor, non-selective attention, learning and memory performance, and the levels of memory-related proteins in the hippocampus, striatum and the piriform cortex. In terms of behavior, the deletion of the Cx31.1 coding DNA in the mouse led to increased exploratory behaviors in a novel environment, and impaired one-trial object recognition at all delays tested. Despite strong Cx31.1 expression in the peripheral and central olfactory system, Cx31.1 KO mice exhibited normal behavioral responses to an odor. We found increased levels of acetylcholine esterase (AChE) and cAMP response element-binding protein (CREB) in the striatum of Cx31.1 KO mice. In the piriform cortex the Cx31.1 KO mice had an increased heterogeneity of CREB expression among neurons. In conclusion, gap-junctions featuring the Cx31.1 protein might be involved in open-field exploration as well as object memory and modulate levels of AChE and CREB in the striatum and piriform cortex.

Presynaptic modulation by somatostatin in the neostriatum

Medium spiny projection neurons (MSNs) are the main neuronal population in the neostriatum. MSNs are inhibitory and GABAergic. MSNs connect with other MSNs via local axon collaterals that produce lateral inhibition, which is thought to select cell assemblies for motor action. MSNs also receive inhibitory inputs from GABAergic local interneurons. This work shows, through the use of the paired pulse protocol, that somatostatin (SST) acts presynaptically to regulate GABA release from the terminals interconnecting MSNs. This SST action is reversible and not mediated through the release of dopamine. It is blocked by the SST receptor (SSTR) antagonist ciclosomatostatin (cicloSST). In contrast, SST does not regulate inhibition coming from interneurons. Because, SST is released by a class of local interneuron, it is concluded that this neuron helps to regulate the selection of motor acts.

Motor-skill learning in a novel running-wheel task is dependent on D1 dopamine receptors in the striatum

Evidence indicates that dopamine receptors regulate processes of procedural learning in the sensorimotor striatum. Our previous studies revealed that the indirect dopamine receptor agonist cocaine alters motor-skill learning-associated gene regulation in the sensorimotor striatum. Cocaine-induced gene regulation in the striatum is principally mediated by D1 dopamine receptors. We investigated the effects of cocaine and striatal D1 receptor antagonism on motor-skill learning. Rats were trained on a running wheel (40-60 min, 2-5 days) to learn a new motor skill, that is, the ability to control the movement of the wheel. Immediately before each training session, the animals received an injection of vehicle or cocaine (25 mg/kg, i.p.), and/or the D1 receptor antagonist SCH-23390 (0, 3, 10 microg/kg, i.p., or 0, 0.3, 1 microg, intrastriatal via chronically implanted cannula). The animal's ability to control/balance the moving wheel (wheel skill) was tested before and repeatedly after the training. Normal wheel-skill memory lasted for at least 4 weeks. Cocaine administered before the training tended to attenuate skill learning. Systemic administration of SCH-23390 alone also impaired skill learning. However, cocaine given in conjunction with the lower SCH-23390 dose (3 microg/kg) reversed the inhibition of skill learning produced by the D1 receptor antagonist, enabling intact skill performance during the whole post-training period. In contrast, when cocaine was administered with the higher SCH-23390 dose (10 microg/kg), skill performance was normalized 1-6 days after the training, but these rats lost their improved wheel skill by day 18 after the training. Similar effects were produced by SCH-23390 (0.3-1 microg) infused into the striatum. Our results indicate that cocaine interferes with normal motor-skill learning, which seems to be dependent on optimal D1 receptor signaling. Furthermore, our findings demonstrate that D1 receptors in the striatum are critical for consolidation of long-term skill memory.

Action and outcome encoding in the primate caudate nucleus

The basal ganglia appear to have a central role in reinforcement learning. Previous experiments, focusing on activity preceding movement execution, support the idea that dorsal striatal neurons bias action selection according to the expected values of actions. However, many phasically active striatal neurons respond at a time too late to initiate or select movements. Given the data suggesting a role for the basal ganglia in reinforcement learning, postmovement activity may therefore reflect evaluative processing important for learning the values of actions. To better understand these postmovement neurons, we determined whether individual striatal neurons encode information about saccade direction, whether a reward had been received, or both. We recorded from phasically active neurons in the caudate nucleus while monkeys performed a probabilistically rewarded delayed saccade task. Many neurons exhibited peak responses after saccade execution (77 of 149) that were often tuned for the direction of the preceding saccade (61 of 77). Of those neurons responding during the reward epoch, one subset showed direction tuning for the immediately preceding saccade (43 of 60), whereas another subset responded differentially on rewarded versus unrewarded trials (35 of 60). We found that there was relatively little overlap of these properties in individual neurons. The encoding of action and outcome was performed by largely separate populations of caudate neurons that were active after movement execution. Thus, striatal neurons active primarily after a movement appear to be segregated into two distinct groups that provide complimentary information about the outcomes of actions.

Behavioral functions of the mesolimbic dopaminergic system: an affective neuroethological perspective

The mesolimbic dopaminergic (ML-DA) system has been recognized for its central role in motivated behaviors, various types of reward, and, more recently, in cognitive processes. Functional theories have emphasized DA's involvement in the orchestration of goal-directed behaviors and in the promotion and reinforcement of learning. The affective neuroethological perspective presented here views the ML-DA system in terms of its ability to activate an instinctual emotional appetitive state (SEEKING) evolved to induce organisms to search for all varieties of life-supporting stimuli and to avoid harms. A description of the anatomical framework in which the ML system is embedded is followed by the argument that the SEEKING disposition emerges through functional integration of ventral basal ganglia (BG) into thalamocortical activities. Filtering cortical and limbic input that spreads into BG, DA transmission promotes the "release" of neural activity patterns that induce active SEEKING behaviors when expressed at the motor level. Reverberation of these patterns constitutes a neurodynamic process for the inclusion of cognitive and perceptual representations within the extended networks of the SEEKING urge. In this way, the SEEKING disposition influences attention, incentive salience, associative learning, and anticipatory predictions. In our view, the rewarding properties of drugs of abuse are, in part, caused by the activation of the SEEKING disposition, ranging from appetitive drive to persistent craving depending on the intensity of the affect. The implications of such a view for understanding addiction are considered, with particular emphasis on factors predisposing individuals to develop compulsive drug seeking behaviors.