Separate groups of dopamine neurons innervate caudate head and tail encoding flexible and stable value memories

Implication of regional selectivity of dopamine deficits in impaired suppressing of involuntary movements in Parkinson's disease

To improve the initiation and speed of intended action, one of the crucial mechanisms is suppressing unwanted movements that interfere with goal-directed behavior, which is observed relatively aberrant in Parkinson's disease patients. Recent research has highlighted that dopamine deficits in Parkinson's disease predominantly occur in the caudal lateral part of the substantia nigra pars compacta (SNc) in human patients. We previously found two parallel circuits within the basal ganglia, primarily divided into circuits mediated by the rostral medial part and caudal lateral part of the SNc dopamine neurons. We have further discovered that the indirect pathway in caudal basal ganglia circuits, facilitated by the caudal lateral part of the SNc dopamine neurons, plays a critical role in suppressing unnecessary involuntary movements when animals perform voluntary goal-directed actions. We thus explored recent research in humans and non-human primates focusing on the distinct functions and networks of the caudal lateral part of the SNc dopamine neurons to elucidate the mechanisms involved in the impairment of suppressing involuntary movements in Parkinson's disease patients.

Robust memory of face moral values is encoded in the human caudate tail: a simultaneous EEG-fMRI study

Moral judgements about people based on their actions is a key component that guides social decision making. It is currently unknown how positive or negative moral judgments associated with a person's face are processed and stored in the brain for a long time. Here, we investigate the long-term memory of moral values associated with human faces using simultaneous EEG-fMRI data acquisition. Results show that only a few exposures to morally charged stories of people are enough to form long-term memories a day later for a relatively large number of new faces. Event related potentials (ERPs) showed a significant differentiation of remembered good vs bad faces over centerofrontal electrode sites (value ERP). EEG-informed fMRI analysis revealed a subcortical cluster centered on the left caudate tail (CDt) as a correlate of the face value ERP. Importantly neither this analysis nor a conventional whole-brain analysis revealed any significant coding of face values in cortical areas, in particular the fusiform face area (FFA). Conversely an fMRI-informed EEG source localization using accurate subject-specific EEG head models also revealed activation in the left caudate tail. Nevertheless, the detected caudate tail region was found to be functionally connected to the FFA, suggesting FFA to be the source of face-specific information to CDt. A further psycho-physiological interaction analysis also revealed task-dependent coupling between CDt and dorsomedial prefrontal cortex (dmPFC), a region previously identified as retaining emotional working memories. These results identify CDt as a main site for encoding the long-term value memories of faces in humans suggesting that moral value of faces activates the same subcortical basal ganglia circuitry involved in processing reward value memory for objects in primates.

Generalizability and Out-of-Sample Predictive Ability of Associations Between Neuromelanin-Sensitive Magnetic Resonance Imaging and Psychosis in Antipsychotic-Free Individuals

Importance

The link between psychosis and dopaminergic dysfunction is established, but no generalizable biomarkers with clear potential for clinical adoption exist.

Objective

To replicate previous findings relating neuromelanin-sensitive magnetic resonance imaging (NM-MRI), a proxy measure of dopamine function, to psychosis severity in antipsychotic-free individuals in the psychosis spectrum and to evaluate the out-of-sample predictive ability of NM-MRI for psychosis severity.

Design, Setting, and Participants

This cross-sectional study recruited participants from 2019 to 2023 in the New York City area (main samples) and Mexico City area (external validation sample). The main samples consisted of 42 antipsychotic-free patients with schizophrenia, 53 antipsychotic-free individuals at clinical high risk for psychosis (CHR), and 52 matched healthy controls. An external validation sample consisted of 16 antipsychotic-naive patients with schizophrenia.

Main Outcomes and Measures

NM-MRI contrast within a subregion of the substantia nigra previously linked to psychosis severity (a priori psychosis region of interest [ROI]) and psychosis severity measured using the Positive and Negative Syndrome Scale (PANSS) in schizophrenia and the Structured Interview for Psychosis-Risk Syndromes (SIPS) in CHR. The cross-validated performance of linear support vector regression to predict psychosis severity across schizophrenia and CHR was assessed, and a final trained model was tested on the external validation sample.

Results

Of the 163 included participants, 76 (46.6%) were female, and the mean (SD) age was 29.2 (10.4) years. In the schizophrenia sample, higher PANSS positive total scores correlated with higher mean NM-MRI contrast in the psychosis ROI (t37 = 2.24, P = .03; partial r = 0.35; 95% CI, 0.05 to 0.55). In the CHR sample, no significant association was found between higher SIPS positive total score and NM-MRI contrast in the psychosis ROI (t48 = -0.55, P = .68; partial r = -0.08; 95% CI, -0.36 to 0.23). The 10-fold cross-validated prediction accuracy of psychosis severity was above chance in held-out test data (mean r = 0.305, P = .01; mean root-mean-square error [RMSE] = 1.001, P = .005). External validation prediction accuracy was also above chance (r = 0.422, P = .046; RMSE = 0.882, P = .047).

Conclusions and Relevance

This study provided a direct ROI-based replication of the in-sample association between NM-MRI contrast and psychosis severity in antipsychotic-free patients with schizophrenia. In turn, it failed to replicate such association in CHR individuals. Most critically, cross-validated machine-learning analyses provided a proof-of-concept demonstration that NM-MRI patterns can be used to predict psychosis severity in new data, suggesting potential for developing clinically useful tools.

Separation of Channels Subserving Approach and Avoidance/Escape at the Level of the Basal Ganglia and Related Brainstem Structures

The basal ganglia have the key function of directing our behavior in the context of events from our environment and/or our internal state. This function relies on afferents targeting the main input structures of the basal ganglia, entering bids for action selection at the level of the striatum or signals for behavioral interruption at the level of the subthalamic nucleus, with behavioral reselection facilitated by dopamine signaling. Numerous experiments have studied action selection in relation to inputs from the cerebral cortex. However, less is known about the anatomical and functional link between the basal ganglia and the brainstem. In this review, we describe how brainstem structures also project to the main input structures of the basal ganglia, namely the striatum, the subthalamic nucleus and midbrain dopaminergic neurons, in the context of approach and avoidance (including escape from threat), two fundamental, mutually exclusive behavioral choices in an animal's repertoire in which the brainstem is strongly involved. We focus on three particularly well-described loci involved in approach and avoidance, namely the superior colliculus, the parabrachial nucleus and the periaqueductal grey nucleus. We consider what is known about how these structures are related to the basal ganglia, focusing on their projections toward the striatum, dopaminergic neurons and subthalamic nucleus, and explore the functional consequences of those interactions.

Implicit Selective Attention: The Role of the Mesencephalic-basal Ganglia System

The ability of the brain to recognize and orient attention to relevant stimuli appearing in the visual field is highlighted by a tuning process, which involves modulating the early visual system by both cortical and subcortical brain areas. Selective attention is coordinated not only by the output of stimulus-based saliency maps but is also influenced by top-down cognitive factors, such as internal states, goals, or previous experiences. The basal ganglia system plays a key role in implicitly modulating the underlying mechanisms of selective attention, favouring the formation and maintenance of implicit sensory-motor memories that are capable of automatically modifying the output of priority maps in sensory-motor structures of the midbrain, such as the superior colliculus. The article presents an overview of the recent literature outlining the crucial contribution of several subcortical structures to the processing of different sources of salient stimuli. In detail, we will focus on how the mesencephalic- basal ganglia closed loops contribute to implicitly addressing and modulating selective attention to prioritized stimuli. We conclude by discussing implicit behavioural responses observed in clinical populations in which awareness is compromised at some level. Implicit (emergent) awareness in clinical conditions that can be accompanied by manifest anosognosic symptomatology (i.e., hemiplegia) or involving abnormal conscious processing of visual information (i.e., unilateral spatial neglect and blindsight) represents interesting neurocognitive "test cases" for inferences about mesencephalicbasal ganglia closed-loops involvement in the formation of implicit sensory-motor memories.

Anatomical and Functional Comparison of the Caudate Tail in Primates and the Tail of the Striatum in Rodents: Implications for Sensory Information Processing and Habitual Behavior

The tail of the striatum (TS) is located at the caudal end in the striatum. Recent studies have advanced our knowledge of the anatomy and function of the TS but also raised questions about the differences between rodent and primate TS. In this review, we compare the anatomy and function of the TS in rodent and primate brains. The primate TS is expanded more caudally during brain development in comparison with the rodent TS. Additionally, five sensory inputs from the cortex and thalamus converge in the rodent TS, but this convergence is not observed in the primate TS. The primate TS, including the caudate tail and putamen tail, primarily receives inputs from the visual areas, implying a specialized function in processing visual inputs for action generation. This anatomical difference leads to further discussion of cellular circuit models to comprehend how the primate brain processes a wider range of complex visual stimuli to produce habitual behavior as compared with the rodent brain. Examining these differences and considering possible neural models may provide better understanding of the anatomy and function of the primate TS.

Hierarchical architecture of dopaminergic circuits enables second-order conditioning in Drosophila

Dopaminergic neurons with distinct projection patterns and physiological properties compose memory subsystems in a brain. However, it is poorly understood whether or how they interact during complex learning. Here, we identify a feedforward circuit formed between dopamine subsystems and show that it is essential for second-order conditioning, an ethologically important form of higher-order associative learning. The Drosophila mushroom body comprises a series of dopaminergic compartments, each of which exhibits distinct memory dynamics. We find that a slow and stable memory compartment can serve as an effective 'teacher' by instructing other faster and transient memory compartments via a single key interneuron, which we identify by connectome analysis and neurotransmitter prediction. This excitatory interneuron acquires enhanced response to reward-predicting odor after first-order conditioning and, upon activation, evokes dopamine release in the 'student' compartments. These hierarchical connections between dopamine subsystems explain distinct properties of first- and second-order memory long known by behavioral psychologists.

The zona incerta in control of novelty seeking and investigation across species

Many organisms rely on a capacity to rapidly replicate, disperse, and evolve when faced with uncertainty and novelty. But mammals do not evolve and replicate quickly. They rely on a sophisticated nervous system to generate predictions and select responses when confronted with these challenges. An important component of their behavioral repertoire is the adaptive context-dependent seeking or avoiding of perceptually novel objects, even when their values have not yet been learned. Here, we outline recent cross-species breakthroughs that shed light on how the zona incerta (ZI), a relatively evolutionarily conserved brain area, supports novelty-seeking and novelty-related investigations. We then conjecture how the architecture of the ZI's anatomical connectivity - the wide-ranging top-down cortical inputs to the ZI, and its specifically strong outputs to both the brainstem action controllers and to brain areas involved in action value learning - place the ZI in a unique role at the intersection of cognitive control and learning.

Salience memories formed by value, novelty and aversiveness jointly shape object responses in the prefrontal cortex and basal ganglia

Ecological fitness depends on maintaining object histories to guide future interactions. Recent evidence shows that value memory changes passive visual responses to objects in ventrolateral prefrontal cortex (vlPFC) and substantia nigra reticulata (SNr). However, it is not known whether this effect is limited to reward history and if not how cross-domain representations are organized within the same or different neural populations in this corticobasal circuitry. To address this issue, visual responses of the same neurons across appetitive, aversive and novelty domains were recorded in vlPFC and SNr. Results showed that changes in visual responses across domains happened in the same rather than separate populations and were related to salience rather than valence of objects. Furthermore, while SNr preferentially encoded outcome related salience memory, vlPFC encoded salience memory across all domains in a correlated fashion, consistent with its role as an information hub to guide behavior.

The Subthalamic Nucleus Exclusively Evokes Dopamine Release in the Tail of the Striatum

A distinct population of dopamine neurons in the substantia nigra pars lateralis (SNL) has a unique projection to the most caudolateral (tail) region of the striatum. Here, using two electrochemical techniques to measure basal dopamine and electrically evoked dopamine release in anesthetized rats, we characterized this pathway, and compared it with the ‘classic’ nigrostriatal pathway from neighboring substantia nigra pars compacta (SNc) dopamine neurons to the dorsolateral striatum. We found that the tail striatum constitutes a distinct dopamine domain compared with the dorsolateral striatum, with consistently lower basal and evoked dopamine, and diverse dopamine release kinetics. Importantly, electrical stimulation of the SNL and SNc evoked dopamine release in entirely separate striatal regions; the tail and dorsolateral striatum, respectively. Furthermore, we showed that stimulation of the subthalamic nucleus (STN) evoked dopamine release exclusively in the tail striatum, likely via the SNL, consistent with previous anatomical evidence of STN afferents to SNL dopamine neurons. Our work identifies the STN as an important modulator of dopamine release in a novel dopamine pathway to the tail striatum, largely independent of the classic nigrostriatal pathway, which necessitates a revision of the basal ganglia circuitry with the STN positioned as a central integrator of striatal information.

Unbiased Stereological Estimates of Dopaminergic and GABAergic Neurons in the A10, A9, and A8 Subregions in the Young Male Macaque

The ventral midbrain is the primary source of dopamine- (DA) expressing neurons in most species. GABA-ergic and glutamatergic cell populations are intermixed among DA-expressing cells and purported to regulate both local and long-range dopamine neuron activity. Most work has been conducted in rodent models, however due to evolutionary expansion of the ventral midbrain in primates, the increased size and complexity of DA subpopulations warrants further investigation. Here, we quantified the number of DA neurons, and their GABA-ergic complement in classic DA cell groups A10 (midline ventral tegmental area nuclei [VTA] and parabrachial pigmented nucleus [PBP]), A9 (substantia nigra, pars compacta [SNc]) and A8 (retrorubral field [RRF]) in the macaque. Because the PBP is a disproportionately expanded feature of the A10 group, and has unique connectional features in monkeys, we analyzed A10 data by dividing it into 'classic' midline nuclei and the PBP. Unbiased stereology revealed total putative DA neuron counts to be 210,238 ± 17,127 (A10 = 110,319 ± 9649, A9 = 87,399 ± 7751 and A8 = 12,520 ± 827). Putative GABAergic neurons were fewer overall, and evenly dispersed across the DA subpopulations (GAD67 = 71,215 ± 5663; A10 = 16,836 ± 2743; A9 = 24,855 ± 3144 and A8 = 12,633 ± 3557). Calculating the GAD67/TH ratio for each subregion revealed differential balances of these two cell types across the DA subregions. The A8 subregion had the highest complement of GAD67-positive neurons compared to TH-positive neurons (1:1), suggesting a potentially high capacity for GABAergic inhibition of DA output in this region.

Conservation of the Direct and Indirect Pathway Dichotomy in Mouse Caudal Striatum With Uneven Distribution of Dopamine Receptor D1- and D2-Expressing Neurons

The striatum is one of the key nuclei for adequate control of voluntary behaviors and reinforcement learning. Two striatal projection neuron types, expressing either dopamine receptor D1 (D1R) or dopamine receptor D2 (D2R) constitute two independent output routes: the direct or indirect pathways, respectively. These pathways co-work in balance to achieve coordinated behavior. Two projection neuron types are equivalently intermingled in most striatal space. However, recent studies revealed two atypical zones in the caudal striatum: the zone in which D1R-neurons are the minor population (D1R-poor zone) and that in which D2R-neurons are the minority (D2R-poor zone). It remains obscure as to whether these imbalanced zones have similar properties on axonal projections and electrophysiology compared to other striatal regions. Based on morphological experiments in mice using immunofluorescence, in situ hybridization, and neural tracing, here, we revealed that the poor zones densely projected to the globus pallidus and substantia nigra pars lateralis, with a few collaterals in substantia nigra pars reticulata and compacta. Similar to that in other striatal regions, D1R-neurons were the direct pathway neurons. We also showed that the membrane properties of projection neurons in the poor zones were largely similar to those in the conventional striatum using in vitro electrophysiological recording. In addition, the poor zones existed irrespective of the age or sex of mice. We also identified the poor zones in the common marmoset as well as other rodents. These results suggest that the poor zones in the caudal striatum follow the conventional projection patterns irrespective of the imbalanced distribution of projection neurons. The poor zones could be an innate structure and common in mammals. The unique striatal zones possessing highly restricted projections could relate to functions different from those of motor-related striatum.

A primate temporal cortex-zona incerta pathway for novelty seeking

Primates interact with the world by exploring visual objects; they seek opportunities to view novel objects even when these have no extrinsic reward value. How the brain controls this novelty seeking is unknown. Here we show that novelty seeking in monkeys is regulated by the zona incerta (ZI). As monkeys made eye movements to familiar objects to trigger an opportunity to view novel objects, many ZI neurons were preferentially activated by predictions of novel objects before the gaze shift. Low-intensity ZI stimulation facilitated gaze shifts, whereas ZI inactivation reduced novelty seeking. ZI-dependent novelty seeking was not regulated by neurons in the lateral habenula or by many dopamine neurons in the substantia nigra, traditionally associated with reward seeking. But the anterior ventral medial temporal cortex, an area important for object vision and memory, was a prominent source of novelty predictions. These data uncover a functional pathway in the primate brain that regulates novelty seeking.

Brain substrates for automatic retrieval of value memory in the primate basal ganglia

Our behavior is often carried out automatically. Automatic behavior can be guided by past experiences, such as learned values associated with objects. Passive-viewing and free-viewing tasks with no immediate outcomes provide a testable condition in which monkeys and humans automatically retrieve value memories and perform habitual searching. Interestingly, in these tasks, caudal regions of the basal ganglia structures are involved in automatic retrieval of learned object values and habitual gaze. In contrast, rostral regions do not participate in these activities but instead monitor the changes in outcomes. These findings indicate that automatic behaviors based on the value memories are processed selectively by the caudal regions of the primate basal ganglia system. Understanding the distinct roles of the caudal basal ganglia may provide insight into finding selective causes of behavioral disorders in basal ganglia disease.

Sox6 expression distinguishes dorsally and ventrally biased dopamine neurons in the substantia nigra with distinctive properties and embryonic origins

Dopamine (DA) neurons in the ventral tier of the substantia nigra pars compacta (SNc) degenerate prominently in Parkinson's disease, while those in the dorsal tier are relatively spared. Defining the molecular, functional, and developmental characteristics of each SNc tier is crucial to understand their distinct susceptibility. We demonstrate that Sox6 expression distinguishes ventrally and dorsally biased DA neuron populations in the SNc. The Sox6+ population in the ventral SNc includes an Aldh1a1+ subset and is enriched in gene pathways that underpin vulnerability. Sox6+ neurons project to the dorsal striatum and show activity correlated with acceleration. Sox6- neurons project to the medial, ventral, and caudal striatum and respond to rewards. Moreover, we show that this adult division is encoded early in development. Overall, our work demonstrates a dual origin of the SNc that results in DA neuron cohorts with distinct molecular profiles, projections, and functions.

Throwing open the doors of perception: The role of dopamine in visual processing

Animals form associations between visual cues and behaviours. Although dopamine is known to be critical in many areas of the brain to bind sensory information with appropriate responses, dopamine's role in the visual system is less well understood. Visual signals, which indicate the likely occurrence of a rewarding or aversive stimulus or indicate the context within which such stimuli may arrive, modulate activity in the superior colliculus and alter behaviour. However, such signals primarily originate in cortical and basal ganglia circuits, and evidence of direct signalling from midbrain dopamine neurons to superior colliculus is lacking. Instead, hypothalamic A13 dopamine neurons innervate the superior colliculus, and dopamine receptors are differentially expressed in the superior colliculus, with D1 receptors in superficial layers and D2 receptors in deep layers. However, it remains unknown if A13 dopamine neurons control behaviours through their effect on afferents within the superior colliculus. We propose that A13 dopamine neurons may play a critical role in processing information in the superior colliculus, modifying behavioural responses to visual cues, and propose some testable hypotheses regarding dopamine's effect on visual perception.

FMRI hemodynamic response function (HRF) as a novel marker of brain function: applications for understanding obsessive-compulsive disorder pathology and treatment response

The hemodynamic response function (HRF) represents the transfer function linking neural activity with the functional MRI (fMRI) signal, modeling neurovascular coupling. Since HRF is influenced by non-neural factors, to date it has largely been considered as a confound or has been ignored in many analyses. However, underlying biophysics suggests that the HRF may contain meaningful correlates of neural activity, which might be unavailable through conventional fMRI metrics. Here, we estimated the HRF by performing deconvolution on resting-state fMRI data from a longitudinal sample of 25 healthy controls scanned twice and 44 adults with obsessive-compulsive disorder (OCD) before and after 4-weeks of intensive cognitive-behavioral therapy (CBT). HRF response height, time-to-peak and full-width at half-maximum (FWHM) in OCD were abnormal before treatment and normalized after treatment in regions including the caudate. Pre-treatment HRF predicted treatment outcome (OCD symptom reduction) with 86.4% accuracy, using machine learning. Pre-treatment HRF response height in the caudate head and time-to-peak in the caudate tail were top-predictors of treatment response. Time-to-peak in the caudate tail, a region not typically identified in OCD studies using conventional fMRI activation or connectivity measures, may carry novel importance. Additionally, pre-treatment response height in caudate head predicted post-treatment OCD severity (R = -0.48, P = 0.001), and was associated with treatment-related OCD severity changes (R = -0.44, P = 0.0028), underscoring its relevance. With HRF being a reliable marker sensitive to brain function, OCD pathology, and intervention-related changes, these results could guide future studies towards novel discoveries not possible through conventional fMRI approaches like standard BOLD activation or connectivity.

Reductions in midbrain GABAergic and dopamine neuron markers are linked in schizophrenia

Reductions in the GABAergic neurotransmitter system exist across multiple brain regions in schizophrenia and encompass both pre- and postsynaptic components. While reduced midbrain GABAergic inhibitory neurotransmission may contribute to the hyperdopaminergia thought to underpin psychosis in schizophrenia, molecular changes consistent with this have not been reported. We hypothesised that reduced GABA-related molecular markers would be found in the midbrain of people with schizophrenia and that these would correlate with dopaminergic molecular changes. We hypothesised that downregulation of inhibitory neuron markers would be exacerbated in schizophrenia cases with high levels of neuroinflammation. Eight GABAergic-related transcripts were measured with quantitative PCR, and glutamate decarboxylase (GAD) 65/67 and GABA_A alpha 3 (α3) (GABRA3) protein were measured with immunoblotting, in post-mortem midbrain (28/28 and 28/26 control/schizophrenia cases for mRNA and protein, respectively), and analysed by both diagnosis and inflammatory subgroups (as previously defined by higher levels of four pro-inflammatory cytokine transcripts). We found reductions (21 – 44%) in mRNA encoding both presynaptic and postsynaptic proteins, vesicular GABA transporter (VGAT), GAD1, and parvalbumin (PV) mRNAs and four alpha subunits (α1, α2, α3, α5) of the GABA_A receptor in people with schizophrenia compared to controls (p < 0.05). Gene expression of somatostatin (SST) was unchanged (p = 0.485). We confirmed the reduction in GAD at the protein level (34%, p < 0.05). When stratifying by inflammation, only GABRA3 mRNA exhibited more pronounced changes in high compared to low inflammatory subgroups in schizophrenia. GABRA3 protein was expressed by 98% of tyrosine hydroxylase-positive neurons and was 23% lower in schizophrenia, though this did not reach statistical significance (p > 0.05). Expression of transcripts for GABA_A receptor alpha subunits 2 and 3 (GABRA2, GABRA3) were positively correlated with tyrosine hydroxylase (TH) and dopamine transporter (DAT) transcripts in schizophrenia cases (GABRA2; r > 0.630, GABRA3; r > 0.762, all p < 0.001) but not controls (GABRA2; r < − 0.200, GABRA3; r < 0.310, all p > 0.05). Taken together, our results support a profound disruption to inhibitory neurotransmission in the substantia nigra regardless of inflammatory status, which provides a potential mechanism for disinhibition of nigrostriatal dopamine neurotransmission.

Common coding of expected value and value uncertainty memories in the prefrontal cortex and basal ganglia output

Recent evidence implicates both basal ganglia and ventrolateral prefrontal cortex (vlPFC) in encoding value memories. However, comparative roles of cortical and basal nodes in value memory are not well understood. Here, single-unit recordings in vlPFC and substantia nigra reticulata (SNr), within macaque monkeys, revealed a larger value signal in SNr that was nevertheless correlated with and had a comparable onset to the vlPFC value signal. The value signal was maintained for many objects (>90) many weeks after reward learning and was resistant to extinction in both regions and to repetition suppression in vlPFC. Both regions showed comparable granularity in encoding expected value and value uncertainty, which was paralleled by enhanced gaze bias during free viewing. The value signal dynamics in SNr could be predicted by combining responses of vlPFC neurons according to their value preferences consistent with a scheme in which cortical neurons reached SNr via direct and indirect pathways.

Primate ventral striatum maintains neural representations of the value of previously rewarded objects for habitual seeking

The ventral striatum (VS) is considered a key region that flexibly updates recent changes in reward values for habit learning. However, this update process may not serve to maintain learned habitual behaviors, which are insensitive to value changes. Here, using fMRI in humans and single-unit electrophysiology in macaque monkeys we report another role of the primate VS: that the value memory subserving habitual seeking is stably maintained in the VS. Days after object-value associative learning, human and monkey VS continue to show increased responses to previously rewarded objects, even when no immediate reward outcomes are expected. The similarity of neural response patterns to each rewarded object increases after learning among participants who display habitual seeking. Our data show that long-term memory of high-valued objects is retained as a single representation in the VS and may be utilized to evaluate visual stimuli automatically to guide habitual behavior.

Environment-based object values learned by local network in the striatum tail

Choosing good objects is a fundamental behavior for all animals, to which the basal ganglia (BG) contribute extensively. However, the object choice needs to be changed in different environments. The mechanism of object choice is based on the neuronal circuits originating from output neurons (MSNs) in the striatum. We found that the environment information is provided by fast-spiking interneurons (FSIs) connecting to the MSN circuit. More critically, the experimental reduction of the FSI-input to MSNs disabled the monkey to learn the environment-based object choice. This proved that the object choice controlled by the downstream BG circuit is modulated by the environmental context controlled by the internal circuits in the top of BG circuit. This is important for our flexible decision.

Basal ganglia contribute to object-value learning, which is critical for survival. The underlying neuronal mechanism is the association of each object with its rewarding outcome. However, object values may change in different environments and we then need to choose different objects accordingly. The mechanism of this environment-based value learning is unknown. To address this question, we created an environment-based value task in which the value of each object was reversed depending on the two scene-environments (X and Y). After experiencing this task repeatedly, the monkeys became able to switch the choice of object when the scene-environment changed unexpectedly. When we blocked the inhibitory input from fast-spiking interneurons (FSIs) to medium spiny projection neurons (MSNs) in the striatum tail by locally injecting IEM-1460, the monkeys became unable to learn scene-selective object values. We then studied the mechanism of the FSI-MSN connection. Before and during this learning, FSIs responded to the scenes selectively, but were insensitive to object values. In contrast, MSNs became able to discriminate the objects (i.e., stronger response to good objects), but this occurred clearly in one of the two scenes (X or Y). This was caused by the scene-selective inhibition by FSI. As a whole, MSNs were divided into two groups that were sensitive to object values in scene X or in scene Y. These data indicate that the local network of striatum tail controls the learning of object values that are selective to the scene-environment. This mechanism may support our flexible switching behavior in various environments.

The Tail of the Striatum: From Anatomy to Connectivity and Function

The dorsal striatum, the largest subcortical structure of the basal ganglia, is critical in controlling motor, procedural, and reinforcement-based behaviors. Although in mammals the striatum extends widely along the rostro-caudal axis, current knowledge and derived theories about its anatomo-functional organization largely rely on results obtained from studies of its rostral sectors, leading to potentially oversimplified working models of the striatum as a whole. Recent findings indicate that the extreme caudal part of the striatum, also referred to as the tail of striatum (TS), represents an additional functional domain. Here, we provide an overview of past and recent studies revealing that the TS displays a heterogeneous cell-type-specific organization, and a unique input-output connectivity, which poises the TS as an integrator of sensory processing.

How Outcome Uncertainty Mediates Attention, Learning, and Decision-Making

Animals and humans evolved sophisticated nervous systems that endowed them with the ability to form internal-models or beliefs and make predictions about the future to survive and flourish in a world in which future outcomes are often uncertain. Crucial to this capacity is the ability to adjust behavioral and learning policies in response to the level of uncertainty. Until recently, the neuronal mechanisms that could underlie such uncertainty-guided control have been largely unknown. In this review, I discuss newly discovered neuronal circuits in primates that represent uncertainty about future rewards and propose how they guide information-seeking, attention, decision-making, and learning to help us survive in an uncertain world. Lastly, I discuss the possible relevance of these findings to learning in artificial systems.

Topographic distinction in long-term value signals between presumed dopamine neurons and presumed striatal projection neurons in behaving monkeys

Nigrostriatal dopamine (DA) projections are anatomically organized along the dorsolateral-ventromedial axis, conveying long-term value signals to the striatum for shaping actions toward multiple future rewards. The present study examines whether the topographic organization of long-term value signals are observed upon activity of presumed DA neurons and presumed striatal projection neurons (phasically active neurons, PANs), as predicted based on anatomical literature. Our results indicate that DA neurons in the dorsolateral midbrain encode long-term value signals on a short timescale, while ventromedial midbrain DA neurons encode such signals on a relatively longer timescale. Activity of the PANs in the dorsal striatum is more heterogeneous for encoding long-term values, although significant differences in long-term value signals were observed between the caudate nucleus and putamen. These findings suggest that topographic DA signals for long-term values are not simply transferred to striatal neurons, possibly due to the contribution of other projections to the striatum.

Heterogeneity in striatal dopamine circuits: Form and function in dynamic reward seeking

The striatal dopamine system has long been studied in the context of reward learning, motivation, and movement. Given the prominent role dopamine plays in a variety of adaptive behavioral states, as well as diseases like addiction, it is essential to understand the full complexity of dopamine neurons and the striatal systems they target. A growing number of studies are uncovering details of the heterogeneity in dopamine neuron subpopulations. Here, we review that work to synthesize current understanding of dopamine system heterogeneity across three levels, anatomical organization, functions in behavior, and modes of action, wherein we focus on signaling profiles and local mechanisms for modulation of dopamine release. Together, these studies reveal new and emerging dimensions of the striatal dopamine system, informing its contribution to dynamic motivational and decision-making processes.

Multiple neuronal circuits for variable object-action choices based on short- and long-term memories

At each time in our life, we choose one or few behaviors, while suppressing many other behaviors. This is the basic mechanism in the basal ganglia, which is done by tonic inhibition and selective disinhibition. Dysfunctions of the basal ganglia then cause 2 types of disorders (difficulty in initiating necessary actions and difficulty in suppressing unnecessary actions) that occur in Parkinson's disease. The basal ganglia generate such opposite outcomes through parallel circuits: The direct pathway for initiation and indirect pathway for suppression. Importantly, the direct pathway processes good information and the indirect pathway processes bad information, which enables the choice of good behavior and the rejection of bad behavior. This is mainly enabled by dopaminergic inputs to these circuits. However, the value judgment is complex because the world is complex. Sometimes, the value must be based on recent events, thus is based on short-term memories. Or, the value must be based on historical events, thus is based on long-term memories. Such memory-based value judgment is generated by another parallel circuit originating from the caudate head and caudate tail. These circuit-information mechanisms allow other brain areas (e.g., prefrontal cortex) to contribute to decisions by sending information to these basal ganglia circuits. Moreover, the basal ganglia mechanisms (i.e., what to choose) are associated with cerebellum mechanisms (i.e., when to choose). Overall, multiple levels of parallel circuits in and around the basal ganglia are essential for coordinated behaviors. Understanding these circuits is useful for creating clinical treatments of disorders resulting from the failure of these circuits.

Neuronal connections of direct and indirect pathways for stable value memory in caudal basal ganglia

Direct and indirect pathways in the basal ganglia work together for controlling behavior. However, it is still a controversial topic whether these pathways are segregated or merged with each other. To address this issue, we studied the connections of these two pathways in the caudal parts of the basal ganglia of rhesus monkeys using anatomical tracers. Our previous studies showed that the caudal basal ganglia control saccades by conveying long-term values (stable values) of many visual objects toward the superior colliculus. In experiment 1, we injected a tracer in the caudate tail (CDt), and found local dense plexuses of axon terminals in the caudal-dorsal-lateral part of substantia nigra pars reticulata (cdlSNr) and the caudal-ventral part of globus pallidus externus (cvGPe). These anterograde projections may correspond to the direct and indirect pathways, respectively. To verify this in experiment 2, we injected different tracers into cdlSNr and cvGPe, and found many retrogradely labeled neurons in CDt and, in addition, the caudal-ventral part of the putamen (cvPut). These cdlSNr-projecting and cvGPe-projecting neurons were found intermingled in both CDt and cvPut (which we call "striatum tail"). A small but significant proportion of neurons (<15%) were double-labeled, indicating that they projected to both cdlSNr and cvGPe. These anatomical results suggest that stable value signals (good vs. bad) are sent from the striatum tail to cdlSNr and cvGPe in a biased (but not exclusive) manner. These connections may play an important role in biasing saccades toward higher valued objects and away from lower valued objects.

Prefrontal Cortex Represents Long-Term Memory of Object Values for Months

As a central hub for cognitive control, prefrontal cortex (PFC) is thought to utilize memories. However, unlike working or short-term memory, the neuronal representation of long-term memory in PFC has not been systematically investigated. Using single-unit recordings in macaques, we show that PFC neurons rapidly update and maintain responses to objects based on short-term reward history. Interestingly, after repeated object-reward association, PFC neurons continue to show value-biased responses to objects even in the absence of reward. This value-biased response is retained for several months after training and is resistant to extinction and to interference from new object-reward learning for many complex objects (>90). Accordingly, the monkeys remember the values of the learned objects for several months in separate testing. These findings reveal that in addition to flexible short-term and low-capacity memories, primate PFC represents stable long-term and high-capacity memories, which could prioritize valuable objects far into the future.

Anatomical Inputs From the Sensory and Value Structures to the Tail of the Rat Striatum

The caudal region of the rodent striatum, called the tail of the striatum (TS), is a relatively small area but might have a distinct function from other striatal subregions. Recent primate studies showed that this part of the striatum has a unique function in encoding long-term value memory of visual objects for habitual behavior. This function might be due to its specific connectivity. We identified inputs to the rat TS and compared those with inputs to the dorsomedial striatum (DMS) in the same animals. The TS directly received anatomical inputs from both sensory structures and value-coding regions, but the DMS did not. First, inputs from the sensory cortex and sensory thalamus to the TS were found; visual, auditory, somatosensory and gustatory cortex and thalamus projected to the TS but not to the DMS. Second, two value systems innervated the TS; dopamine and serotonin neurons in the lateral part of the substantia nigra pars compacta (SNc) and dorsal raphe nucleus projected to the TS, respectively. The DMS received inputs from the separate group of dopamine neurons in the medial part of the SNc. In addition, learning-related regions of the limbic system innervated the TS; the temporal areas and the basolateral amygdala selectively innervated the TS, but not the DMS. Our data showed that both sensory and value-processing structures innervated the TS, suggesting its plausible role in value-guided sensory-motor association for habitual behavior.

Direct and indirect pathways for choosing objects and actions

A prominent target of the basal ganglia is the superior colliculus (SC) which controls gaze orientation (saccadic eye movement in primates) to an important object. This 'object choice' is crucial for choosing an action on the object. SC is innervated by the substantia nigra pars reticulata (SNr) which is controlled mainly by the caudate nucleus (CD). This CD-SNr-SC circuit is sensitive to the values of individual objects and facilitates saccades to good objects. The object values are processed differently in two parallel circuits: flexibly by the caudate head (CDh) and stably by the caudate tail (CDt). To choose good objects, we need to reject bad objects. In fact, these contrasting functions are accomplished by the circuit originating from CDt: The direct pathway focuses on good objects and facilitates saccades to them; the indirect pathway focuses on bad objects and suppresses saccades to them. Inactivation of CDt deteriorated the object choice, because saccades to bad objects were no longer suppressed. This suggests that the indirect pathway is important for object choice. However, the direct and indirect pathways for 'object choice', which aim at the same action (i.e., saccade), may not work for 'action choice'. One possibility is that circuits controlling different actions are connected through the indirect pathway. Additional connections of the indirect pathway with brain areas outside the basal ganglia may also provide a wider range of behavioral choice. In conclusion, basal ganglia circuits are composed of the basic direct/indirect pathways and additional connections and thus have acquired multiple functions.

Flexible and Stable Value Coding Areas in Caudate Head and Tail Receive Anatomically Distinct Cortical and Subcortical Inputs

Anatomically distinct areas within the basal ganglia encode flexible- and stable-value memories for visual objects (Hikosaka et al., 2014), but an important question remains: do they receive inputs from the same or different brain areas or neurons? To answer this question, we first located flexible and stable value-coding areas in the caudate head (CDh) and caudate tail (CDt) of two rhesus macaque monkeys, and then injected different retrograde tracers into these areas of each monkey. We found that CDh and CDt received different inputs from several cortical and subcortical areas including temporal cortex, prefrontal cortex, cingulate cortex, amygdala, claustrum and thalamus. Superior temporal cortex and inferior temporal cortex projected to both CDh and CDt, with more CDt-projecting than CDh-projecting neurons. In superior temporal cortex and dorsal inferior temporal cortex, layers 3 and 5 projected to CDh while layers 3 and 6 projected to CDt. Prefrontal and cingulate cortex projected mostly to CDh bilaterally, less to CDt unilaterally. A cluster of neurons in the basolateral amygdala projected to CDt. Rostral-dorsal claustrum projected to CDh while caudal-ventral claustrum projected to CDt. Within the thalamus, different nuclei projected to either CDh or CDt. The medial centromedian nucleus and lateral parafascicular nucleus projected to CDt while the medial parafascicular nucleus projected to CDh. The inferior pulvinar and lateral dorsal nuclei projected to CDt. The ventral anterior and medial dorsal nuclei projected to CDh. We found little evidence of neurons projecting to both CDh and CDt across the brain. These data suggest that CDh and CDt can control separate functions using anatomically separate circuits. Understanding the roles of these striatal projections will be important for understanding how value memories are created and stored.

The many worlds hypothesis of dopamine prediction error: implications of a parallel circuit architecture in the basal ganglia

Computational models of reinforcement learning (RL) strive to produce behavior that maximises reward, and thus allow software or robots to behave adaptively [1]. At the core of RL models is a learned mapping between 'states'-situations or contexts that an agent might encounter in the world-and actions. A wealth of physiological and anatomical data suggests that the basal ganglia (BG) is important for learning these mappings [2,3]. However, the computations performed by specific circuits are unclear. In this brief review, we highlight recent work concerning the anatomy and physiology of BG circuits that suggest refinements in our understanding of computations performed by the basal ganglia. We focus on one important component of basal ganglia circuitry, midbrain dopamine neurons, drawing attention to data that has been cast as supporting or departing from the RL framework that has inspired experiments in basal ganglia research over the past two decades. We suggest that the parallel circuit architecture of the BG might be expected to produce variability in the response properties of different dopamine neurons, and that variability in response profile may not reflect variable functions, but rather different arguments that serve as inputs to a common function: the computation of prediction error.

Indirect Pathway of Caudal Basal Ganglia for Rejection of Valueless Visual Objects

The striatum controls behavior in two ways: facilitation and suppression through the direct and indirect pathways, respectively. However, it is still unclear what information is processed in these pathways. To address this question, we studied two pathways originating from the primate caudate tail (CDt). We found that the CDt innervated the caudal-dorsal-lateral part of the substantia nigra pars reticulata (cdlSNr), directly or indirectly through the caudal-ventral part of the globus pallidus externus (cvGPe). Notably, cvGPe neurons receiving inputs from the CDt were mostly visual neurons that encoded stable reward values of visual objects based on long-past experiences. Their dominant response was inhibition by valueless objects, which generated disinhibition of cdlSNr neurons and inhibition of superior colliculus neurons. Our data suggest that low-value signals are sent by the CDt-indirect pathway to suppress saccades to valueless objects, whereas high-value signals are sent by the CDt-direct pathway to facilitate saccades to valuable objects.

Vision and Action

Investigation of natural behavior has contributed a number of insights to our understanding of visual guidance of actions by highlighting the importance of behavioral goals and focusing attention on how vision and action play out in time. In this context, humans make continuous sequences of sensory-motor decisions to satisfy current behavioral goals, and the role of vision is to provide the relevant information for making good decisions in order to achieve those goals. This conceptualization of visually guided actions as a sequence of sensory-motor decisions has been formalized within the framework of statistical decision theory, which structures the problem and provides the context for much recent progress in vision and action. Components of a good decision include the task, which defines the behavioral goals, the rewards and costs associated with those goals, uncertainty about the state of the world, and prior knowledge.

Parallel basal ganglia circuits for decision making

The basal ganglia control body movements, mainly, based on their values. Critical for this mechanism is dopamine neurons, which sends unpredicted value signals, mainly, to the striatum. This mechanism enables animals to change their behaviors flexibly, eventually choosing a valuable behavior. However, this may not be the best behavior, because the flexible choice is focused on recent, and, therefore, limited, experiences (i.e., short-term memories). Our old and recent studies suggest that the basal ganglia contain separate circuits that process value signals in a completely different manner. They are insensitive to recent changes in value, yet gradually accumulate the value of each behavior (i.e., movement or object choice). These stable circuits eventually encode values of many behaviors and then retain the value signals for a long time (i.e., long-term memories). They are innervated by a separate group of dopamine neurons that retain value signals, even when no reward is predicted. Importantly, the stable circuits can control motor behaviors (e.g., hand or eye) quickly and precisely, which allows animals to automatically acquire valuable outcomes based on historical life experiences. These behaviors would be called 'skills', which are crucial for survival. The stable circuits are localized in the posterior part of the basal ganglia, separately from the flexible circuits located in the anterior part. To summarize, the flexible and stable circuits in the basal ganglia, working together but independently, enable animals (and humans) to reach valuable goals in various contexts.

Using a novel PV-Cre rat model to characterize pallidonigral cells and their terminations

In the present study, we generated a novel parvalbumin (PV)-Cre rat model and conducted detailed morphological and electrophysiological investigations of axons from PV neurons in globus pallidus (GP). The GP is considered as a relay nucleus in the indirect pathway of the basal ganglia (BG). Previous studies have used molecular profiling and projection patterns to demonstrate cellular heterogeneity in the GP; for example, PV-expressing neurons are known to comprise approximately 50% of GP neurons and represent majority of prototypic neurons that project to the subthalamic nucleus and/or output nuclei of BG, entopeduncular nucleus and substantia nigra (SN). The present study aimed to identify the characteristic projection patterns of PV neurons in the GP (PV-GP neurons) and determine whether these neurons target dopaminergic or GABAergic neurons in SN pars compacta (SNc) or reticulata (SNr), respectively. We initially found that (1) 57% of PV neurons co-expressed Lim-homeobox 6, (2) the PV-GP terminals were preferentially distributed in the ventral part of dorsal tier of SNc, (3) PV-GP neurons formed basket-like appositions with the somata of tyrosine hydroxylase, PV, calretinin and cholecystokinin immunoreactive neurons in the SN, and (4) in vitro whole-cell recording during optogenetic photo-stimulation of PV-GP terminals in SNc demonstrated that PV-GP neurons strongly inhibited dopamine neurons via GABA_A receptors. These results suggest that dopamine neurons receive direct focal inputs from PV-GP prototypic neurons. The identification of high-contrast inhibitory systems on dopamine neurons might represent a key step toward understanding the BG function.

Ecological Origins of Object Salience: Reward, Uncertainty, Aversiveness, and Novelty

Among many objects around us, some are more salient than others (i.e., attract our attention automatically). Some objects may be inherently salient (e.g., brighter), while others may become salient by virtue of their ecological relevance through experience. However, the role of ecological experience in automatic attention has not been studied systematically. To address this question, we let subjects (macaque monkeys) view a large number of complex objects (>300), each experienced repeatedly (>5 days) with rewarding, aversive or no outcome association (mere-perceptual exposure). Test of salience was done on separate days using free viewing with no outcome. We found that gaze was biased among the objects from the outset, affecting saccades to objects or fixations within objects. When the outcome was rewarding, gaze preference was stronger (i.e., positive) for objects with larger or equal but uncertain rewards. The effects of aversive outcomes were variable. Gaze preference was positive for some outcome associations (e.g., airpuff), but negative for others (e.g., time-out), possibly due to differences in threat levels. Finally, novel objects attracted gaze, but mere perceptual exposure of objects reduced their salience (learned negative salience). Our results show that, in primates, object salience is strongly influenced by previous ecological experience and is supported by a large memory capacity. Owing to such high capacity for learned salience, the ability to rapidly choose important objects can grow during the entire life to promote biological fitness.

Object-finding skill created by repeated reward experience

For most animals, survival depends on rapid detection of rewarding objects, but search for an object surrounded by many others is known to be difficult and time consuming. However, there is neuronal evidence for robust and rapid differentiation of objects based on their reward history in primates (Hikosaka, Kim, Yasuda, & Yamamoto, 2014). We hypothesized that such robust coding should support efficient search for high-value objects, similar to a pop-out mechanism. To test this hypothesis, we let subjects (n = 4, macaque monkeys) view a large number of complex objects with consistently biased rewards with variable training durations (1, 5, or 30 + days). Following training, subjects searched for a high-value object (Good) among a variable number of low-value objects (Bad). Consistent with our hypothesis, we found that Good objects were accurately and quickly targeted, often by a single and direct saccade with a very short latency (<200 ms). The dependence of search times on display size reduced significantly with longer reward training, giving rise to a more efficient search (40 ms/item to 16 ms/item). This object-finding skill showed a large capacity for value-biased objects and was maintained in the long-term memory with no interference from reward learning with other objects. Such object-finding skill, and in particular its large capacity and long term retention, would be crucial for maximizing rewards and biological fitness throughout life where many objects are experienced continuously and/or intermittently.

Dopamine Neurons Encoding Long-Term Memory of Object Value for Habitual Behavior

Dopamine neurons promote learning by processing recent changes in reward values, such that reward may be maximized. However, such a flexible signal is not suitable for habitual behaviors that are sustained regardless of recent changes in reward outcome. We discovered a type of dopamine neuron in the monkey substantia nigra pars compacta (SNc) that retains past learned reward values stably. After reward values of visual objects are learned, these neurons continue to respond differentially to the objects, even when reward is not expected. Responses are strengthened by repeated learning and are evoked upon presentation of the objects long after learning is completed. These "sustain-type" dopamine neurons are confined to the caudal-lateral SNc and project to the caudate tail, which encodes long-term value memories of visual objects and guides gaze automatically to stably valued objects. This population of dopamine neurons thus selectively promotes learning and retention of habitual behavior.

Communication in Neural Circuits: Tools, Opportunities, and Challenges

Communication, the effective delivery of information, is fundamental to life across all scales and species. Nervous systems (by necessity) may be most specifically adapted among biological tissues for high rate and complexity of information transmitted, and thus, the properties of neural tissue and principles of its organization into circuits may illuminate capabilities and limitations of biological communication. Here, we consider recent developments in tools for studying neural circuits with particular attention to defining neuronal cell types by input and output information streams--i.e., by how they communicate. Complementing approaches that define cell types by virtue of genetic promoter/enhancer properties, this communication-based approach to defining cell types operationally by input/output (I/O) relationships links structure and function, resolves difficulties associated with single-genetic-feature definitions, leverages technology for observing and testing significance of precisely these I/O relationships in intact brains, and maps onto processes through which behavior may be adapted during development, experience, and evolution.

Reward signal in a recurrent circuit drives appetitive long-term memory formation

Dopamine signals reward in animal brains. A single presentation of a sugar reward to Drosophila activates distinct subsets of dopamine neurons that independently induce short- and long-term olfactory memories (STM and LTM, respectively). In this study, we show that a recurrent reward circuit underlies the formation and consolidation of LTM. This feedback circuit is composed of a single class of reward-signaling dopamine neurons (PAM-α1) projecting to a restricted region of the mushroom body (MB), and a specific MB output cell type, MBON-α1, whose dendrites arborize that same MB compartment. Both MBON-α1 and PAM-α1 neurons are required during the acquisition and consolidation of appetitive LTM. MBON-α1 additionally mediates the retrieval of LTM, which is dependent on the dopamine receptor signaling in the MB α/β neurons. Our results suggest that a reward signal transforms a nascent memory trace into a stable LTM using a feedback circuit at the cost of memory specificity.

DOI:http://dx.doi.org/10.7554/eLife.10719.001

An animal that finds particularly nutritious and palatable food will often develop a long-lasting memory—even if they experience that event only once.

One example of this is the ability of the fruit fly Drosophila to form a long-term association between a sugar reward and a specific odor that was present when they received the reward. The consumption of sugar triggers the release of a chemical called dopamine on specific compartments of a brain structure called the mushroom body. Dopamine then acts to modify the connection between cells called “Kenyon cells”, which encode specific odors, and the neurons that send signals out from the mushroom body (called MBONs). The result is the formation of a memory that links the odor with the reward.

However, little is known about how this process differs for long-term vs. short-term memories, and how it can occur when the fly has experienced the odor and reward together on only a single occasion. To find out, Ichinose et al. combined behavioral testing of fruit flies with genetics. The results confirmed that the dopamine neurons and the MBONs that project to a single compartment of the mushroom body, called α1, are both required for the formation of long-term odor-reward memories, but not their short-term equivalents. These neurons are called PAM-α1 and MBON-α1, respectively.

Unexpectedly, anatomical data revealed that PAM-α1 dopamine neurons receive input from MBON-α1; that is, long-term memory formation involves a feedback circuit: from PAM-α1 to Kenyon cells, then to MBON-α1 and back to PAM-α1. Blocking feedback from the MBON-α1 onto the PAM-α1 neurons shortly after odor-reward training disrupted long-term memory formation. Conversely, blocking feedback at a later stage did not. This suggests that prolonged activation of PAM-α1 by MBON-α1 helps to strengthen newly established memories, converting them into memories that will last for a long time.

The discovery of a specific circuit that supports long-term, but not short-term, memory formation in fruit flies is consistent with evidence of distinct mechanisms underlying these processes in mammals. Further work is now required to determine whether feedback circuits similar to those in fruit flies also contribute to reward-based learning in other animals.

DOI:http://dx.doi.org/10.7554/eLife.10719.002

Dopamine neurons projecting to the posterior striatum form an anatomically distinct subclass

Combining rabies-virus tracing, optical clearing (CLARITY), and whole-brain light-sheet imaging, we mapped the monosynaptic inputs to midbrain dopamine neurons projecting to different targets (different parts of the striatum, cortex, amygdala, etc) in mice. We found that most populations of dopamine neurons receive a similar set of inputs rather than forming strong reciprocal connections with their target areas. A common feature among most populations of dopamine neurons was the existence of dense ‘clusters’ of inputs within the ventral striatum. However, we found that dopamine neurons projecting to the posterior striatum were outliers, receiving relatively few inputs from the ventral striatum and instead receiving more inputs from the globus pallidus, subthalamic nucleus, and zona incerta. These results lay a foundation for understanding the input/output structure of the midbrain dopamine circuit and demonstrate that dopamine neurons projecting to the posterior striatum constitute a unique class of dopamine neurons regulated by different inputs.

DOI:http://dx.doi.org/10.7554/eLife.10032.001

Most neurons send their messages to recipient neurons by releasing a substance called a ‘neurotransmitter’ that binds to receptors on the target cell. The sites of this type of signal transmission are called synapses. Some small populations of neurons modulate the activity of hundreds or thousands of these synapses all across the brain by releasing ‘neuromodulators’ that affect how they work. These neuromodulators are essential because they broadcast information that is likely to be useful to many brain regions, like a ‘news channel’ for the brain.

One important neuromodulator in the mammalian brain is dopamine, which contributes to motivation, learning, and the control of movement. Clusters of cells deep in the brain release dopamine, and people with Parkinson's disease gradually lose these cells. This makes it increasingly difficult for their brains to produce the correct amount of dopamine, and results in symptoms such as tremors and stiff muscles.

Individual dopamine neurons typically send information to a single part of the brain. This suggests that dopamine neurons with different targets might have different roles. To explore this possibility, Menegas et al. classified dopamine neurons in the mouse brain into eight types based on the areas to which they project, and then mapped which neurons send input signals to each type. These inputs are likely to shape the activity of each type (that is, their ‘message’ to the rest of the brain). The mapping revealed that most dopamine neurons do not receive substantial input from the area to which they project (i.e., they do not form ‘closed loops’). Instead, most of their input comes from a common set of brain regions, including a particularly large number of inputs from the ventral striatum.

However, Menegas et al. found one exception. Dopamine neurons that target part of the brain called the posterior striatum receive relatively little input from the ventral striatum. Their input comes instead from a set of other brain structures, and in particular from a region called the subthalamic nucleus. Electrical stimulation of the subthalamic nucleus can help to relieve the symptoms of Parkinson's disease. Therefore, the results presented by Menegas et al. suggest that this population of dopamine neurons might be particularly relevant to Parkinson's disease and that focusing future studies on them could ultimately be beneficial for patients.

DOI:http://dx.doi.org/10.7554/eLife.10032.002

Hippocampus and subregions of the dorsal striatum respond differently to a behavioral strategy change on a spatial navigation task

Goal-directed and habit-based behaviors are driven by multiple but dissociable decision making systems involving several different brain areas, including the hippocampus and dorsal striatum. On repetitive tasks, behavior transitions from goal directed to habit based with experience. Hippocampus has been implicated in initial learning and dorsal striatum in automating behavior, but recent studies suggest that subregions within the dorsal striatum have distinct roles in mediating habit-based and goal-directed behavior. We compared neural activity in the CA1 region of hippocampus with anterior dorsolateral and posterior dorsomedial striatum in rats on a spatial choice task, in which subjects experienced reward delivery changes that forced them to adjust their behavioral strategy. Our results confirm the importance of the hippocampus in evaluating predictive steps during goal-directed behavior, while separate circuits in the basal ganglia integrated relevant information during automation of actions and recognized when new behaviors were needed to continue obtaining rewards.

Parallel basal ganglia circuits for voluntary and automatic behaviour to reach rewards

The basal ganglia control body movements, value processing and decision-making. Many studies have shown that the inputs and outputs of each basal ganglia structure are topographically organized, which suggests that the basal ganglia consist of separate circuits that serve distinct functions. A notable example is the circuits that originate from the rostral (head) and caudal (tail) regions of the caudate nucleus, both of which target the superior colliculus. These two caudate regions encode the reward values of visual objects differently: flexible (short-term) values by the caudate head and stable (long-term) values by the caudate tail. These value signals in the caudate guide the orienting of gaze differently: voluntary saccades by the caudate head circuit and automatic saccades by the caudate tail circuit. Moreover, separate groups of dopamine neurons innervate the caudate head and tail and may selectively guide the flexible and stable learning/memory in the caudate regions. Studies focusing on manual handling of objects also suggest that rostrocaudally separated circuits in the basal ganglia control the action differently. These results suggest that the basal ganglia contain parallel circuits for two steps of goal-directed behaviour: finding valuable objects and manipulating the valuable objects. These parallel circuits may underlie voluntary behaviour and automatic skills, enabling animals (including humans) to adapt to both volatile and stable environments. This understanding of the functions and mechanisms of the basal ganglia parallel circuits may inform the differential diagnosis and treatment of basal ganglia disorders.

Differentiating neural systems mediating the acquisition vs. expression of goal-directed and habitual behavioral control

Considerable behavioral data indicate that operant actions can become habitual, as demonstrated by insensitivity to changes in the action-outcome contingency and in subjective outcome values. Notably, although several studies have investigated the neural substrates of habits, none has clearly differentiated the areas of the human brain that support habit formation from those that implement habitual control. We scanned participants with functional magnetic resonance imaging as they learned and performed an operant task in which the conditional structure of the environment encouraged either goal-directed encoding of the consequences of actions, or a habit-like mapping of actions to antecedent cues. Participants were also scanned during a subsequent assessment of insensitivity to outcome devaluation. We identified dissociable roles of the cerebellum and ventral striatum, across learning and test performance, in behavioral insensitivity to outcome devaluation. We also showed that the inferior parietal lobule (an area previously implicated in several aspects of goal-directed action selection, including the attribution of intent and awareness of agency) predicted sensitivity to outcome devaluation. Finally, we revealed a potential functional homology between the human subgenual cortex and rodent infralimbic cortex in the implementation of habitual control. In summary, our findings suggested a broad systems division, at the cortical and subcortical levels, between brain areas mediating the encoding and expression of action-outcome and stimulus-response associations.

Attention, reward, and information seeking

Decision making is thought to be guided by the values of alternative options and involve the accumulation of evidence to an internal bound. However, in natural behavior, evidence accumulation is an active process whereby subjects decide when and which sensory stimulus to sample. These sampling decisions are naturally served by attention and rapid eye movements (saccades), but little is known about how saccades are controlled to guide future actions. Here we review evidence that was discussed at a recent symposium, which suggests that information selection involves basal ganglia and cortical mechanisms and that, across different contexts, it is guided by two central factors: the gains in reward and gains in information (uncertainty reduction) associated with sensory cues.