Ventral striatum: a critical look at models of learning and evaluation

From avoidance to new action: the multifaceted role of the striatal indirect pathway

A hallmark of optimal reinforcement learning is that an agent learns to avoid actions that lead to negative outcomes while still exploring alternative actions that could lead to better outcomes. Although the basal ganglia have been hypothesized to contribute to this computation, the mechanisms by which they do so are still unclear. Here, we focus on the function of the striatal indirect pathway and propose that it is regulated by a synaptic plasticity rule that allows an animal to avoid actions that lead to suboptimal outcomes. We consider current theories of striatal indirect pathway function in light of recent experimental findings and discuss studies that suggest that indirect pathway activity is potentiated by the suppression of dopamine release in the striatum. Furthermore, we highlight recent studies showing that activation of the indirect pathway can trigger an action, allowing animals to explore new actions while suppressing suboptimal actions. We show how our framework can reconcile previously conflicting results regarding the indirect pathway and suggest experiments for future investigation.

Pharmacological Modulation of Dopamine Receptors Reveals Distinct Brain-Wide Networks Associated with Learning and Motivation in Nonhuman Primates

The neurotransmitter dopamine (DA) has a multifaceted role in healthy and disordered brains through its action on multiple subtypes of dopaminergic receptors. How the modulation of these receptors influences learning and motivation by altering intrinsic brain-wide networks remains unclear. Here, we performed parallel behavioral and resting-state functional MRI experiments after administration of two different DA receptor antagonists in male and female macaque monkeys. Systemic administration of SCH-23390 (D1 antagonist) slowed probabilistic learning when subjects had to learn new stimulus-reward associations and diminished functional connectivity (FC) in corticocortical and frontostriatal connections. In contrast, haloperidol (D2 antagonist) improved learning and broadly enhanced FC in cortical connections. Further comparisons between the effect of SCH-23390/haloperidol on behavioral and resting-state FC revealed specific cortical and subcortical networks associated with the cognitive and motivational effects of DA manipulation, respectively. Thus, we reveal distinct brain-wide networks that are associated with the dopaminergic control of learning and motivation via DA receptors.

The primate putamen processes cognitive flexibility alongside the caudate and ventral striatum with similar speeds of updating values

The putamen is thought to generate habitual actions by processing value information relayed from the ventral striatum through the caudate nucleus. However, it is a question what value the putamen neurons process and whether the putamen receives serially processed value through the striatal structures. We found that neurons in the primate putamen, caudate, and ventral striatum selectively encoded flexibly updated values for adaptive behaviors with similar learning speeds, rather than stably sustained values for habit. In reversal value learning, rostral striatum neurons dynamically adjusted their responses to object values in alignment with changes in saccade reaction times following reversals. Notably, the value acquisition speeds within trials were similar, proposing a parallel value update in each striatal region. However, in stable value retrieval, most did not encode the values for habitual saccades. Our findings suggest that the rostral striatum including the putamen is selectively involved in the parallel processing of cognitive flexibility.

Hyperactivity and altered functional connectivity of the ventral striatum in schizophrenia compared with bipolar disorder: A resting state fMRI study

BACKGROUND

Schizophrenia patients frequently present with structural and functional abnormalities of the ventral striatum (VS).

METHODS

we examined basal activation state and functional connectivity (FC) in four subregions of the bilateral ventral striatum: left inferior ventral striatum (VSi_L), left superior ventral striatum(VSs_L), right inferior ventral striatum(VSi_R), and right superior ventral striatum(VSs_R). Resting-state functional magnetic resonance images were obtained from 62 schizophrenia patients (SCH), 57 bipolar disorder (BD) patients, and 26 healthy controls (HCs).

RESULTS

The schizophrenia group exhibited greater fALFF in bilateral VS subregions compared to BD and HC groups as well as greater FC between the bilateral VSi and multiple brain regions, including the thalamus, putamen, posterior cingulate gyrus (PCC), frontal cortex and caudate. Moreover, the fALFF values of the bilateral ventral striatum were positively correlated with the severity of positive symptoms. We also found the functional connectivity between the bilateral inferior ventral striatum and some brain regions aforementioned were positively correlated with the severity of negative symptoms.

CONCLUSION

These findings confirm a crucial contribution of ventral striatum dysfunction, especially of the bilateral VSi in schizophrenia. Functionally dissociated regions of the ventral striatum are differentially disturbed in schizophrenia.

Compositional pretraining improves computational efficiency and matches animal behavior on complex tasks

1Recurrent neural networks (RNN) are ubiquitously used in neuroscience to capture both neural dynamics and behaviors of living systems. However, when it comes to complex cognitive tasks, training RNNs with traditional methods can prove difficult and fall short of capturing crucial aspects of animal behavior. Here we propose a principled approach for identifying and incorporating compositional tasks as part of RNN training. Taking as target a temporal wagering task previously studied in rats, we design a pretraining curriculum of simpler cognitive tasks that reflect relevant sub-computations. We show that this pretraining substantially improves learning efficacy and is critical for RNNs to adopt similar strategies as rats, including long-timescale inference of latent states, which conventional pretraining approaches fail to capture. Mechanistically, our pretraining supports the development of slow dynamical systems features needed for implementing both inference and value-based decision making. Overall, our approach is an important step for endowing RNNs with relevant inductive biases, which is important when modeling complex behaviors that rely on multiple cognitive computations.

Distinct Action Signals by Subregions in the Nucleus Accumbens during STOP-Change Performance

The nucleus accumbens (NAc) is thought to contribute to motivated behavior by signaling the value of reward-predicting cues and the delivery of anticipated reward. The NAc is subdivided into core and shell, with each region containing different populations of neurons that increase or decrease firing to rewarding events. While there are numerous theories of functions pertaining to these subregions and cell types, most are in the context of reward processing, with fewer considering that the NAc might serve functions related to action selection more generally. We recorded from single neurons in the NAc as rats of both sexes performed a STOP-change task that is commonly used to study motor control and impulsivity. In this task, rats respond quickly to a spatial cue on 80% of trials (GO) and must stop and redirect planned movement on 20% of trials (STOP). We found that the activity of reward-excited neurons signaled accurate response direction on GO, but not STOP, trials and that these neurons exhibited higher precue firing after correct trials. In contrast, reward-inhibited neurons significantly represented response direction on STOP trials at the time of the instrumental response. Finally, the proportion of reward-excited to reward-inhibited neurons and the strength of precue firing decreased as the electrode traversed the NAc. We conclude that reward-excited cells (more common in core) promote proactive action selection, while reward-inhibited cells (more common in shell) contribute to accurate responding on STOP trials that require reactive suppression and redirection of behavior.

Examining neural responses to anticipating or receiving monetary rewards and the development of binge eating in youth. A registered report using data from the Adolescent Brain Cognitive Development (ABCD) study

Binge eating is characterized as eating a large amount of food and feeling a loss of control while eating. However, the neurobiological mechanisms associated with the onset and maintenance of binge eating are largely unknown. Recent neuroimaging work has suggested that increased responsivity within reward regions of the brain to the anticipation or receipt of rewards is related to binge eating; however, limited longitudinal data has precluded understanding of the role of reward responsivity in the development of binge eating. The current study used data from the Adolescent Brain and Cognitive Development® (ABCD) longitudinal study dataset to assess whether heightened neural responses to different phases of reward processing (reward anticipation and receipt) (1) differentiated individuals with binge eating from matched controls, and (2) predicted the onset of binge eating in an "at risk" sample. Consistent with hypotheses, heightened neural responsivity in the right caudate and bilateral VS during reward anticipation differentiated youth with and without binge eating. Moreover, greater VS response to reward anticipation predicted binge eating two years later. Neural responses to reward receipt also were consistent with hypotheses, such that heightened VS and OFC responses differentiated youth with and without binge eating and predicted the presence of binge eating two years later. Findings from the current study suggest that hypersensitivity to rewards may contribute to the development of binge eating during early adolescence.

Annual Research Review: Puberty and the development of anhedonia - considering childhood adversity and inflammation

Anhedonia, or diminished pleasure and motivation, is a symptom of severe mental illness (e.g., depressive disorder, bipolar disorder, schizophrenia) that emerges during adolescence. Anhedonia is a pernicious symptom that is related to social impairments, treatment resistance, and suicide. As the mechanisms of anhedonia are postulated to include the frontostriatal circuitry and the dopamine neuromodulatory system, the development and plasticity of these systems during the vulnerable period of adolescence, as well as their sensitivity to pubertal hormones, suggest that pubertal maturation could play a role in the development of anhedonia. This review takes a developmental perspective, considering the possibility that anhedonia emerges in the context of pubertal maturation and adolescent development, with childhood adversity and chronic inflammation influencing neural reward systems to accelerate anhedonia's progression. Here, we review the relevant extant literature on the components of this model and suggest directions for future research.

Greater reliance on model-free learning in adolescent anorexia nervosa: An examination of dual-system reinforcement learning

Alterations in learning and decision-making systems are thought to contribute to core features of anorexia nervosa (AN), a psychiatric disorder characterized by persistent dietary restriction and weight loss. Instrumental learning theory identifies a dual-system of habit and goal-directed decision-making, linked to model-free and model-based reinforcement learning algorithms. Difficulty arbitrating between these systems, resulting in an over-reliance on one strategy over the other, has been implicated in compulsivity and extreme goal pursuit, both of which are observed in AN. Characterizing alterations in model-free and model-based systems, and their neural correlates, in AN may clarify mechanisms contributing to symptom heterogeneity (e.g., binge/purge symptoms). This study tested whether adolescents with restricting AN (AN-R; n = 36) and binge/purge AN (AN-BP; n = 20) differentially utilized model-based and model-free learning systems compared to a healthy control group (HC; n = 28) during a Markov two-step decision-making task under conditions of reward and punishment. Associations between model-free and model-based learning and resting-state functional connectivity between neural regions of interest, including orbitofrontal cortex (OFC), nucleus accumbens (NAcc), putamen, and sensory motor cortex (SMC) were examined. AN-R showed higher utilization of model-free learning compared to HC for reward, but attenuated model-free and model-based learning for punishment. In AN-R only, higher model-based learning was associated with stronger OFC-to-left NAcc functional connectivity, regions linked to goal-directed behavior. Greater utilization of model-free learning for reward in AN-R may differentiate this group, particularly during adolescence, and facilitate dietary restriction by prioritizing habitual control in rewarding contexts.

Nucleus accumbens and dorsal medial striatal dopamine and neural activity are essential for action sequence performance

Separable striatal circuits have unique functions in Pavlovian and instrumental behaviors but how these roles relate to performance of sequences of actions with and without associated cues are less clear. Here, we tested whether dopamine transmission and neural activity more generally in three striatal subdomains are necessary for performance of an action chain leading to reward delivery. Male and female Long-Evans rats were trained to press a series of three spatially distinct levers to receive reward. We assessed the contribution of neural activity or dopamine transmission within each striatal subdomain when progression through the action sequence was explicitly cued and in the absence of cues. Behavior in both task variations was substantially impacted following microinfusion of the dopamine antagonist, flupenthixol, into nucleus accumbens core (NAc) or dorsomedial striatum (DMS), with impairments in sequence timing and numbers of rewards earned after NAc flupenthixol. In contrast, after pharmacological inactivation to suppress overall activity, there was minimal impact on total rewards earned. Instead, inactivation of both NAc and DMS impaired sequence timing and led to sequence errors in the uncued, but not cued task. There was no impact of dopamine antagonism or reversible inactivation of dorsolateral striatum on either cued or uncued action sequence completion. These results highlight an essential contribution of NAc and DMS dopamine systems in motivational and performance aspects of chains of actions, whether cued or internally generated, as well as the impact of intact NAc and DMS function for correct sequence performance.

Pain, from perception to action: A computational perspective

Pain is driven by sensation and emotion, and in turn, it motivates decisions and actions. To fully appreciate the multidimensional nature of pain, we formulate the study of pain within a closed-loop framework of sensory-motor prediction. In this closed-loop cycle, prediction plays an important role, as the interaction between prediction and actual sensory experience shapes pain perception and subsequently, action. In this Perspective, we describe the roles of two prominent computational theories-Bayesian inference and reinforcement learning-in modeling adaptive pain behaviors. We show that prediction serves as a common theme between these two theories, and that each of these theories can explain unique aspects of the pain perception-action cycle. We discuss how these computational theories and models can improve our mechanistic understandings of pain-centered processes such as anticipation, attention, placebo hypoalgesia, and pain chronification.

Neural connectome features of procrastination: Current progress and future direction

Procrastination refers to an irrationally delay for intended courses of action despite of anticipating a negative consequence due to this delay. Previous studies tried to reveal the neural substrates of procrastination in terms of connectome-based biomarkers. Based on this, we proposed a unified triple brain network model for procrastination and pinpointed out what challenges we are facing in understanding neural mechanism of procrastination. Specifically, based on neuroanatomical features, the unified triple brain network model proposed that connectome-based underpinning of procrastination could be ascribed to the abnormalities of self-control network (i.e., dorsolateral prefrontal cortex, DLPFC), emotion-regulation network (i.e., orbital frontal cortex, OFC), and episodic prospection network (i.e., para-hippocampus cortex, PHC). Moreover, based on the brain functional features, procrastination had been attributed to disruptive neural circuits on FPN (frontoparietal network)-SCN (subcortical network) and FPN-SAN (salience network), which led us to hypothesize the crucial roles of interplay between these networks on procrastination in unified triple brain network model. Despite of these findings, poor interpretability and computational model limited further understanding for procrastination from theoretical and neural perspectives. On balance, the current study provided an overview to show current progress on the connectome-based biomarkers for procrastination, and proposed the integrative neurocognitive model of procrastination.

Context coding in the mouse nucleus accumbens modulates motivationally relevant information

Neural activity in the nucleus accumbens (NAc) is thought to track fundamentally value-centric quantities linked to reward and effort. However, the NAc also contributes to flexible behavior in ways that are difficult to explain based on value signals alone, raising the question of if and how nonvalue signals are encoded in NAc. We recorded NAc neural ensembles while head-fixed mice performed an odor-based biconditional discrimination task where an initial discrete cue modulated the behavioral significance of a subsequently presented reward-predictive cue. We extracted single-unit and population-level correlates related to the cues and found value-independent coding for the initial, context-setting cue. This context signal occupied a population-level coding space orthogonal to outcome-related representations and was predictive of subsequent behaviorally relevant responses to the reward-predictive cues. Together, these findings support a gating model for how the NAc contributes to behavioral flexibility and provide a novel population-level perspective from which to view NAc computations.

Reinforcement learning approaches to hippocampus-dependent flexible spatial navigation

Humans and non-human animals show great flexibility in spatial navigation, including the ability to return to specific locations based on as few as one single experience. To study spatial navigation in the laboratory, watermaze tasks, in which rats have to find a hidden platform in a pool of cloudy water surrounded by spatial cues, have long been used. Analogous tasks have been developed for human participants using virtual environments. Spatial learning in the watermaze is facilitated by the hippocampus. In particular, rapid, one-trial, allocentric place learning, as measured in the delayed-matching-to-place variant of the watermaze task, which requires rodents to learn repeatedly new locations in a familiar environment, is hippocampal dependent. In this article, we review some computational principles, embedded within a reinforcement learning framework, that utilise hippocampal spatial representations for navigation in watermaze tasks. We consider which key elements underlie their efficacy, and discuss their limitations in accounting for hippocampus-dependent navigation, both in terms of behavioural performance (i.e. how well do they reproduce behavioural measures of rapid place learning) and neurobiological realism (i.e. how well do they map to neurobiological substrates involved in rapid place learning). We discuss how an actor-critic architecture, enabling simultaneous assessment of the value of the current location and of the optimal direction to follow, can reproduce one-trial place learning performance as shown on watermaze and virtual delayed-matching-to-place tasks by rats and humans, respectively, if complemented with map-like place representations. The contribution of actor-critic mechanisms to delayed-matching-to-place performance is consistent with neurobiological findings implicating the striatum and hippocampo-striatal interaction in delayed-matching-to-place performance, given that the striatum has been associated with actor-critic mechanisms. Moreover, we illustrate that hierarchical computations embedded within an actor-critic architecture may help to account for aspects of flexible spatial navigation. The hierarchical reinforcement learning approach separates trajectory control via a temporal-difference error from goal selection via a goal prediction error and may account for flexible, trial-specific, navigation to familiar goal locations, as required in some arm-maze place memory tasks, although it does not capture one-trial learning of new goal locations, as observed in open field, including watermaze and virtual, delayed-matching-to-place tasks. Future models of one-shot learning of new goal locations, as observed on delayed-matching-to-place tasks, should incorporate hippocampal plasticity mechanisms that integrate new goal information with allocentric place representation, as such mechanisms are supported by substantial empirical evidence.

Phasic dopamine reinforces distinct striatal stimulus encoding in the olfactory tubercle driving dopaminergic reward prediction

The learning of stimulus-outcome associations allows for predictions about the environment. Ventral striatum and dopaminergic midbrain neurons form a larger network for generating reward prediction signals from sensory cues. Yet, the network plasticity mechanisms to generate predictive signals in these distributed circuits have not been entirely clarified. Also, direct evidence of the underlying interregional assembly formation and information transfer is still missing. Here we show that phasic dopamine is sufficient to reinforce the distinctness of stimulus representations in the ventral striatum even in the absence of reward. Upon such reinforcement, striatal stimulus encoding gives rise to interregional assemblies that drive dopaminergic neurons during stimulus-outcome learning. These assemblies dynamically encode the predicted reward value of conditioned stimuli. Together, our data reveal that ventral striatal and midbrain reward networks form a reinforcing loop to generate reward prediction coding.

It is not entirely understood how network plasticity produces the coding of predicted value during stimulus-outcome learning. Here, the authors reveal a reinforcing loop in distributed limbic circuits, transforming sensory stimuli into reward prediction coding broadcasted by dopamine neurons to the brain.

The ventral striatum dissociates information expectation, reward anticipation, and reward receipt

Do dopaminergic reward structures represent the expected utility of information similarly to a reward? Optimal experimental design models from Bayesian decision theory and statistics have proposed a theoretical framework for quantifying the expected value of information that might result from a query. In particular, this formulation quantifies the value of information before the answer to that query is known, in situations where payoffs are unknown and the goal is purely epistemic: That is, to increase knowledge about the state of the world. Whether and how such a theoretical quantity is represented in the brain is unknown. Here we use an event-related functional MRI (fMRI) task design to disentangle information expectation, information revelation and categorization outcome anticipation, and response-contingent reward processing in a visual probabilistic categorization task. We identify a neural signature corresponding to the expectation of information, involving the left lateral ventral striatum. Moreover, we show a temporal dissociation in the activation of different reward-related regions, including the nucleus accumbens, medial prefrontal cortex, and orbitofrontal cortex, during information expectation versus reward-related processing.

Learning, memory and consolidation mechanisms for behavioral control in hierarchically organized cortico-basal ganglia systems

This article aims to provide a synthesis on the question how brain structures cooperate to accomplish hierarchically organized behaviors, characterized by low‐level, habitual routines nested in larger sequences of planned, goal‐directed behavior. The functioning of a connected set of brain structures—prefrontal cortex, hippocampus, striatum, and dopaminergic mesencephalon—is reviewed in relation to two important distinctions: (a) goal‐directed as opposed to habitual behavior and (b) model‐based and model‐free learning. Recent evidence indicates that the orbitomedial prefrontal cortices not only subserve goal‐directed behavior and model‐based learning, but also code the “landscape” (task space) of behaviorally relevant variables. While the hippocampus stands out for its role in coding and memorizing world state representations, it is argued to function in model‐based learning but is not required for coding of action–outcome contingencies, illustrating that goal‐directed behavior is not congruent with model‐based learning. While the dorsolateral and dorsomedial striatum largely conform to the dichotomy between habitual versus goal‐directed behavior, ventral striatal functions go beyond this distinction. Next, we contextualize findings on coding of reward‐prediction errors by ventral tegmental dopamine neurons to suggest a broader role of mesencephalic dopamine cells, viz. in behavioral reactivity and signaling unexpected sensory changes. We hypothesize that goal‐directed behavior is hierarchically organized in interconnected cortico‐basal ganglia loops, where a limbic‐affective prefrontal‐ventral striatal loop controls action selection in a dorsomedial prefrontal–striatal loop, which in turn regulates activity in sensorimotor‐dorsolateral striatal circuits. This structure for behavioral organization requires alignment with mechanisms for memory formation and consolidation. We propose that frontal corticothalamic circuits form a high‐level loop for memory processing that initiates and temporally organizes nested activities in lower‐level loops, including the hippocampus and the ripple‐associated replay it generates. The evidence on hierarchically organized behavior converges with that on consolidation mechanisms in suggesting a frontal‐to‐caudal directionality in processing control.

An Analysis of Decision under Risk in Rats

In 1979, Daniel Kahneman and Amos Tversky published a ground-breaking paper titled "Prospect Theory: An Analysis of Decision under Risk," which presented a behavioral economic theory that accounted for the ways in which humans deviate from economists' normative workhorse model, Expected Utility Theory [1, 2]. For example, people exhibit probability distortion (they overweight low probabilities), loss aversion (losses loom larger than gains), and reference dependence (outcomes are evaluated as gains or losses relative to an internal reference point). We found that rats exhibited many of these same biases, using a task in which rats chose between guaranteed and probabilistic rewards. However, prospect theory assumes stable preferences in the absence of learning, an assumption at odds with alternative frameworks such as animal learning theory and reinforcement learning [3-7]. Rats also exhibited trial history effects, consistent with ongoing learning. A reinforcement learning model in which state-action values were updated by the subjective value of outcomes according to prospect theory reproduced rats' nonlinear utility and probability weighting functions and also captured trial-by-trial learning dynamics.

Hippocampal pattern separation supports reinforcement learning

Animals rely on learned associations to make decisions. Associations can be based on relationships between object features (e.g., the three leaflets of poison ivy leaves) and outcomes (e.g., rash). More often, outcomes are linked to multidimensional states (e.g., poison ivy is green in summer but red in spring). Feature-based reinforcement learning fails when the values of individual features depend on the other features present. One solution is to assign value to multi-featural conjunctive representations. Here, we test if the hippocampus forms separable conjunctive representations that enables the learning of response contingencies for stimuli of the form: AB+, B-, AC-, C+. Pattern analyses on functional MRI data show the hippocampus forms conjunctive representations that are dissociable from feature components and that these representations, along with those of cortex, influence striatal prediction errors. Our results establish a novel role for hippocampal pattern separation and conjunctive representation in reinforcement learning.

Neural signatures of reinforcement learning correlate with strategy adoption during spatial navigation

Human navigation is generally believed to rely on two types of strategy adoption, route-based and map-based strategies. Both types of navigation require making spatial decisions along the traversed way although formal computational and neural links between navigational strategies and mechanisms of value-based decision making have so far been underexplored in humans. Here we employed functional magnetic resonance imaging (fMRI) while subjects located different objects in a virtual environment. We then modelled their paths using reinforcement learning (RL) algorithms, which successfully explained decision behavior and its neural correlates. Our results show that subjects used a mixture of route and map-based navigation and their paths could be well explained by the model-free and model-based RL algorithms. Furthermore, the value signals of model-free choices during route-based navigation modulated the BOLD signals in the ventro-medial prefrontal cortex (vmPFC), whereas the BOLD signals in parahippocampal and hippocampal regions pertained to model-based value signals during map-based navigation. Our findings suggest that the brain might share computational mechanisms and neural substrates for navigation and value-based decisions such that model-free choice guides route-based navigation and model-based choice directs map-based navigation. These findings open new avenues for computational modelling of wayfinding by directing attention to value-based decision, differing from common direction and distances approaches.

Cocaine Place Conditioning Strengthens Location-Specific Hippocampal Coupling to the Nucleus Accumbens

Conditioned place preference (CPP) is a widely used model of addiction-related behavior whose underlying mechanisms are not understood. In this study, we used dual site silicon probe recordings in freely moving mice to examine interactions between the hippocampus and nucleus accumbens in cocaine CPP. We found that CPP was associated with recruitment of D2-positive nucleus accumbens medium spiny neurons to fire in the cocaine-paired location, and this recruitment was driven predominantly by selective strengthening of coupling with hippocampal place cells that encode the cocaine-paired location. These findings provide in vivo evidence suggesting that the synaptic potentiation in the accumbens caused by repeated cocaine administration preferentially affects inputs that were active at the time of drug exposure. This provides a potential physiological mechanism by which drug use becomes associated with specific environmental contexts.

Interactions of spatial strategies producing generalization gradient and blocking: A computational approach

We present a computational model of spatial navigation comprising different learning mechanisms in mammals, i.e., associative, cognitive mapping and parallel systems. This model is able to reproduce a large number of experimental results in different variants of the Morris water maze task, including standard associative phenomena (spatial generalization gradient and blocking), as well as navigation based on cognitive mapping. Furthermore, we show that competitive and cooperative patterns between different navigation strategies in the model allow to explain previous apparently contradictory results supporting either associative or cognitive mechanisms for spatial learning. The key computational mechanism to reconcile experimental results showing different influences of distal and proximal cues on the behavior, different learning times, and different abilities of individuals to alternatively perform spatial and response strategies, relies in the dynamic coordination of navigation strategies, whose performance is evaluated online with a common currency through a modular approach. We provide a set of concrete experimental predictions to further test the computational model. Overall, this computational work sheds new light on inter-individual differences in navigation learning, and provides a formal and mechanistic approach to test various theories of spatial cognition in mammals.

Functional dysconnectivity of the limbic loop of frontostriatal circuits in first-episode, treatment-naive schizophrenia

Frontostriatal circuits dysfunction has been implicated in the etiology and psychopathology of patients with schizophrenia (SZ). However, few studies have investigated SZ-related functional connectivity (FC) alterations in discrete frontostriatal circuits and their relationship with pathopsychology in first-episode schizophrenia (FESZ). The goal of this study was to identify dysfunctions in discrete frontostriatal circuits that are associated with key features of FESZ. To this end, a case-control, cross-sectional study was conducted, wherein resting-state (RS) functional magnetic resonance (fMRI) data were collected from 37 treatment-naïve FESZ patients and 29 healthy control (HC) subjects. Seed-based FC analyses were performed by placing six bilateral pairs of seeds within a priori defined subdivisions of the striatum. We observed significantly decreased FC for the FESZ group relative to the HC group [p < .05, family-wise error (FWE)-corrected] in the limbic loop, but not in the sensorimotor or associative loops, of frontostriatal circuitry. Moreover, bilaterally decreased inferior ventral striatum/nucleus accumbens (VSi)-dorsal anterior cingulate cortex (dACC) FC within the limbic loop correlated inversely with overall FESZ symptom severity and the disorganization factor score of PANSS. These findings provide new insight into the role of frontostriatal limbic loop hypoconnectivity in early-stage schizophrenia pathology and suggest potential novel therapeutic targets.

Dorsal striatum mediates deliberate decision making, not late-stage, stimulus-response learning

We investigated a controversy regarding the role of the dorsal striatum (DS) in deliberate decision-making versus late-stage, stimulus-response learning to the point of automatization. Participants learned to associate abstract images with right or left button presses explicitly before strengthening these associations through stimulus-response trials with (i.e., Session 1) and without (i.e., Session 2) feedback. In Session 1, trials were divided into response-selection and feedback events to separately assess decision versus learning processes. Session 3 evaluated stimulus-response automaticity using a location Stroop task. DS activity correlated with response-selection and not feedback events in Phase 1 (i.e., Blocks 1-3), Session 1. Longer response times (RTs), lower accuracy, and greater intertrial variability characterized Phase 1, suggesting deliberation. DS activity extinguished in Phase 2 (i.e., Blocks 4-12), Session 1, once RTs, response variability, and accuracy stabilized, though stimulus-response automatization continued. This was signaled by persisting improvements in RT and accuracy into Session 2. Distraction between Sessions 1 and 2 briefly reintroduced response uncertainty, and correspondingly, significant DS activity reappeared in Block 1 of Session 2 only. Once stimulus-response associations were again refamiliarized and deliberation unnecessary, DS activation disappeared for Blocks 2-8, Session 2. Interference from previously learned right or left button responses with incongruent location judgments in a location Stroop task provided evidence that automaticity of stimulus-specific button-press responses had developed by the end of Session 2. These results suggest that DS mediates decision making and not late-stage learning, reconciling two, independently evolving and well-supported literatures that implicate DS in different cognitive functions. Hum Brain Mapp 38:6133-6156, 2017. © 2017 Wiley Periodicals, Inc.

Post-conventional moral reasoning is associated with increased ventral striatal activity at rest and during task

People vary considerably in moral reasoning. According to Kohlberg’s theory, individuals who reach the highest level of post-conventional moral reasoning judge moral issues based on deeper principles and shared ideals rather than self-interest or adherence to laws and rules. Recent research has suggested the involvement of the brain’s frontostriatal reward system in moral judgments and prosocial behaviors. However, it remains unknown whether moral reasoning level is associated with differences in reward system function. Here, we combined arterial spin labeling perfusion and blood oxygen level-dependent functional magnetic resonance imaging and measured frontostriatal reward system activity both at rest and during a sequential risky decision making task in a sample of 64 participants at different levels of moral reasoning. Compared to individuals at the pre-conventional and conventional level of moral reasoning, post-conventional individuals showed increased resting cerebral blood flow in the ventral striatum and ventromedial prefrontal cortex. Cerebral blood flow in these brain regions correlated with the degree of post-conventional thinking across groups. Post-conventional individuals also showed greater task-induced activation in the ventral striatum during risky decision making. These findings suggest that high-level post-conventional moral reasoning is associated with increased activity in the brain’s frontostriatal system, regardless of task-dependent or task-independent states.

Gamma Oscillations in the Rat Ventral Striatum Originate in the Piriform Cortex

Local field potentials (LFPs) recorded from the human and rodent ventral striatum (vStr) exhibit prominent, behaviorally relevant gamma-band oscillations. These oscillations are related to local spiking activity and transiently synchronize with anatomically related areas, suggesting a possible role in organizing vStr activity. However, the origin of vStr gamma is unknown. We recorded vStr gamma oscillations across a 1.4 mm² grid spanned by 64 recording electrodes as male rats rested and foraged for rewards, revealing a highly consistent power gradient originating in the adjacent piriform cortex. Phase differences across the vStr were consistently small (<15°) and current source density analysis further confirmed the absence of local sink-source pairs in the vStr. Reversible occlusions of the ipsilateral (but not contralateral) nostril, known to abolish gamma oscillations in the piriform cortex, strongly reduced vStr gamma power and the occurrence of transient gamma-band events. These results imply that local circuitry is not a major contributor to gamma oscillations in the vStr LFP and that piriform cortex is an important driver of gamma-band oscillations in the vStr and associated limbic areas.SIGNIFICANCE STATEMENT The ventral striatum (vStr) is an area of anatomical convergence in circuits underlying motivated behavior, but it remains unclear how its inputs from different sources interact. A major proposal about how neural circuits may switch dynamically between convergent inputs is through temporal organization reflected in local field potential (LFP) oscillations. Our results show that, in the rat, the mechanisms controlling gamma-band oscillations in the vStr LFP are primarily located in the in the adjacent piriform cortex rather than in the vStr itself, providing a novel interpretation of previous rodent work on gamma oscillations in the vStr and related circuits and an important consideration for future work seeking to use oscillations in these areas as biomarkers for behavioral and neurological disorders.

Lenticular nucleus correlates of general self-efficacy in young adults

General self-efficacy (GSE) is an important factor in education, social participation, and medical treatment. However, the only study that has investigated the direct association between GSE and a neural correlate did not identify specific brain regions, rather only assessed brain structures, and included older adult subjects. GSE is related to motivation, physical activity, learning, the willingness to initiate behaviour and expend effort, and adjustment. Thus, it was hypothesized in the present study that the neural correlates of GSE might be related to changes in the basal ganglia, which is a region related to the abovementioned self-efficacy factors. This study aimed to identify the brain structures associated with GSE in healthy young adults (n = 1204, 691 males and 513 females, age 20.7 ± 1.8 years) using regional grey matter density and volume (rGMD and rGMV), fractional anisotropy (FA) and mean diffusivity (MD) analyses of magnetic resonance imaging (MRI) data. The findings showed that scores on the GSE Scale (GSES) were associated with a lower MD value in regions from the right putamen to the globus pallidum; however, there were no significant association between GSES scores and regional brain structures using the other analyses (rGMD, rGMV, and FA). Thus, the present findings indicated that the lenticular nucleus is a neural correlate of GSE.

Dopamine Does Double Duty in Motivating Cognitive Effort

Cognitive control is subjectively costly, suggesting that engagement is modulated in relationship to incentive state. Dopamine appears to play key roles. In particular, dopamine may mediate cognitive effort by two broad classes of functions: (1) modulating the functional parameters of working memory circuits subserving effortful cognition, and (2) mediating value-learning and decision-making about effortful cognitive action. Here, we tie together these two lines of research, proposing how dopamine serves "double duty", translating incentive information into cognitive motivation.

The Neural Representation of Goal-Directed Actions and Outcomes in the Ventral Striatum's Olfactory Tubercle

The ventral striatum is critical for evaluating reward information and the initiation of goal-directed behaviors. The many cellular, afferent, and efferent similarities between the ventral striatum's nucleus accumbens and olfactory tubercle (OT) suggests the distributed involvement of neurons within the ventral striatopallidal complex in motivated behaviors. Although the nucleus accumbens has an established role in representing goal-directed actions and their outcomes, it is not known whether this function is localized within the nucleus accumbens or distributed also within the OT. Answering such a fundamental question will expand our understanding of the neural mechanisms underlying motivated behaviors. Here we address whether the OT encodes natural reinforcers and serves as a substrate for motivational information processing. In recordings from mice engaged in a novel water-motivated instrumental task, we report that OT neurons modulate their firing rate during initiation and progression of the instrumental licking behavior, with some activity being internally generated and preceding the first lick. We further found that as motivational drive decreases throughout a session, the activity of OT neurons is enhanced earlier relative to the behavioral action. Additionally, OT neurons discriminate the types and magnitudes of fluid reinforcers. Together, these data suggest that the processing of reward information and the orchestration of goal-directed behaviors is a global principle of the ventral striatum and have important implications for understanding the neural systems subserving addiction and mood disorders.

SIGNIFICANCE STATEMENT

Goal-directed behaviors are widespread among animals and underlie complex behaviors ranging from food intake, social behavior, and even pathological conditions, such as gambling and drug addiction. The ventral striatum is a neural system critical for evaluating reward information and the initiation of goal-directed behaviors. Here we show that neurons in the olfactory tubercle subregion of the ventral striatum robustly encode the onset and progression of motivated behaviors, and discriminate the type and magnitude of a reward. Our findings are novel in showing that olfactory tubercle neurons participate in such coding schemes and are in accordance with the principle that ventral striatum substructures may cooperate to guide motivated behaviors.

Acute ethanol effects on neural encoding of reward size and delay in the nucleus accumbens

Acute ethanol administration can cause impulsivity, resulting in increased preference for immediately available rewards over delayed but more valuable alternatives. The manner in which reward size and delay are represented in neural firing is not fully understood, and very little is known about ethanol effects on this encoding. To address this issue, we used in vivo electrophysiology to characterize neural firing in the core of the nucleus accumbens (NAcc) in rats responding for rewards that varied in size or delay after vehicle or ethanol administration. The NAcc is a central element in the circuit that governs decision-making and importantly, promotes choice of delayed rewards. We found that NAcc firing in response to reward-predictive cues encoded anticipated reward value after vehicle administration, but ethanol administration disrupted this encoding, resulting in a loss of discrimination between immediate and delayed rewards in cue-evoked neural responses. In addition, NAcc firing occurring at the time of the operant response (lever pressing) was inversely correlated with behavioral response latency, such that increased firing rates were associated with decreased latencies to lever press. Ethanol administration selectively attenuated this lever press-evoked firing when delayed but not immediate rewards were expected. These effects on neural firing were accompanied by increased behavioral latencies to respond for delayed rewards. Our results suggest that ethanol effects on NAcc cue- and lever press-evoked encoding may contribute to ethanol-induced impulsivity.

The Computational Complexity of Valuation and Motivational Forces in Decision-Making Processes

The concept of value is fundamental to most theories of motivation and decision making. However, value has to be measured experimentally. Different methods of measuring value produce incompatible valuation hierarchies. Taking the agent's perspective (rather than the experimenter's), we interpret the different valuation measurement methods as accessing different decision-making systems and show how these different systems depend on different information processing algorithms. This identifies the translation from these multiple decision-making systems into a single action taken by a given agent as one of the most important open questions in decision making today. We conclude by looking at how these different valuation measures accessing different decision-making systems can be used to understand and treat decision dysfunction such as in addiction.

Mesolimbic dopamine signals the value of work

Dopamine cell firing can encode errors in reward prediction, providing a learning signal to guide future behavior. Yet dopamine is also a key modulator of motivation, invigorating current behavior. Existing theories propose that fast (phasic) dopamine fluctuations support learning, whereas much slower (tonic) dopamine changes are involved in motivation. We examined dopamine release in the nucleus accumbens across multiple time scales, using complementary microdialysis and voltammetric methods during adaptive decision-making. We found that minute-by-minute dopamine levels covaried with reward rate and motivational vigor. Second-by-second dopamine release encoded an estimate of temporally discounted future reward (a value function). Changing dopamine immediately altered willingness to work and reinforced preceding action choices by encoding temporal-difference reward prediction errors. Our results indicate that dopamine conveys a single, rapidly evolving decision variable, the available reward for investment of effort, which is employed for both learning and motivational functions.

The olfactory tubercle encodes odor valence in behaving mice

Sensory information acquires meaning to adaptively guide behaviors. Despite odors mediating a number of vital behaviors, the components of the olfactory system responsible for assigning meaning to odors remain unclear. The olfactory tubercle (OT), a ventral striatum structure that receives monosynaptic input from the olfactory bulb, is uniquely positioned to transform odor information into behaviorally relevant neural codes. No information is available, however, on the coding of odors among OT neurons in behaving animals. In recordings from mice engaged in an odor discrimination task, we report that the firing rate of OT neurons robustly and flexibly encodes the valence of conditioned odors over identity, with rewarded odors evoking greater firing rates. This coding of rewarded odors occurs before behavioral decisions and represents subsequent behavioral responses. We predict that the OT is an essential region whereby odor valence is encoded in the mammalian brain to guide goal-directed behaviors.

Solving the detour problem in navigation: a model of prefrontal and hippocampal interactions

Adapting behavior to accommodate changes in the environment is an important function of the nervous system. A universal problem for motile animals is the discovery that a learned route is blocked and a detour is required. Given the substantial neuroscience research on spatial navigation and decision-making it is surprising that so little is known about how the brain solves the detour problem. Here we review the limited number of relevant functional neuroimaging, single unit recording and lesion studies. We find that while the prefrontal cortex (PFC) consistently responds to detours, the hippocampus does not. Recent evidence suggests the hippocampus tracks information about the future path distance to the goal. Based on this evidence we postulate a conceptual model in which: Lateral PFC provides a prediction error signal about the change in the path, frontopolar and superior PFC support the re-formulation of the route plan as a novel subgoal and the hippocampus simulates the new path. More data will be required to validate this model and understand (1) how the system processes the different options; and (2) deals with situations where a new path becomes available (i.e., shortcuts).

Neurons in the nucleus accumbens promote selection bias for nearer objects

Both animals and humans often prefer rewarding options that are nearby over those that are distant, but the neural mechanisms underlying this bias are unclear. Here we present evidence that a proximity signal encoded by neurons in the nucleus accumbens drives proximate reward bias by promoting impulsive approach to nearby reward-associated objects. On a novel decision-making task, rats chose the nearer option even when it resulted in greater effort expenditure and delay to reward; therefore, proximate reward bias was unlikely to be caused by effort or delay discounting. The activity of individual neurons in the nucleus accumbens did not consistently encode the reward or effort associated with specific alternatives, suggesting that it does not participate in weighing the values of options. In contrast, proximity encoding was consistent and did not depend on the subsequent choice, implying that accumbens activity drives approach to the nearest rewarding option regardless of its specific associated reward size or effort level.

From ventral-medial to dorsal-lateral striatum: neural correlates of reward-guided decision-making

The striatum is critical for reward-guided and habitual behavior. Anatomical and interference studies suggest a functional heterogeneity within striatum. Medial regions, such as nucleus accumbens core and dorsal medial striatum play roles in goal-directed behavior, while dorsal lateral striatum is critical for control of habitual action. Subdivisions of striatum are topographically connected with different cortical and subcortical structures forming channels that carry information related to limbic, associative, and sensorimotor functions. Here, we describe data showing that as one progresses from ventral-medial to dorsal-lateral striatum, there is a shift from more prominent value encoding to activity more closely related to associative and motor aspects of decision-making. In addition, we will describe data suggesting that striatal circuits work in parallel to control behavior and that regions within striatum can compensate for each other when functions are disrupted.

Neurophysiology of Reward-Guided Behavior: Correlates Related to Predictions, Value, Motivation, Errors, Attention, and Action

Many brain areas are activated by the possibility and receipt of reward. Are all of these brain areas reporting the same information about reward? Or are these signals related to other functions that accompany reward-guided learning and decision-making? Through carefully controlled behavioral studies, it has been shown that reward-related activity can represent reward expectations related to future outcomes, errors in those expectations, motivation, and signals related to goal- and habit-driven behaviors. These dissociations have been accomplished by manipulating the predictability of positively and negatively valued events. Here, we review single neuron recordings in behaving animals that have addressed this issue. We describe data showing that several brain areas, including orbitofrontal cortex, anterior cingulate, and basolateral amygdala signal reward prediction. In addition, anterior cingulate, basolateral amygdala, and dopamine neurons also signal errors in reward prediction, but in different ways. For these areas, we will describe how unexpected manipulations of positive and negative value can dissociate signed from unsigned reward prediction errors. All of these signals feed into striatum to modify signals that motivate behavior in ventral striatum and guide responding via associative encoding in dorsolateral striatum.

Neurons in the ventral striatum exhibit cell-type-specific representations of outcome during learning

The ventromedial striatum (VMS) is a node in circuits underpinning both affect and reinforcement learning. The cellular bases of these functions and especially their potential linkages have been unclear. VMS cholinergic interneurons, however, have been singled out as being related both to affect and to reinforcement-based conditioning, raising the possibility that unique aspects of their signaling could account for these functions. Here we show that VMS tonically active neurons (TANs), including putative cholinergic interneurons, generate unique bidirectional outcome responses during reward-based learning, reporting both positive (reward) and negative (reward omission) outcomes when behavioral change is prompted by switches in reinforcement contingencies. VMS output neurons (SPNs), by contrast, are nearly insensitive to switches in reinforcement contingencies, gradually losing outcome signaling while maintaining responses at trial initiation and goal approach. Thus, TANs and SPNs in the VMS provide distinct signals optimized for different aspects of the learning process.

Experimental predictions drawn from a computational model of sign-trackers and goal-trackers

Gaining a better understanding of the biological mechanisms underlying the individual variation observed in response to rewards and reward cues could help to identify and treat individuals more prone to disorders of impulsive control, such as addiction. Variation in response to reward cues is captured in rats undergoing autoshaping experiments where the appearance of a lever precedes food delivery. Although no response is required for food to be delivered, some rats (goal-trackers) learn to approach and avidly engage the magazine until food delivery, whereas other rats (sign-trackers) come to approach and engage avidly the lever. The impulsive and often maladaptive characteristics of the latter response are reminiscent of addictive behaviour in humans. In a previous article, we developed a computational model accounting for a set of experimental data regarding sign-trackers and goal-trackers. Here we show new simulations of the model to draw experimental predictions that could help further validate or refute the model. In particular, we apply the model to new experimental protocols such as injecting flupentixol locally into the core of the nucleus accumbens rather than systemically, and lesioning of the core of the nucleus accumbens before or after conditioning. In addition, we discuss the possibility of removing the food magazine during the inter-trial interval. The predictions from this revised model will help us better understand the role of different brain regions in the behaviours expressed by sign-trackers and goal-trackers.

Double dissociation between the contributions of the septal and temporal hippocampus to spatial learning: the role of prior experience

The mammalian hippocampus is anatomically heterogeneous along its longitudinal axis, and there is evidence that distinct functions are executed by different septotemporal subregions. The best documented example is the dependency of spatial learning on the septal, but not the temporal, hippocampus. Here, we carried out a watermaze memory task in rats with partial lesions of the septal or temporal hippocampus made either before or after training. We then studied memory retention, reversal, and new spatial learning in a novel environment. This resulted in the surprising finding that spatial learning in a new environment is dependent on the temporal hippocampus in rats with preoperative experience of a different pool. Rats with septal hippocampal lesions made after learning not only retained the focused search strategy that was acquired during preoperative training, but were also capable of rapid spatial learning in a second pool. This demonstrates that once spatial information has been acquired in one context, related new learning in a different context can be mediated by the temporal hippocampus, a result that challenges the widely held view that spatial memory is an exclusive function of the septal hippocampus.

Competitor suppresses neuronal representation of food reward in the nucleus accumbens/medial striatum of domestic chicks

To investigate the role of social contexts in controlling the neuronal representation of food reward, we recorded single neuron activity in the medial striatum/nucleus accumbens of domestic chicks and examined whether activities differed between two blocks with different contexts. Chicks were trained in an operant task to associate light-emitting diode color cues with three trial types that differed in the type of food reward: no reward (S-), a small reward/short-delay option (SS), and a large reward/long-delay alternative (LL). Amount and duration of reward were set such that both of SS and LL were chosen roughly equally. Neurons showing distinct cue-period activity in rewarding trials (SS and LL) were identified during an isolation block, and activity patterns were compared with those recorded from the same neuron during a subsequent pseudo-competition block in which another chick was allowed to forage in the same area, but was separated by a transparent window. In some neurons, cue-period activity was lower in the pseudo-competition block, and the difference was not ascribed to the number of repeated trials. Comparison at neuronal population level revealed statistically significant suppression in the pseudo-competition block in both SS and LL trials, suggesting that perceived competition generally suppressed the representation of cue-associated food reward. The delay- and reward-period activities, however, did not significantly different between blocks. These results demonstrate that visual perception of a competitive forager per se weakens the neuronal representation of predicted food reward. Possible functional links to impulse control are discussed.

Impact of appetitive and aversive outcomes on brain responses: linking the animal and human literatures

Decision-making is motivated by the possibility of obtaining reward and/or avoiding punishment. Though many have investigated behavior associated with appetitive or aversive outcomes, few have examined behaviors that rely on both. Fewer still have addressed questions related to how anticipated appetitive and aversive outcomes interact to alter neural signals related to expected value, motivation, and salience. Here we review recent rodent, monkey, and human research that address these issues. Further development of this area will be fundamental to understanding the etiology behind human psychiatric diseases and cultivating more effective treatments.

Ventral striatum lesions enhance stimulus and response encoding in dorsal striatum

BACKGROUND

The development of addiction is thought to reflect a transition from goal-directed to stimulus-response driven behavior, functions attributed to ventral (VS) and dorsal striatum (DS), respectively. In line with this theory, neuroadaptations that occur during prolonged drug use progress from VS to DS. Here we ask if VS dysfunction alone, independent of drug use, can affect neural selectivity in DS.

METHODS

To address this issue, we recorded from single neurons in DS while rats performed an odor-guided choice task for differently valued rewards in rats with and without unilateral VS lesions. In a separate group of animals, we used bilateral VS lesions to determine if VS was critical for performance on this task.

RESULTS

We describe data showing that unilateral lesions of VS enhance neural representations in DS during performance of a task that is dependent on VS. Furthermore, we show that VS is critical for reward-guided decision-making initially, but that rats regain function after several days.

CONCLUSIONS

These results suggest that loss of VS function, independent of chronic drug use, can trigger stronger encoding in DS in a reward-guided decision-making task and that the transition from VS to DS governed behavior observed in addiction might be due, in part, to initial loss of VS function.

Adenosine receptor control of cognition in normal and disease

Adenosine and adenosine receptors (ARs) are increasingly recognized as important therapeutic targets for controlling cognition under normal and disease conditions for its dual roles of neuromodulation as well as of homeostatic function in the brain. This chapter first presents the unique ability of adenosine, by acting on the inhibitory A1 and facilitating A2A receptor, to integrate dopamine, glutamate, and BNDF signaling and to modulate synaptic plasticity (e.g., long-term potentiation and long-term depression) in brain regions relevant to learning and memory, providing the molecular and cellular bases for adenosine receptor (AR) control of cognition. This led to the demonstration of AR modulation of social recognition memory, working memory, reference memory, reversal learning, goal-directed behavior/habit formation, Pavlovian fear conditioning, and effort-related behavior. Furthermore, human and animal studies support that AR activity can also, through cognitive enhancement and neuroprotection, reverse cognitive impairments in animal models of Alzheimer's disease (AD), Parkinson's disease (PD), Huntington's disease, and schizophrenia. Lastly, epidemiological evidence indicates that regular human consumption of caffeine, the most widely used psychoactive drug and nonselective AR antagonists, is associated with the reduced cognitive decline in aging and AD patients, and with the reduced risk in developing PD. Thus, there is a convergence of the molecular studies revealing AR as molecular targets for integrating neurotransmitter signaling and controlling synaptic plasticity, with animal studies demonstrating the strong procognitive impact upon AR antagonism in normal and disease brains and with epidemiological and clinical evidences in support of caffeine and AR drugs for therapeutic modulation of cognition. Since some of adenosine A2A receptor antagonists are already in phase III clinical trials for motor benefits in PD patients with remarkable safety profiles, additional animal and human studies to better understand the mechanism underlying the AR-mediated control of cognition under normal and disease conditions will provide the required rationale to stimulate the necessary clinical investigation to rapidly translate adenosine and AR drug as a novel strategy to control memory impairment in neuropsychiatric disorders.

Nucleus accumbens core lesions enhance two-way active avoidance

The majority of work examining the nucleus accumbens core (NAc) has focused on functions pertaining to behaviors guided by appetitive outcomes. These studies have pointed to the NAc as being critical for motivating behavior toward desirable outcomes. For example, we have recently shown that lesions of the NAc impaired performance on a reward-guided decision-making task that required rats to choose between differently valued rewards. Unfortunately, much less is known about the role that the NAc plays in motivating behavior when aversive outcomes are predicted. To address this issue we asked if NAc lesions impact performance on a two-way active avoidance task in which rats must learn to shuttle back and forth in a behavioral training box in order to avoid a footshock predicted by an auditory tone. Although bilateral NAc lesions initially impaired reward-guided decision-making, we found that the same lesions improved acquisition and retention of two-way active avoidance.

Understanding decision neuroscience: a multidisciplinary perspective and neural substrates

The neuroscience of decision making is a rapidly evolving multidisciplinary research area that employs neuroscientific techniques to explain various parameters associated with decision making behavior. In this chapter, we emphasize the role of multiple disciplines such as psychology, economics, neuroscience, and computational approaches in understanding the phenomenon of decision making. Further, we present a theoretical approach that suggests understanding the building blocks of decision making as bottom-up processes and integrate these with top-down modulatory factors. Relevant neurophysiological and neuroimaging findings that have used the building-block approach are reviewed. A unifying framework emphasizing multidisciplinary views would bring further insights into the active research area of decision making. Pointing to future directions for research, we focus on the role of computational approaches in such a unifying framework.

Nucleus accumbens responses differentiate execution and restraint in reward-directed behavior

Our behavior is powerfully driven by environmental cues that signal the availability of rewarding stimuli. We frequently encounter stimuli-a bowl of candy or an alert from our smartphone-that trigger actions to obtain those rewards, even though there may be positive outcomes associated with not acting. The inability to restrain one's action in the presence of reward-associated cues is one type of impulsive behavior and a component of such maladaptive behaviors as overeating, gambling, and substance abuse. The nucleus accumbens (NAc) is ideally situated to integrate multiple cognitive and affective inputs to bias action via outputs through the basal ganglia. NAc neurons have been shown to respond to cues that predict reward availability, goal-directed behaviors aimed at obtaining them, and delivery of the reward itself. As these processes are typically associated, it is difficult to discern whether signals in the NAc are more closely related to processing reward-predictive aspects of goal-directed behavior or selection of behavioral response. To dissociate these possibilities, we recorded the activity of NAc neurons while rats performed a task in which two different cues both informed rats of reward availability but required them to either press a lever (Go) or withhold pressing (NoGo) to obtain the reward. Individual cue-responsive neurons showed either increases or decreases in activity at cue onset. Increases in activity were larger, and decreases smaller, when rats withheld lever pressing, whether correctly for NoGo trials or in error on Go trials. Thus NAc cue responses correlated with action, regardless of cue type or accuracy.

A network state linking motivation and action in the nucleus accumbens

In this issue of Neuron, McGinty et al. (2013) report that the activity of cue-responsive neurons in the rat nucleus accumbens predicts the vigor of the subsequent approach movement. The characterization of this network state between motivation and action lends novel mechanistic insight to a spectrum of cue-elicited behaviors.