Overriding Phasic Dopamine Signals Redirects Action Selection during Risk/Reward Decision Making

Identification of the subventricular tegmental nucleus as brainstem reward center

Rewards are essential for motivation, decision-making, memory, and mental health. We identified the subventricular tegmental nucleus (SVTg) as a brainstem reward center. In mice, reward and its prediction activate the SVTg, and SVTg stimulation leads to place preference, reduced anxiety, and accumbal dopamine release. Mice self-stimulate the SVTg, which can also be activated directly by the neocortex, resulting in effective inhibition of the lateral habenula, a region associated with depression. This mechanism may also explain why SVTg suppression induces aversion and increases fear. The translational relevance of these findings is supported by evidence in the rat, monkey, and human brainstem, establishing SVTg as a key hub for reward processing, emotional valence, and motivation.

Serotonin and Effort-Based Decision-Making: Dissociating Behavioral Effects of 8-OH-DPAT and PCPA

Effort-based decision-making is particularly relevant to psychiatric conditions where motivation deficits are prominent features. Despite its clinical significance, the neurochemical mechanisms of this cognitive process remain unclarified. This study explores the impact of serotonin synthesis inhibition (PCPA) and modulation of serotonin release and 5-HT1A receptor agonism (8-OH-DPAT) on effort-based decision-making in rats. Adult male rats were trained in a modified T-maze task where they could obtain a high reward for climbing a mesh barrier or a low reward for no extra effort. Following training, rats received either acute 8-OH-DPAT treatment or subchronic PCPA treatment and were tested on their choices between high- and low-effort arms. The goal-arm choices and goal-arm entrance latencies were recorded. Next, homovanillic acid and 5-hydroxyindoleacetic acid, metabolites of dopamine and serotonin, respectively, were quantified in the rats' prefrontal cortex, striatum, and hippocampus. 8-OH-DPAT significantly increased low-effort, low-reward choices and increased goal-arm latency. In contrast, PCPA treatment did not affect these measures. Both PCPA and 8-OH-DPAT significantly decreased 5-hydroxyindoleacetic acid levels in the prefrontal cortex and the hippocampus. 8-OH-DPAT treatment was also associated with decreased homovanillic acid levels in the hippocampus. Our findings suggest that the overall reduction of serotonin levels alone does not affect effort-based decision-making and highlights the possible role of the hippocampus and the 5-HT1A receptor in this cognitive process.

Model of a striatal circuit exploring biological mechanisms underlying decision-making during normal and disordered states

Decision-making requires continuous adaptation to internal and external contexts. Changes in decision-making are reliable transdiagnostic symptoms of neuropsychiatric disorders. We created a computational model demonstrating how the striosome compartment of the striatum constructs a mathematical space for decision-making computations depending on context, and how the matrix compartment defines action value depending on the space. The model explains multiple experimental results and unifies other theories like reward prediction error, roles of the direct versus indirect pathways, and roles of the striosome versus matrix, under one framework. We also found, through new analyses, that striosome and matrix neurons increase their synchrony during difficult tasks, caused by a necessary increase in dimensionality of the space. The model makes testable predictions about individual differences in disorder susceptibility, decision-making symptoms shared among neuropsychiatric disorders, and differences in neuropsychiatric disorder symptom presentation. The model reframes the role of the striosomal circuit in neuroeconomic and disorder-affected decision-making.

Highlights

Striosomes prioritize decision-related data used by matrix to set action values. Striosomes and matrix have different roles in the direct and indirect pathways. Abnormal information organization/valuation alters disorder presentation. Variance in data prioritization may explain individual differences in disorders.

eTOC

Beck et al. developed a computational model of how a striatal circuit functions during decision-making. The model unifies and extends theories about the direct versus indirect pathways. It further suggests how aberrant circuit function underlies decision-making phenomena observed in neuropsychiatric disorders.

Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.

The lateral habenula: A hub for value-guided behavior

The habenula is an evolutionarily highly conserved diencephalic brain region divided into two major parts, medial and lateral. Over the past two decades, studies of the lateral habenula (LHb), in particular, have identified key functions in value-guided behavior in health and disease. In this review, we focus on recent insights into LHb connectivity and its functional relevance for different types of aversive and appetitive value-guided behavior. First, we give an overview of the anatomical organization of the LHb and its main cellular composition. Next, we elaborate on how distinct LHb neuronal subpopulations encode aversive and appetitive stimuli and on their involvement in more complex decision-making processes. Finally, we scrutinize the afferent and efferent connections of the LHb and discuss their functional implications for LHb-dependent behavior. A deepened understanding of distinct LHb circuit components will substantially contribute to our knowledge of value-guided behavior.

Reward Circuit Local Field Potential Modulations Precede Risk Taking

Risk taking behavior is a symptom of multiple neuropsychiatric disorders and often lacks effective treatments. Reward circuitry regions including the amygdala, orbitofrontal cortex, insula, and anterior cingulate have been implicated in risk-taking by neuroimaging studies. Electrophysiological activity associated with risk taking in these regions is not well understood in humans. Further characterizing the neural signalling that underlies risk-taking may provide therapeutic insight into disorders associated with risk-taking. Eleven patients with pharmacoresistant epilepsy who underwent stereotactic electroencephalography with electrodes in the amygdala, orbitofrontal cortex, insula, and/or anterior cingulate participated. Patients participated in a gambling task where they wagered on a visible playing card being higher than a hidden card, betting $5 or $20 on this outcome, while local field potentials were recorded from implanted electrodes. We used cluster-based permutation testing to identify reward prediction error signals by comparing oscillatory power following unexpected and expected rewards. We also used cluster-based permutation testing to compare power preceding high and low bets in high-risk (<50% chance of winning) trials and two-way ANOVA with bet and risk level to identify signals associated with risky, risk averse, and optimized decisions. We used linear mixed effects models to evaluate the relationship between reward prediction error and risky decision signals across trials, and a linear regression model for associations between risky decision signal power and Barratt Impulsiveness Scale scores for each patient. Reward prediction error signals were identified in the amygdala (p=0.0066), anterior cingulate (p=0.0092), and orbitofrontal cortex (p=6.0E-4, p=4.0E-4). Risky decisions were predicted by increased oscillatory power in high-gamma frequency range during card presentation in the orbitofrontal cortex (p=0.0022), and by increased power following bet cue presentation across the theta-to-beta range in the orbitofrontal cortex ( p =0.0022), high-gamma in the anterior cingulate ( p =0.0004), and high-gamma in the insula ( p =0.0014). Risk averse decisions were predicted by decreased orbitofrontal cortex gamma power ( p =2.0E-4). Optimized decisions that maximized earnings were preceded by decreases within the theta to beta range in orbitofrontal cortex ( p =2.0E-4), broad frequencies in amygdala ( p =2.0E-4), and theta to low-gamma in insula ( p =4.0E-4). Insula risky decision power was associated with orbitofrontal cortex high-gamma reward prediction error signal ( p =0.0048) and with patient impulsivity ( p =0.00478). Our findings identify and help characterize reward circuitry activity predictive of risk-taking in humans. These findings may serve as potential biomarkers to inform the development of novel treatment strategies such as closed loop neuromodulation for disorders of risk taking.

Dopamine activity in the nigrostriatal pathway alters cue-induced risky choice patterns in female rats

Deficits in cost/benefit decision making is a critical risk factor for gambling disorder. Reward-paired cues may play an important role, as these stimuli can enhance risk preference in rats. Despite extensive research implicating the dorsal striatum in the compulsive aspects of addiction, the role of nigrostriatal dopaminergic activity in cue-induced risk preference remains unclear, particularly in females. Accordingly, we examined the effects of manipulating the dopaminergic nigrostriatal pathway on cue-induced risky choice in female rats. TH:Cre rats were trained on the cued version of the rat Gambling Task. This task was designed such that maximal reward is attained by avoiding the high-risk, high-reward options and instead favouring the options associated with lower per-trial gains, as they feature less frequent and shorter time-out penalties. Adding reward-paired audiovisual cues to the task leads to greater risky choice on average. To assess the role of the nigrostriatal pathway, a viral vector carrying either Cre-dependent inhibitory or excitatory DREADD was infused into the substantia nigra. Rats then received clozapine-N-oxide either during task acquisition or after a stable performance baseline was reached. Inhibition of this pathway accelerated the development of risk preference in early sessions and increased risky choice during performance, but long-term inhibition actually improved decision making. Activation of this pathway had minimal effects. These results provide evidence for the involvement of the dopaminergic nigrostriatal pathway in cue-induced risk preference in females, therefore shedding light on its role in cost/benefit decision-making deficits and expanding our knowledge of the female dopaminergic system.

AKAP150-anchored PKA regulates synaptic transmission and plasticity, neuronal excitability and CRF neuromodulation in the mouse lateral habenula

The scaffolding A-kinase anchoring protein 150 (AKAP150) is critically involved in kinase and phosphatase regulation of synaptic transmission/plasticity, and neuronal excitability. Emerging evidence also suggests that AKAP150 signaling may play a key role in brain's processing of rewarding/aversive experiences, however its role in the lateral habenula (LHb, as an important brain reward circuitry) is completely unknown. Using whole cell patch clamp recordings in LHb of male wildtype and ΔPKA knockin mice (with deficiency in AKAP-anchoring of PKA), here we show that the genetic disruption of PKA anchoring to AKAP150 significantly reduces AMPA receptor-mediated glutamatergic transmission and prevents the induction of presynaptic endocannabinoid-mediated long-term depression in LHb neurons. Moreover, ΔPKA mutation potentiates GABAA receptor-mediated inhibitory transmission while increasing LHb intrinsic excitability through suppression of medium afterhyperpolarizations. ΔPKA mutation-induced suppression of medium afterhyperpolarizations also blunts the synaptic and neuroexcitatory actions of the stress neuromodulator, corticotropin releasing factor (CRF), in mouse LHb. Altogether, our data suggest that AKAP150 complex signaling plays a critical role in regulation of AMPA and GABAA receptor synaptic strength, glutamatergic plasticity and CRF neuromodulation possibly through AMPA receptor and potassium channel trafficking and endocannabinoid signaling within the LHb.

Win-Paired Cues Modulate the Effect of Dopamine Neuron Sensitization on Decision Making and Cocaine Self-administration: Divergent Effects Across Sex

BACKGROUND

Both psychostimulant use and engagement with probabilistic schedules of reward sensitize the mesocorticolimbic dopamine (DA) system. Such behaviors may act synergistically to explain the high comorbidity between stimulant use and gambling disorder. The salient audiovisual stimuli of modern electronic gambling may exacerbate the situation.

METHODS

To probe these interactions, we sensitized ventral tegmental area DA neurons via chronic chemogenetic stimulation while rats (n = 134) learned a rat gambling task in the presence or absence of casino-like cues. The same rats then learned to self-administer cocaine. In a separate cohort (n = 25), we confirmed that our chemogenetic methods sensitized the locomotor response to cocaine and potentiated phasic excitability of ventral tegmental area DA neurons through in vivo electrophysiological recordings.

RESULTS

In the absence of cues, sensitization promoted risk taking in both sexes. When rewards were cued, sensitization expedited the development of a risk-preferring phenotype in males while attenuating cue-induced risk taking in females.

CONCLUSIONS

While these results provide further confirmation that ventral tegmental area DA neurons critically modulate risky decision making, they also reveal stark sex differences in the decisional impact that dopaminergic signals exert when winning outcomes are cued. As previously observed, risky decision making on the cued rat gambling task increased as both males and females learned to self-administer cocaine. The combination of DA sensitization and win-paired cues while gambling led to significantly greater cocaine taking, but these rats did not show any increase in risky choice as a result. Therefore, cocaine and heavily cued gambles may partially substitute for each other once the DA system has been rendered labile through sensitization, thereby compounding addiction risk across modalities.

A neural mechanism for conserved value computations integrating information and rewards

Behavioral and economic theory dictate that we decide between options based on their values. However, humans and animals eagerly seek information about uncertain future rewards, even when this does not provide any objective value. This implies that decisions are made by endowing information with subjective value and integrating it with the value of extrinsic rewards, but the mechanism is unknown. Here, we show that human and monkey value judgements obey strikingly conserved computational principles during multi-attribute decisions trading off information and extrinsic reward. We then identify a neural substrate in a highly conserved ancient structure, the lateral habenula (LHb). LHb neurons signal subjective value, integrating information's value with extrinsic rewards, and the LHb predicts and causally influences ongoing decisions. Neurons in key input areas to the LHb largely signal components of these computations, not integrated value signals. Thus, our data uncover neural mechanisms of conserved computations underlying decisions to seek information about the future.

Sensory Reinforced Corticostriatal Plasticity

BACKGROUND

Regional changes in corticostriatal transmission induced by phasic dopaminergic signals are an essential feature of the neural network responsible for instrumental reinforcement during discovery of an action. However, the timing of signals that are thought to contribute to the induction of corticostriatal plasticity is difficult to reconcile within the framework of behavioural reinforcement learning, because the reinforcer is normally delayed relative to the selection and execution of causally-related actions.

OBJECTIVE

While recent studies have started to address the relevance of delayed reinforcement signals and their impact on corticostriatal processing, our objective was to establish a model in which a sensory reinforcer triggers appropriately delayed reinforcement signals relayed to the striatum via intact neuronal pathways and to investigate the effects on corticostriatal plasticity.

METHODS

We measured corticostriatal plasticity with electrophysiological recordings using a light flash as a natural sensory reinforcer, and pharmacological manipulations were applied in an in vivo anesthetized rat model preparation.

RESULTS

We demonstrate that the spiking of striatal neurons evoked by single-pulse stimulation of the motor cortex can be potentiated by a natural sensory reinforcer, operating through intact afferent pathways, with signal timing approximating that required for behavioural reinforcement. The pharmacological blockade of dopamine receptors attenuated the observed potentiation of corticostriatal neurotransmission.

CONCLUSION

This novel in vivo model of corticostriatal plasticity offers a behaviourally relevant framework to address the physiological, anatomical, cellular, and molecular bases of instrumental reinforcement learning.

AKAP150-anchored PKA regulation of synaptic transmission and plasticity, neuronal excitability and CRF neuromodulation in the lateral habenula

Numerous studies of hippocampal synaptic function in learning and memory have established the functional significance of the scaffolding A-kinase anchoring protein 150 (AKAP150) in kinase and phosphatase regulation of synaptic receptor and ion channel trafficking/function and hence synaptic transmission/plasticity, and neuronal excitability. Emerging evidence also suggests that AKAP150 signaling may play a critical role in brain's processing of rewarding/aversive experiences. Here we focused on an unexplored role of AKAP150 in the lateral habenula (LHb), a diencephalic brain region that integrates and relays negative reward signals from forebrain striatal and limbic structures to midbrain monoaminergic centers. LHb aberrant activity (specifically hyperactivity) is also linked to depression. Using whole cell patch clamp recordings in LHb of male wildtype (WT) and ΔPKA knockin mice (with deficiency in AKAP-anchoring of PKA), we found that the genetic disruption of PKA anchoring to AKAP150 significantly reduced AMPA receptor (AMPAR)-mediated glutamatergic transmission and prevented the induction of presynaptic endocannabinoid (eCB)-mediated long-term depression (LTD) in LHb neurons. Moreover, ΔPKA mutation potentiated GABAA receptor (GABAAR)-mediated inhibitory transmission postsynaptically while increasing LHb intrinsic neuronal excitability through suppression of medium afterhyperpolarizations (mAHPs). Given that LHb is a highly stress-responsive brain region, we further tested the effects of corticotropin releasing factor (CRF) stress neuromodulator on synaptic transmission and intrinsic excitability of LHb neurons in WT and ΔPKA mice. As in our earlier study in rat LHb, CRF significantly suppressed GABAergic transmission onto LHb neurons and increased intrinsic excitability by diminishing small-conductance potassium (SK) channel-mediated mAHPs. ΔPKA mutation-induced suppression of mAHPs also blunted the synaptic and neuroexcitatory actions of CRF in mouse LHb. Altogether, our data suggest that AKAP150 complex signaling plays a critical role in regulation of AMPAR and GABAAR synaptic strength, glutamatergic plasticity and CRF neuromodulation possibly through AMPAR and potassium channel trafficking and eCB signaling within the LHb.

Dynamics of Lateral Habenula-Ventral Tegmental Area Microcircuit on Pain-Related Cognitive Dysfunctions

Chronic pain is a health problem that affects the ability to work and perform other activities, and it generally worsens over time. Understanding the complex pain interaction with brain circuits could help predict which patients are at risk of developing central dysfunctions. Increasing evidence from preclinical and clinical studies suggests that aberrant activity of the lateral habenula (LHb) is associated with depressive symptoms characterized by excessive negative focus, leading to high-level cognitive dysfunctions. The primary output region of the LHb is the ventral tegmental area (VTA), through a bidirectional connection. Recently, there has been growing interest in the complex interactions between the LHb and VTA, particularly regarding their crucial roles in behavior regulation and their potential involvement in the pathological impact of chronic pain on cognitive functions. In this review, we briefly discuss the structural and functional roles of the LHb-VTA microcircuit and their impact on cognition and mood disorders in order to support future studies addressing brain plasticity during chronic pain conditions.

Not all discounts are created equal: Regional activity and brain networks in temporal and effort discounting

Reward outcomes associated with costs like time delay and effort investment are generally discounted in decision-making. Standard economic models predict rewards associated with different types of costs are devalued in a similar manner. However, our review of rodent lesion studies indicated partial dissociations between brain regions supporting temporal- and effort-based decision-making. Another debate is whether options involving low and high costs are processed in different brain substrates (dual-system) or in the same regions (single-system). This research addressed these issues using coordinate-based, connectivity-based, and activation network-based meta-analyses to identify overlapping and separable neural systems supporting temporal (39 studies) and effort (20 studies) discounting. Coordinate-based activation likelihood estimation and resting-state connectivity analyses showed immediate-small reward and delayed-large reward choices engaged distinct regions with unique connectivity profiles, but their activation network mapping was found to engage the default mode network. For effort discounting, salience and sensorimotor networks supported low-effort choices, while the frontoparietal network supported high-effort choices. There was little overlap between the temporal and effort networks. Our findings underscore the importance of differentiating different types of costs in decision-making and understanding discounting at both regional and network levels.

Dorsal raphe neurons signal integrated value during multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.

Risk taking for potential losses but not gains increases with time of day

Humans exhibit distinct risk preferences when facing choices involving potential gains and losses. These preferences are believed to be subject to neuromodulatory influence, particularly from dopamine and serotonin. As neuromodulators manifest circadian rhythms, this suggests decision making under risk might be affected by time of day. Here, in a large subject sample collected using a smartphone application, we found that risky options with potential losses were increasingly chosen over the course of the day. We observed this result in both a within-subjects design (N = 2599) comparing risky options chosen earlier and later in the day in the same individuals, and in a between-subjects design (N = 26,720) showing our effect generalizes across ages and genders. Using computational modelling, we show this diurnal change in risk preference reflects a decrease in sensitivity to increasing losses, but no change was observed in the relative impacts of gains and losses on choice (i.e., loss aversion). Thus, our findings reveal a striking diurnal modulation in human decision making, a pattern with potential importance for real-life decisions that include voting, medical decisions, and financial investments.

A neurocognitive perspective on the relationship between exercise and reward: Implications for weight management and drug addiction

The impact of exercise on food reward is increasingly being discussed as an interplay between executive function (EF), homeostasis and mechanisms promoting or undermining intentional behaviour change. Integrating current knowledge of neurocognitive processes encompassing cognitive and affective networks within an energy balance framework will provide a more comprehensive account. Reward circuitry affected by recreational drugs and food overlap. Therefore the underlying processes explaining changes in drug-taking behaviour may offer new insights into how exercise affects the reward value of recreational drugs and food. EF is important for successful self-regulation, and training EF may boost inhibitory control in relation to food- and drug-related reward. Preclinical and clinical observations suggest that reward-seeking can transfer within and between categories of reward. This may have clinical implications beyond exercise improving metabolic health in people with obesity to understanding therapeutic responses to exercise in people with neurocognitive deficits in non-food reward-based decision making such as drug dependence.

Temporal Dynamics Underlying Prelimbic Prefrontal Cortical Regulation of Action Selection and Outcome Evaluation during Risk/Reward Decision-Making

Risk/reward decision-making is a dynamic process that includes periods of deliberation before action selection and evaluation of the action outcomes that bias subsequent choices. Inactivation of the prelimbic (PL) cortex has revealed its integral role in updating decision biases in the face of changes in probabilistic reward contingencies, yet how phasic PL signals during different phases of the decision process influence choice remains unclear. We used temporally specific optogenetic inhibition to selectively disrupt PL activity coinciding with action selection and outcome phases to examine how these signals influence choice. Male rats expressing the inhibitory opsin eArchT within PL excitatory neurons were well trained on a probabilistic discounting task, entailing choice between small/certain versus large/risky rewards, the probability of which varied over a session (50-12.5%). During testing, brief light pulses suppressed PL activity before choice or after different outcomes. Prechoice suppression reduced bias toward more preferred/higher utility options and disrupted how recent outcomes influenced subsequent choice. Inhibition during risky losses induced a similar profile, but here, the impact of reward omissions were either amplified or diminished, relative to the context of the estimated profitability of the risky option. Inhibition during large or small reward receipt reduced risky choice when this option was more profitable, suggesting these signals can both reinforce rewarded risky choices and also act as a relative value comparator signal that augments incentive for larger rewards. These findings reveal multifaceted contributions by the PL in implementing decisions and integrating action-outcome feedback to assign context to the decision space.SIGNIFICANCE STATEMENT The PL prefrontal cortex plays an integral role in guiding risk/reward decisions, but how activity in this region during different phases of the decision process influences choice is unclear. By using temporally specific optogenetic manipulations of this activity, the present study unveiled previously uncharacterized and differential contributions by PL in implementing decision policies and how evaluation of decision outcomes shape subsequent choice. These findings provide novel insight into the dynamic processes engaged by the PL that underlie action selection in situations involving reward uncertainty that may aid in understanding the mechanism underlying normal and aberrant decision-making processes.

The effects of early life stress on impulsivity

Elevated impulsivity is a symptom shared by various psychiatric disorders such as substance use disorder, bipolar disorder, and attention-deficit/hyperactivity disorder. However, impulsivity is not a unitary construct and impulsive behaviors fall into two subcategories: impulsive action and impulsive choice. Impulsive choice refers to the tendency to prefer immediate, small rewards over delayed, large rewards, whereas impulsive action involves difficulty inhibiting rash, premature, or mistimed behaviors. These behaviors are mediated by the mesocorticolimbic dopamine (DA) system, which consists of projections from the ventral tegmental area to the nucleus accumbens and prefrontal cortex. Early life stress (ELS) alters both impulsive choice and impulsive action in rodents. ELS also changes DA receptor expression, transmission, and activity within the mesocorticolimbic system. This review integrates the dopamine, impulsivity, and ELS literature to provide evidence that ELS alters impulsivity via inducing changes in the mesocorticolimbic DA system. Understanding how ELS affects brain circuits associated with impulsivity can help advance treatments aimed towards reducing impulsivity symptoms in a variety of psychiatric disorders.

Social Determinants of Inter-Individual Variability and Vulnerability: The Role of Dopamine

Individuals differ in their traits and preferences, which shape their interactions, their prospects for survival and their susceptibility to diseases. These correlations are well documented, yet the neurophysiological mechanisms underlying the emergence of distinct personalities and their relation to vulnerability to diseases are poorly understood. Social ties, in particular, are thought to be major modulators of personality traits and psychiatric vulnerability, yet the majority of neuroscience studies are performed on rodents in socially impoverished conditions. Rodent micro-society paradigms are therefore key experimental paradigms to understand how social life generates diversity by shaping individual traits. Dopamine circuitry is implicated at the interface between social life experiences, the expression of essential traits, and the emergence of pathologies, thus proving a possible mechanism to link these three concepts at a neuromodulatory level. Evaluating inter-individual variability in automated social testing environments shows great promise for improving our understanding of the link between social life, personality, and precision psychiatry – as well as elucidating the underlying neurophysiological mechanisms.

Fos response of the tail of the ventral tegmental area to food restriction entails a prediction error processing

The tail of the ventral tegmental area (tVTA) or rostromedial tegmental nucleus (RMTg) receives lateral habenula inputs and projects heavily to midbrain dopamine neurons. Midbrain dopamine and lateral habenula neurons participate in learning processes predicting the outcomes of actions, placing the tVTA in a critical location into prediction error pathways. tVTA GABA neurons show electrophysiological inhibition or activation after reward and aversive stimuli, respectively, and their predictive cues. tVTA molecular recruitment, however, is not elicited by all aversive stimuli. Indeed, precipitated opioid withdrawal, repeated footshocks or food restriction raise tVTA Fos expression, whereas various other unpleasant, stressful or painful stimuli does not elicit that molecular response. However, the basis of that difference remains unknown. In the present study, we tried to disentangle whether the tVTA c-Fos induction observed after food restriction was due to the aversive state of food restriction or to procedure-related reward prediction error. To this end, male Sprague-Dawley rats were food-restricted for 7-8 days. During this period, animals were handled and weighed every day before feeding. On the test day, rats underwent several behavioral procedures to explore the impact of food restriction and food-predictive cue exposure on tVTA c-Fos expression. We showed that food restriction per se was not able to recruit c-Fos in the tVTA. On the contrary, the food-predicting cues induced c-Fos locally in the absence of feeding, whereas the food-predicting cues followed by feeding evoked lower c-Fos expression. Overall, our results support the proposed involvement of the tVTA in reward prediction error.

The role of orphan receptor GPR139 in neuropsychiatric behavior

Orphan G protein Coupled Receptors (GPCRs) present attractive targets both for understanding neuropsychiatric diseases and for development of novel therapeutics. GPR139 is an orphan GPCR expressed in select brain circuits involved in controlling movement, motivation and reward. It has been linked to the opioid and dopamine neuromodulatory systems; however, its role in animal behavior and neuropsychiatric processes is poorly understood. Here we present a comprehensive behavioral characterization of a mouse model with a GPR139 null mutation. We show that loss of GPR139 in mice results in delayed onset hyperactivity and prominent neuropsychiatric manifestations including elevated stereotypy, increased anxiety-related traits, delayed acquisition of operant responsiveness, disruption of cued fear conditioning and social interaction deficits. Furthermore, mice lacking GPR139 exhibited complete loss of pre-pulse inhibition and developed spontaneous 'hallucinogenic' head-twitches, altogether suggesting schizophrenia-like symptomatology. Remarkably, a number of these behavioral deficits could be rescued by the administration of μ-opioid and D2 dopamine receptor (D2R) antagonists: naltrexone and haloperidol, respectively, suggesting that loss of neuropsychiatric manifestations in mice lacking GPR139 are driven by opioidergic and dopaminergic hyper-functionality. The inhibitory influence of GPR139 on D2R signaling was confirmed in cell-based functional assays. These observations define the role of GPR139 in controlling behavior and implicate in vivo actions of this receptor in the neuropsychiatric process with schizophrenia-like pathology.

Circadian Influences on the Habenula and Their Potential Contribution to Neuropsychiatric Disorders

The neural circadian system consists of the master circadian clock in the hypothalamic suprachiasmatic nuclei (SCN) communicating time of day cues to the rest of the body including other brain areas that also rhythmically express circadian clock genes. Over the past 16 years, evidence has emerged to indicate that the habenula of the epithalamus is a candidate extra-SCN circadian oscillator. When isolated from the SCN, the habenula sustains rhythms in clock gene expression and neuronal activity, with the lateral habenula expressing more robust rhythms than the adjacent medial habenula. The lateral habenula is responsive to putative SCN output factors as well as light information conveyed to the perihabenula area. Neuronal activity in the lateral habenula is altered in depression and intriguingly disruptions in circadian rhythms can elevate risk of developing mental health disorders including depression. In this review, we will principally focus on how circadian and light signals affect the lateral habenula and evaluate the possibility that alteration in these influences contribute to mental health disorders.

Reward and aversion encoding in the lateral habenula for innate and learned behaviours

Throughout life, individuals experience a vast array of positive and aversive events that trigger adaptive behavioural responses. These events are often unpredicted and engage actions that are likely anchored on innate behavioural programs expressed by each individual member of virtually all animal species. In a second step, environmental cues, that are initially neutral, acquire value through the association with external sensory stimuli, and become instrumental to predict upcoming positive or negative events. This process ultimately prompts learned goal-directed actions allowing the pursuit of rewarding experience or the avoidance of a danger. Both innate and learned behavioural programs are evolutionarily conserved and fundamental for survival. Among the brain structures participating in the encoding of positive/negative stimuli and contributing to innate and learned behaviours is the epithalamic lateral habenula (LHb). The LHb provides top-down control of monoaminergic systems, responds to unexpected appetitive/aversive stimuli as well as external cues that predict the upcoming rewards or punishments. Accordingly, the LHb controls a number of behaviours that are innate (originating from unpredicted stimuli), and learned (stemming from predictive cues). In this review, we will discuss the progresses that rodent's experimental work made in identifying how LHb activity governs these vital processes, and we will provide a view on how these findings integrate within a complex circuit connectivity.

Update on GPCR-based targets for the development of novel antidepressants

Traditional antidepressants largely interfere with monoaminergic transport or degradation systems, taking several weeks to have their therapeutic actions. Moreover, a large proportion of depressed patients are resistant to these therapies. Several atypical antidepressants have been developed which interact with G protein coupled receptors (GPCRs) instead, as direct targeting of receptors may achieve more efficacious and faster antidepressant actions. The focus of this review is to provide an update on how distinct GPCRs mediate antidepressant actions and discuss recent insights into how GPCRs regulate the pathophysiology of Major Depressive Disorder (MDD). We also discuss the therapeutic potential of novel GPCR targets, which are appealing due to their ligand selectivity, expression pattern, or pharmacological profiles. Finally, we highlight recent advances in understanding GPCR pharmacology and structure, and how they may provide new avenues for drug development.

Dopamine Circuit Mechanisms of Addiction-Like Behaviors

Addiction is a complex disease that impacts millions of people around the world. Clinically, addiction is formalized as substance use disorder (SUD), with three primary symptom categories: exaggerated substance use, social or lifestyle impairment, and risky substance use. Considerable efforts have been made to model features of these criteria in non-human animal research subjects, for insight into the underlying neurobiological mechanisms. Here we review evidence from rodent models of SUD-inspired criteria, focusing on the role of the striatal dopamine system. We identify distinct mesostriatal and nigrostriatal dopamine circuit functions in behavioral outcomes that are relevant to addictions and SUDs. This work suggests that striatal dopamine is essential for not only positive symptom features of SUDs, such as elevated intake and craving, but also for impairments in decision making that underlie compulsive behavior, reduced sociality, and risk taking. Understanding the functional heterogeneity of the dopamine system and related networks can offer insight into this complex symptomatology and may lead to more targeted treatments.

Sign tracking predicts suboptimal behavior in a rodent gambling task

RATIONALE

Reward-associated cues can promote maladaptive behavior, including risky decision-making in a gambling setting. A propensity for sign tracking over goal tracking-i.e., interaction with a reward-predictive cue rather than the site of reward-demonstrates an individual's tendency to transfer motivational value to a cue. However, the relationship of sign tracking to risky decision-making remains unclear.

OBJECTIVES

To determine whether sign tracking predicts risky choice, we used a Pavlovian conditioned approach task to evaluate the tendency of male rats to sign track to a lever cue and then trained rats on a rodent gambling task (rGT) with win-associated cues. We also tested the effects of D-amphetamine, quinpirole (a D₂/D₃ receptor agonist), and PD128907 (a D₃ receptor agonist) on gambling behavior in sign tracker and goal tracker individuals.

RESULTS

Increased sign tracking relative to goal tracking was associated with suboptimal performance on the rGT, including decreased selection of the optimal choice, increased selection of a high-risk/high-reward option, and increased impulsive premature choices. Amphetamine increased choices of a low-risk/low-reward option at the expense of optimal and high-risk choices, whereas quinpirole and PD128907 had little effect on choice allocation, but reduced impulsivity. Drug effects were similar across sign tracker and goal tracker individuals.

CONCLUSIONS

Cue reactivity, as measured by sign tracking, is predictive and may be an important driver of risky and impulsive choices in a gambling setting laden with salient audiovisual cues. Evaluating an individual's sign tracking behavior may be an avenue to predict vulnerability to pathological gambling and the efficacy of treatments.

Looking beneath the surface: the importance of subcortical structures in frontotemporal dementia

Whilst initial anatomical studies of frontotemporal dementia focussed on cortical involvement, the relevance of subcortical structures to the pathophysiology of frontotemporal dementia has been increasingly recognized over recent years. Key structures affected include the caudate, putamen, nucleus accumbens, and globus pallidus within the basal ganglia, the hippocampus and amygdala within the medial temporal lobe, the basal forebrain, and the diencephalon structures of the thalamus, hypothalamus and habenula. At the most posterior aspect of the brain, focal involvement of brainstem and cerebellum has recently also been shown in certain subtypes of frontotemporal dementia. Many of the neuroimaging studies on subcortical structures in frontotemporal dementia have been performed in clinically defined sporadic cases. However, investigations of genetically- and pathologically-confirmed forms of frontotemporal dementia are increasingly common and provide molecular specificity to the changes observed. Furthermore, detailed analyses of sub-nuclei and subregions within each subcortical structure are being added to the literature, allowing refinement of the patterns of subcortical involvement. This review focuses on the existing literature on structural imaging and neuropathological studies of subcortical anatomy across the spectrum of frontotemporal dementia, along with investigations of brain–behaviour correlates that examine the cognitive sequelae of specific subcortical involvement: it aims to ‘look beneath the surface’ and summarize the patterns of subcortical involvement have been described in frontotemporal dementia.

Advances in understanding meso-cortico-limbic-striatal systems mediating risky reward seeking

The risk of an aversive consequence occurring as the result of a reward-seeking action can have a profound effect on subsequent behavior. Such aversive events can be described as punishers, as they decrease the probability that the same action will be produced again in the future and increase the exploration of less risky alternatives. Punishment can involve the omission of an expected rewarding event ("negative" punishment) or the addition of an unpleasant event ("positive" punishment). Although many individuals adaptively navigate situations associated with the risk of negative or positive punishment, those suffering from substance use disorders or behavioral addictions tend to be less able to curtail addictive behaviors despite the aversive consequences associated with them. Here, we discuss the psychological processes underpinning reward seeking despite the risk of negative and positive punishment and consider how behavioral assays in animals have been employed to provide insights into the neural mechanisms underlying addictive disorders. We then review the critical contributions of dopamine signaling to punishment learning and risky reward seeking, and address the roles of interconnected ventral striatal, cortical, and amygdala regions to these processes. We conclude by discussing the ample opportunities for future study to clarify critical gaps in the literature, particularly as related to delineating neural contributions to distinct phases of the risky decision-making process.

Dopamine receptors regulate preference between high-effort and high-risk rewards

RATIONALE

Optimal decision-making necessitates evaluation of multiple rewards that are each offset by distinct costs, such as high effort requirement or high risk of failure. The neurotransmitter dopamine is fundamental toward these cost-benefit analyses, and D1-like and D2-like dopamine receptors differently modulate the reward-discounting effects of both effort and risk. However, measuring the role of dopamine in regulating decision-making between options associated with distinct costs exceeds the scope of traditional rodent economic decision-making paradigms.

OBJECTIVES

We developed the effort vs probability economic conflict task (EvP) to model multimodal economic decision-making in rats. This task measures choice between two rewards of uniform magnitude associated with either a high effort requirement or risk of reward omission. We then tested the modulatory effects of systemic cocaine and D1/D2 blockade or activation on the preference between high-effort and high-risk alternatives.

METHODS

In the EvP, two reinforcers of equal magnitude are associated with either (1) an effort requirement that increases throughout the session (1, 5, 10, and 20 lever presses), or (2) a low probability of reward receipt (25% of probabilistic choices). Critically, the reinforcer for each choice is comparable (one pellet), which eliminates the influence of magnitude discrimination on the decision-making process. After establishing the task, the dopamine transporter blocker cocaine and D1/D2 antagonists and agonists were administered prior to EvP performance.

RESULTS

Preference shifted away from either effortful or probabilistic choice when either option became more costly, and this preference was highly variable between subjects and stable over time. Cocaine, D1 activation, and D2 blockade produced limited, dose-dependent shifts in choice preference contingent on high or low effort conditions. In contrast, D2 activation across multiple doses evoked a robust shift from effortful to risky choice that was evident even when clearly disadvantageous.

CONCLUSIONS

The EvP clearly demonstrates that rats can evaluate distinct effortful or risky costs associated with rewards of comparable magnitude, and shift preference away from either option with increasing cost. This preference is more tightly linked to D2 than D1 receptor manipulation, suggesting D2-like receptors as a possible therapeutic target for maladaptive biases toward risk-taking over effort.

Eating to dare - Nutrition impacts human risky decision and related brain function

Macronutrient composition modulates plasma amino acids that are precursors of neurotransmitters and can impact brain function and decisions. Neurotransmitter serotonin has been shown to regulate not only food intake, but also economic decisions. We investigated whether an acute nutrition-manipulation inducing plasma tryptophan fluctuation affects brain function, thereby affecting risky decisions. Breakfasts differing in carbohydrate/protein ratios were offered to test changes in risky decision-making while metabolic and neural dynamics were tracked. We identified that a high-carbohydrate/protein breakfast increased plasma tryptophan/LNAA (large neutral amino acids) ratio which mapped to individual risk propensity changes. The nutrition-manipulation and tryptophan/LNAA fluctuation effects on risk propensity changes were further modulated by individual differences in body fat mass. Using fMRI, we further identified activation in the parietal lobule during risk-processing, of which activities 1) were sensitive to the tryptophan/LNAA fluctuation, 2) were modulated by individual's body fat mass, and 3) predicted the risk propensity changes in decision-making. Our results provide evidence for a personalized nutrition-driven modulation on human risky decision and its metabolic and neural mechanisms.

Neuronal activity associated with cocaine preference: Effects of differential cocaine intake

Differences in overall cocaine intake can directly affect neuroadaptations, and this relationship can make it difficult to interpret neurobiological changes seen in drug-choice studies, since drug intake varies between subjects. Herein, a choice procedure that controls for cocaine intake was utilized to explore if neuronal activity, measured as cFos expression in the orbitofrontal cortex (OFC) and nucleus accumbens (NAc), was reflective of preference. Results demonstrated that cFos expression, in both the OFC and NAc, was independent of cocaine preference when cocaine intake was kept constant across individuals. However, when cocaine intake was systematically varied, the expression of cFos associated with cocaine preference was related to overall cocaine intake in the OFC, but not the NAc. Altogether, these results demonstrate that cocaine intake during choice can affect neurobiological outcome measures; thus, the neurobehavioral mechanisms underlying cocaine preference may be better isolated when controlling for cocaine frequency and intake. In all, some caution is warranted when interpreting results from choice studies evaluating the neurobehavioral mechanisms that underlie drug preference when drug frequency and intake are uncontrolled, and future research is needed to determine the role of drug frequency and intake on neurobiological measures associated with drug choice.

Distinct signals in medial and lateral VTA dopamine neurons modulate fear extinction at different times

Dopamine (DA) neurons are thought to encode reward prediction error (RPE), in addition to other signals, such as salience. While RPE is known to support learning, the role of salience in learning remains less clear. To address this, we recorded and manipulated VTA DA neurons in mice during fear extinction. We applied deep learning to classify mouse freezing behavior, eliminating the need for human scoring. Our fiber photometry recordings showed DA neurons in medial and lateral VTA have distinct activity profiles during fear extinction: medial VTA activity more closely reflected RPE, while lateral VTA activity more closely reflected a salience-like signal. Optogenetic inhibition of DA neurons in either region slowed fear extinction, with the relevant time period for inhibition differing across regions. Our results indicate salience-like signals can have similar downstream consequences to RPE-like signals, although with different temporal dependencies.

Median raphe controls acquisition of negative experience in the mouse

Adverse events need to be quickly evaluated and memorized, yet how these processes are coordinated is poorly understood. We discovered a large population of excitatory neurons in mouse median raphe region (MRR) expressing vesicular glutamate transporter 2 (vGluT2) that received inputs from several negative experience-related brain centers, projected to the main aversion centers, and activated the septohippocampal system pivotal for learning of adverse events. These neurons were selectively activated by aversive but not rewarding stimuli. Their stimulation induced place aversion, aggression, depression-related anhedonia, and suppression of reward-seeking behavior and memory acquisition-promoting hippocampal theta oscillations. By contrast, their suppression impaired both contextual and cued fear memory formation. These results suggest that MRR vGluT2 neurons are crucial for the acquisition of negative experiences and may play a central role in depression-related mood disorders.

Learning from Action: Reconsidering Movement Signaling in Midbrain Dopamine Neuron Activity

Animals infer when and where a reward is available from experience with informative sensory stimuli and their own actions. In vertebrates, this is thought to depend upon the release of dopamine from midbrain dopaminergic neurons. Studies of the role of dopamine have focused almost exclusively on their encoding of informative sensory stimuli; however, many dopaminergic neurons are active just prior to movement initiation, even in the absence of sensory stimuli. How should current frameworks for understanding the role of dopamine incorporate these observations? To address this question, we review recent anatomical and functional evidence for action-related dopamine signaling. We conclude by proposing a framework in which dopaminergic neurons encode subjective signals of action initiation to solve an internal credit assignment problem.

Adaptive learning under expected and unexpected uncertainty

The outcome of a decision is often uncertain, and outcomes can vary over repeated decisions. Whether decision outcomes should substantially affect behaviour and learning depends on whether they are representative of a typically experienced range of outcomes or signal a change in the reward environment. Successful learning and decision-making therefore require the ability to estimate expected uncertainty (related to the variability of outcomes) and unexpected uncertainty (related to the variability of the environment). Understanding the bases and effects of these two types of uncertainty and the interactions between them - at the computational and the neural level - is crucial for understanding adaptive learning. Here, we examine computational models and experimental findings to distil computational principles and neural mechanisms for adaptive learning under uncertainty.

Contribution of the prefrontal cortex and basolateral amygdala to behavioral decision-making under reward/punishment conflict

RATIONALE

Control of reward-seeking behavior under conditions of punishment is an important function for survival.

OBJECTIVES AND METHODS

We designed a task in which rats could choose to either press a lever and obtain a food pellet accompanied by a footshock or refrain from pressing the lever to avoid footshock, in response to tone presentation. In the task, footshock intensity steadily increased, and the task was terminated when the lever press probability reached < 25% (last intensity). Rats were trained until the last intensity was stable. Subsequently, we investigated the effects of the pharmacological inactivation of the ventromedial prefrontal cortex (vmPFC), lateral orbitofrontal cortex (lOFC), and basolateral amygdala (BLA) on task performance.

RESULTS

Bilateral inactivation of the vmPFC, lOFC, and BLA did not alter lever press responses at the early stage of the task. The number of lever presses increased following vmPFC and BLA inactivation but decreased following lOFC inactivation during the later stage of the task. The last intensity was elevated by vmPFC or BLA inactivation but lowered by lOFC inactivation. Disconnection of the vmPFC-BLA pathway induced behavioral alterations that were similar to vmPFC or BLA inactivation. Inactivation of any regions did not alter footshock sensitivity and anxiety levels.

CONCLUSIONS

Our results demonstrate a strong role of the vmPFC and BLA and their interactions in reward restraint to avoid punishment and a prominent role of the lOFC in reward-seeking under reward/punishment conflict situations.

Risky decision-making predicts dopamine release dynamics in nucleus accumbens shell

The risky decision-making task (RDT) measures risk-taking in a rat model by assessing preference between a small, safe reward and a large reward with increasing risk of punishment (mild foot shock). It is well-established that dopaminergic drugs modulate risk-taking; however, little is known about how differences in baseline phasic dopamine signaling drive individual differences in risk preference. Here, we used in vivo fixed potential amperometry in male Long-Evans rats to test if phasic nucleus accumbens shell (NACs) dopamine dynamics are associated with risk-taking. We observed a positive correlation between medial forebrain bundle-evoked dopamine release in the NACs and risky decision-making, suggesting that risk-taking is associated with elevated dopamine sensitivity. Moreover, "risk-taking" subjects were found to demonstrate greater phasic dopamine release than "risk-averse" subjects. Risky decision-making also predicted enhanced sensitivity to the dopamine reuptake inhibitor nomifensine, and elevated autoreceptor function. Importantly, this hyperdopaminergic phenotype was selective for risky decision-making, as delay discounting performance was not predictive of phasic dopamine release or dopamine supply. These data identify phasic NACs dopamine release as a possible therapeutic target for alleviating the excessive risk-taking observed across multiple forms of psychopathology.

From controlled to compulsive drug-taking: The role of the habenula in addiction

Addiction is now recognized as a neurobiological and cognitive brain disorder and is generally viewed as a switch from recreational or voluntary to compulsive substance use despite aversive consequences. The habenula, composed of medial (MHb) and lateral (LHb) domains, has been implicated in regulating behavioral flexibility and anxiety-related behaviors and is considered a core component of the brain "anti-reward" system. These functions position the habenula to influence voluntary behaviors. Consistent with this view, emerging evidence points to alterations in habenula activity as important factors to contributing the loss of control over the use of drugs of abuse and the emergence of compulsive drug seeking behaviors. In this review, we will discuss the general functions of the MHb and LHb and describe how these functional properties allow this brain region to promote or suppress volitional behaviors. Then, we highlight mechanisms by which drugs of abuse may alter habenular activity, precipitating the emergence of addiction-relevant behavioral abnormalities.

Transient Input-Specific Neural Plasticity in the Lateral Habenula Facilitates Learning

In this issue of Neuron, Trusel et al. (2019) demonstrate that circuit-specific plasticity in the lateral habenula is dynamically involved in translating CS-US contingencies into cue-driven avoidance behavior. Disruption of this plasticity prevents learning about CS-US relationships when they are uncertain.

Three Rostromedial Tegmental Afferents Drive Triply Dissociable Aspects of Punishment Learning and Aversive Valence Encoding

Persistence of reward seeking despite punishment or other negative consequences is a defining feature of mania and addiction, and numerous brain regions have been implicated in such punishment learning, but in disparate ways that are difficult to reconcile. We now show that the ability of an aversive punisher to inhibit reward seeking depends on coordinated activity of three distinct afferents to the rostromedial tegmental nucleus (RMTg) arising from cortex, brainstem, and habenula that drive triply dissociable RMTg responses to aversive cues, outcomes, and prediction errors, respectively. These three pathways drive correspondingly dissociable aspects of punishment learning. The RMTg in turn drives negative, but not positive, valence encoding patterns in the ventral tegmental area (VTA). Hence, punishment learning involves pathways and functions that are highly distinct, yet tightly coordinated.

Endogenous fluctuations in the dopaminergic midbrain drive behavioral choice variability

Humans are surprisingly inconsistent in their behavior, often making different choices under identical conditions. Previous research suggests that intrinsic fluctuations in brain activity can influence low-level processes, such as the amount of force applied in a motor response. Here, we show that intrinsic prestimulus brain activity in the dopaminergic midbrain influences how we choose between risky and safe options. Using computational modeling, we demonstrate that endogenous fluctuations alter phasic responses in a decision network and thereby modulate risk taking. Our findings demonstrate that higher-order cognition is influenced by fluctuations in internal brain states, providing a physiological basis for variability in complex human behavior.

Human behavior is surprisingly variable, even when facing the same problem under identical circumstances. A prominent example is risky decision making. Economic theories struggle to explain why humans are so inconsistent. Resting-state studies suggest that ongoing endogenous fluctuations in brain activity can influence low-level perceptual and motor processes, but it remains unknown whether endogenous fluctuations also influence high-level cognitive processes including decision making. Here, using real-time functional magnetic resonance imaging, we tested whether risky decision making is influenced by endogenous fluctuations in blood oxygenation level-dependent (BOLD) activity in the dopaminergic midbrain, encompassing ventral tegmental area and substantia nigra. We show that low prestimulus brain activity leads to increased risky choice in humans. Using computational modeling, we show that increased risk taking is explained by enhanced phasic responses to offers in a decision network. Our findings demonstrate that endogenous brain activity provides a physiological basis for variability in complex human behavior.

Optogenetic and chemogenetic approaches to manipulate attention, impulsivity and behavioural flexibility in rodents

Studies manipulating neural activity acutely with optogenetic or chemogenetic intervention in behaving rodents have increased considerably in recent years. More often, these circuit-level neural manipulations are tested within an existing framework of behavioural testing that strives to model complex executive functions or symptomologies relevant to multidimensional psychiatric disorders in humans, such as attentional control deficits, impulsivity or behavioural (in)flexibility. This methods perspective argues in favour of carefully implementing these acute circuit-based approaches to better understand and model cognitive symptomologies or their similar isomorphic animal behaviours, which often arise and persist in overlapping brain circuitries. First, we offer some practical considerations for combining long-term, behavioural paradigms with optogenetic or chemogenetic interventions. Next, we examine how cell-type or projection-specific manipulations to the ascending neuromodulatory systems, local brain region or descending cortical glutamatergic projections influence aspects of cognitive control. For this, we primarily focus on the influence exerted on attentional and motor impulsivity performance in the (3-choice or) 5-choice serial reaction time task, and impulsive, risky or inflexible choice biases during alternative preference, reward discounting or reversal learning tasks.

Positive reward prediction errors during decision-making strengthen memory encoding

Dopamine is thought to provide reward prediction error signals to temporal lobe memory systems, but the role of these signals in episodic memory has not been fully characterized. Here we developed an incidental memory paradigm to (i) estimate the influence of reward prediction errors on the formation of episodic memories, (ii) dissociate this influence from surprise and uncertainty, (iii) characterize the role of temporal correspondence between prediction error and memoranda presentation and (iv) determine the extent to which this influence is dependent on memory consolidation. We found that people encoded incidental memoranda more strongly when they gambled for potential rewards. Moreover, the degree to which gambling strengthened encoding scaled with the reward prediction error experienced when memoranda were presented (and not before or after). This encoding enhancement was detectable within minutes and did not differ substantially after 24 h, indicating that it is not dependent on memory consolidation. These results suggest a computationally and temporally specific role for reward prediction error signalling in memory formation.

Deconstructing value-based decision making via temporally selective manipulation of neural activity: Insights from rodent models

The ability to choose among options that differ in their rewards and costs (value-based decision making) has long been a topic of interest for neuroscientists, psychologists, and economists alike. This is likely because this is a cognitive process in which all animals (including humans) engage on a daily basis, be it routine (which road to take to work) or consequential (which graduate school to attend). Studies of value-based decision making (particularly at the preclinical level) often treat it as a uniform process. The results of such studies have been invaluable for our understanding of the brain substrates and neurochemical systems that contribute to decision making involving a range of different rewards and costs. Value-based decision making is not a unitary process, however, but is instead composed of distinct cognitive operations that function in concert to guide choice behavior. Within this conceptual framework, it is therefore important to consider that the known neural substrates supporting decision making may contribute to temporally distinct and dissociable components of the decision process. This review will describe this approach for investigating decision making, drawing from published studies that have used techniques that allow temporal dissection of the decision process, with an emphasis on the literature in animal models. The review will conclude with a discussion of the implications of this work for understanding pathological conditions that are characterized by impaired decision making.

Stress transforms lateral habenula reward responses into punishment signals

Neuronal activity in the lateral habenula (LHb), a brain region implicated in depression [C. D. Proulx, O. Hikosaka, R. Malinow, Nat. Neurosci. 17, 1146-1152 (2014)], decreases during reward and increases during punishment or reward omission [M. Matsumoto, O. Hikosaka, Nature 447, 1111-1115 (2007)]. While stress is a major risk factor for depression and strongly impacts the LHb, its effect on LHb reward signals is unknown. Here we image LHb neuronal activity in behaving mice and find that acute stress transforms LHb reward responses into punishment-like neural signals; punishment-like responses to reward omission also increase. These neural changes matched the onset of anhedonic behavior and were specific to LHb neurons that distinguished reward and its omission. Thus, stress distorts LHb responsivity to positive and negative feedback, which could bias individuals toward negative expectations, a key aspect of the proposed pathogenesis of depression [A. T. Beck, Depression: Clinical, Experimental, and Theoretical Aspects, sixth Ed (1967)].

Dopamine blockade impairs the exploration-exploitation trade-off in rats

In a volatile environment where rewards are uncertain, successful performance requires a delicate balance between exploitation of the best option and exploration of alternative choices. It has theoretically been proposed that dopamine contributes to the control of this exploration-exploitation trade-off, specifically that the higher the level of tonic dopamine, the more exploitation is favored. We demonstrate here that there is a formal relationship between the rescaling of dopamine positive reward prediction errors and the exploration-exploitation trade-off in simple non-stationary multi-armed bandit tasks. We further show in rats performing such a task that systemically antagonizing dopamine receptors greatly increases the number of random choices without affecting learning capacities. Simulations and comparison of a set of different computational models (an extended Q-learning model, a directed exploration model, and a meta-learning model) fitted on each individual confirm that, independently of the model, decreasing dopaminergic activity does not affect learning rate but is equivalent to an increase in random exploration rate. This study shows that dopamine could adapt the exploration-exploitation trade-off in decision-making when facing changing environmental contingencies.