Dopamine, Updated: Reward Prediction Error and Beyond

Natural behaviour is learned through dopamine-mediated reinforcement

Many natural motor skills, such as speaking or locomotion, are acquired through a process of trial-and-error learning over the course of development. It has long been hypothesized, motivated by observations in artificial learning experiments, that dopamine has a crucial role in this process. Dopamine in the basal ganglia is thought to guide reward-based trial-and-error learning by encoding reward prediction errors1, decreasing after worse-than-predicted reward outcomes and increasing after better-than-predicted ones. Our previous work in adult zebra finches-in which we changed the perceived song quality with distorted auditory feedback-showed that dopamine in Area X, the singing-related basal ganglia, encodes performance prediction error: dopamine is suppressed after worse-than-predicted (distorted syllables) and activated after better-than-predicted (undistorted syllables) performance2. However, it remains unknown whether the learning of natural behaviours, such as developmental vocal learning, occurs through dopamine-based reinforcement. Here we tracked song learning trajectories in juvenile zebra finches and used fibre photometry3 to monitor concurrent dopamine activity in Area X. We found that dopamine was activated after syllable renditions that were closer to the eventual adult version of the song, compared with recent renditions, and suppressed after renditions that were further away. Furthermore, the relationship between dopamine and song fluctuations revealed that dopamine predicted the future evolution of song, suggesting that dopamine drives behaviour. Finally, dopamine activity was explained by the contrast between the quality of the current rendition and the recent history of renditions-consistent with dopamine's hypothesized role in encoding prediction errors in an actor-critic reinforcement-learning model4,5. Reinforcement-learning algorithms6 have emerged as a powerful class of model to explain learning in reward-based laboratory tasks, as well as for driving autonomous learning in artificial intelligence7. Our results suggest that complex natural behaviours in biological systems can also be acquired through dopamine-mediated reinforcement learning.

Assessing acute effects of methylphenidate and modafinil on inhibitory capacity, time estimation, attentional lapses, and compulsive-like behavior in rats

Medications known as 'cognitive enhancers' are increasingly being consumed off-label by healthy people, raising concerns about their safety. The aim of our study was to profile behavioral performance upon oral administration of methylphenidate (2.5 mg/kg) and modafinil (64 mg/kg) - two popular cognitive enhancers - and upon their discontinuation. We modeled cognitively demanding challenges in neurotypical individuals using a behavioral task where Wistar - Lewis rats had to withhold responses for a specified time to obtain food rewards. This task allowed us to extract several measures of behavioral performance associated with clinically meaningful indices, such as compulsive-like responding, incapacity to wait (impulsivity), time estimation (precision and accuracy), and attentional lapses. Our study design involved examining these behavioral indices in subjects administered either methylphenidate, modafinil, or vehicle. We found that subjects administered modafinil obtained fewer rewards and were less efficient in reward pursuing than the vehicle group; this result was likely due to a drug-induced inability to wait. Upon modafinil discontinuation, subjects earned more rewards but did not entirely catch up with the vehicle group. As for methylphenidate, neither favorable nor unfavorable effects were found in our main analyses. However, an exploratory analysis of changes in behavioral performance within sessions suggested that methylphenidate fostered favorable, yet short-lived, effects. We discuss our results in terms of the risks and cost-benefits of doses above or below the effective dose of cognitive enhancement drugs.

Aging reduces motivation through decreased Bdnf expression in the ventral tegmental area

Age-associated reduced motivation is a hallmark of neuropsychiatric disorders in the elderly. In our rapidly aging societies, it is critical to keep motivation levels high enough to promote healthspan and lifespan. However, how motivation is reduced during aging remains unknown. Here, we used multiple mouse models to evaluate motivation and related affective states in young and old mice. We also compared the effect of social isolation, a common stressor in aged populations, to those of aging. We found that both social isolation and aging decreased motivation in mice, but that Bdnf expression in the ventral tegmental area (VTA) was selectively decreased during aging. Furthermore, VTA-specific Bdnf knockdown in young mice recapitulated reduced motivation observed in old mice. These results demonstrate that maintaining Bdnf expression in the VTA could promote motivation to engage in effortful activities and potentially prevent age-associated neuropsychiatric disorders.

The Role of Dopamine in Neurological, Psychiatric, and Metabolic Disorders and Cancer: A Complex Web of Interactions

Dopamine, a key neurotransmitter in the central nervous system, is essential for regulating a wide range of physiological processes, including motor control, reward processing, mood regulation, and decision-making [...].

Unraveling the role of Nrf2 in dopaminergic neurons: a review of oxidative stress and mitochondrial dysfunction in Parkinson's disease

Nuclear factor erythroid 2-related factor 2 (Nrf2) is an essential transcriptional factor, involved in the regulation of countenance of various anti-oxidant enzymes and cytoprotective genes that respond to mitochondrial dysfunctions, oxidative stress, and neuroinflammation, thus potentially contributing to several neurodegenerative diseases (NDDs), including Parkison's disease (PD). PD is the second most prevalent progressive NDD, characterized by gradual neuronal death in substantia nigra pars compacta (SNpc), depletion of dopamine level, and a wide range of motor symptoms, including bradykinesia, tremor, tingling, and muscle fatigue. The etiopathology of PD is caused by multifactorial intertwined with the onset and progression of the disease. In this context, Nrf2 exhibits neuroprotective action by preserving dopaminergic neurons in the striatum and retarding the disease progression; thus, Nrf2 activation plays a crucial role in PD. Additionally, Nrf2 binds with the antioxidant response element, which is located in the promoter region of most of the genes that are responsible for coding antioxidant enzymes. Moreover, protein kinase C (PKC) mitogen-activated protein kinase (MAPK), and phosphatidylinositol 3-kinase (PI3K) are also involved in the regulation of Keap1 pathway-mediated Nrf2 activation. As Nrf2 revealed its defensive and protective role in the central nervous system (CNS), it is gaining enough interest in treating PD. The treatments that are currently available are intended to alleviate the symptoms of PD; however, they are unable to halt the progression and severity of the disease. Therefore, in this review we delve deeper into various molecular mechanisms associated with oxidative stress, mitochondrial dysfunction, and neuroinflammation in PD. Additionally, we elaborated on the substantial role that NRF2 plays in mitigating these adverse effects and its potential as a therapeutic target.

The role of the dopamine system in autism spectrum disorder revealed using machine learning: an ABIDE database-based study

This study explores the diagnostic value of dopamine system imaging characteristics in children with autism spectrum disorder. Functional magnetic resonance data from 551 children in the Autism Brain Imaging Data Exchange database were analyzed, focusing on six dopamine-related brain regions as regions of interest. Functional connectivity between these ROIs and across the whole brain was assessed. Machine learning techniques then evaluated the ability of the dopamine system's imaging features to predict autism spectrum disorder. Functional connectivity was significantly higher in autism spectrum disorder children between the ventral tegmental area and substantia nigra, prefrontal cortex, nucleus accumbens, and between the substantia nigra and hypothalamus compared to typically developing children. Additionally, clustering methods identified two autism spectrum disorder subtypes, achieving over 0.8 accuracy. Subtype 1 showed higher stereotyped behavior scores than subtype 2 in both genders, with subtype-specific functional connectivity differences between male and female autism spectrum disorder groups. These findings suggest that abnormal functional connectivity in the dopamine system serves as a diagnostic biomarker for autism spectrum disorder and can support clinical decision-making and personalized treatment optimization.

Interactions of sex and stress in modulation of ventral tegmental area dopaminergic activity

Dopaminergic (DA) neurons of the ventral tegmental area (VTA) have long been studied for their role in reward prediction and goal-directed behaviors. However, appreciation is growing for a complementary role of VTA DA neurons in responding to aversive stimuli and as critical substrates for behavioral sequelae of stressful experiences. As is the case across neuroscience, the majority of our knowledge about VTA DA neurons comes from studies in male subjects. Recent years have seen an increase in inclusion of female subjects and exploration of sex differences. There is now an emerging body of literature showing that although there are minimal basal structural and functional sex differences in VTA DA neurons, experience-dependent changes in these neurons can differ significantly between males and females. Here, we discuss potential implications of sex differences in VTA function and review recent data on sex differences and similarities of DA neurons at baseline and following stress.

How Dopamine Enables Learning from Aversion

Dopamine is heavily studied for its role in reward learning, but it is becoming increasingly appreciated that dopamine can also enable learning from aversion. Dopamine neurons modulate their firing and neurotransmitter release patterns in response to aversive outcomes. However, there is considerable heterogeneity in the timing and directionality of the modulation. Open questions remain as to the factors that determine this heterogeneity and how varying patterns of responses to aversion in different dopamine-receptive brain regions contribute to value learning, decision-making, and avoidance. Here, we review recent progress in this area and highlight important future directions.

Electrical brain activations in preadolescents during a probabilistic reward-learning task reflect cognitive processes and behavior strategies

Both adults and children learn through feedback to associate environmental events and choices with reward, a process known as reinforcement learning (RL). However, tasks to assess RL-related neurocognitive processes in children have been limited. This study validated a child version of the Probabilistic Reward Learning task in preadolescents (8-12 years) while recording event-related-potential (ERPs), focusing on: (1) reward-feedback sensitivity (frontal Reward-related Positivity, RewP), (2) late attention-related responses to feedback (parietal P300), and (3) attentional shifting toward favored stimuli (N2pc). Behaviorally, as expected, preadolescents could learn stimulus-reward outcome associations, but with varying performance levels. Poor learners showed greater RewP amplitudes compared to good learners. Learning strategies (i.e., Win-Lose-Stay-Shift) were reflected by feedback-elicited P300 amplitudes. Lastly, attention shifted toward to-be-chosen stimuli, as evidenced by the N2pc, but not toward more highly rewarded stimuli as in adults. These findings provide novel insights into the neural processes underlying RL in preadolescents.

Deficiency of orexin receptor type 1 in dopaminergic neurons increases novelty-induced locomotion and exploration

Orexin signaling in the ventral tegmental area and substantia nigra promotes locomotion and reward processing, but it is not clear whether dopaminergic neurons directly mediate these effects. We show that dopaminergic neurons in these areas mainly express orexin receptor subtype 1 (Ox1R). In contrast, only a minor population in the medial ventral tegmental area express orexin receptor subtype 2 (Ox2R). To analyze the functional role of Ox1R signaling in dopaminergic neurons, we deleted Ox1R specifically in dopamine transporter-expressing neurons of mice and investigated the functional consequences. Deletion of Ox1R increased locomotor activity and exploration during exposure to novel environments or when intracerebroventricularely injected with orexin A. Spontaneous activity in home cages, anxiety, reward processing, and energy metabolism did not change. Positron emission tomography imaging revealed that Ox1R signaling in dopaminergic neurons affected distinct neural circuits depending on the stimulation mode. In line with an increase of neural activity in the lateral paragigantocellular nucleus (LPGi) of Ox1RΔDAT mice, we found that dopaminergic projections innervate the LPGi in regions where the inhibitory dopamine receptor subtype D2 but not the excitatory D1 subtype resides. These data suggest a crucial regulatory role of Ox1R signaling in dopaminergic neurons in novelty-induced locomotion and exploration.

A neurocomputational account of multi-line electronic gambling machines

Multi-line electronic gambling machines (EGMs) are strongly associated with problem gambling. Dopamine (DA) plays a central role in substance-use disorders, which share clinical and behavioral features with disordered gambling. The structural design features of multi-line EGMs likely lead to the elicitation of various dopaminergic effects within their nested anticipation-outcome structure. The present account draws an analogy between EGM gambling and latent state inference accounts of conditioning, and links maladaptive gambling-related beliefs and expectancies to a process of erroneous latent state inference that may be exacerbated by EGM design features and associated dopaminergic processes. Over the course of repeated exposure to gambling, these processes may foster the emergence of maladaptive state priors, which clinically manifest as gambling-related cognitions, beliefs, and expectancies.

Epigenetic regulation-mediated disorders in dopamine transporter endocytosis: A novel mechanism for the pathogenesis of Parkinson's disease

Mechanisms such as DNA methylation, histone modifications, and non-coding RNA regulation may impact the endocytosis of dopamine transporter (DAT) by influencing processes like neuronal survival, thereby contributing to the initiation and progression of Parkinson's Disease (PD). Some small molecule inhibitors or natural bioactive compounds have the potential to modulate epigenetic processes, thereby reversing induced pluripotent stem cells (iPSCs) reprogramming and abnormal differentiation, offering potential therapeutic effects for PD. Although no specific DNA modification enzyme directly regulates DAT endocytosis, enzymes such as DNA methyltransferases (DNMTs) may indirectly influence DAT endocytosis by regulating the expression of genes associated with this process. DNA modifications impact DAT endocytosis by modulating key signaling pathways, including the (protein kinase C) PKC and D2 receptor (D2R) pathways. Key enzymes involved in RNA modifications that influence DAT endocytosis include m6A methyltransferases and other related enzymes. This regulation impacts the synthesis and function of proteins involved in DAT endocytosis, thereby indirectly affecting the process itself. RNA modifications regulate DAT endocytosis through various indirect pathways, as well as histone modifications. Key enzymes influence the expression of genes associated with DAT endocytosis by modulating the chromatin's accessibility and compaction state. These enzymes control the expression of proteins involved in regulating endocytosis, promoting endosome formation, and facilitating recycling processes. Through the modulation exerted by these enzymes, the speed of DAT endocytosis and recycling patterns are indirectly regulated, establishing a crucial epigenetic control point for the regulation of neurotransmitter transport. Based on this understanding, we anticipate that targeting these processes could lead to favorable therapeutic effects for early PD pathogenesis.

Prefrontal dopamine activity is critical for rapid threat avoidance learning

The medial prefrontal cortex (mPFC) is required for learning associations that determine whether animals approach or avoid potential threats in the environment. Dopaminergic (DA) projections from the ventral tegmental area (VTA) to the mPFC carry information, particularly about aversive outcomes, that may inform prefrontal computations. But the role of prefrontal DA in learning based on aversive outcomes remains poorly understood. Here, we used platform mediated avoidance (PMA) to study the role of mPFC DA in threat avoidance learning in mice. We show that activity in VTA-mPFC dopaminergic terminals is required for avoidance learning, but not for escape, conditioned fear, or to recall a previously learned avoidance strategy. mPFC DA is most dynamic in the early stages of learning, and encodes aversive outcomes, their omissions, and threat-induced behaviors. Computational models of PMA behavior and DA activity revealed that mPFC DA influences learning rates and encodes the predictive relationships between cues and adaptive behaviors. Taken together, these data indicate that mPFC DA is necessary to rapidly learn behaviors required to avoid signaled threats, but not for learning cue-threat associations.

Neuronal encoding of behaviors and instrumental learning in the dorsal striatum

The dorsal striatum is instrumental in regulating motor control and goal-directed behaviors. The classical description of the two output pathways of the dorsal striatum highlights their antagonistic control over actions. However, recent experimental evidence implicates both pathways and their coordinated activities during actions. In this review, we examine the different models proposed for striatal encoding of actions during self-paced behaviors and how they can account for evidence harvested during goal-directed behaviors. We also discuss how the activation of striatal ensembles can be reshaped and reorganized to support the formation of instrumental learning and behavioral flexibility. Future work integrating these considerations may resolve controversies regarding the control of actions by striatal networks.

Applications of Behavioral Economics and Neuroeconomics in Mental Health

Objective: The integration of behavioral economics and neuroeconomics into mental health offers innovative perspectives on understanding and addressing psychological disorders. This overview aims to synthesize current knowledge and explore the implications of these interdisciplinary approaches in the context of mental health. Method : In this narrative review, we summarized the current evidence regarding the applications of behavioral economics and neuroeconomics approaches in the field of mental health. Results: Behavioral economics and neuroeconomics provide valuable insights into the cognitive and emotional processes underlying mental health disorders, such as irrational decision-making, impulsivity, and self-control issues. Concepts such as loss aversion, temporal discounting, and framing effects inform the development of innovative interventions and policy initiatives. Behavioral economic interventions, including nudges, incentives, and commitment devices, show promise in promoting treatment adherence, reducing risky behaviors, and enhancing mental well-being. Neuroeconomics contributes by identifying neural markers predictive of treatment response and relapse risk, paving the way for personalized treatment approaches. Conclusion: The integration of behavioral economics and neuroeconomics into mental health research and practice holds significant potential for improving the understanding of psychological disorders and developing more effective, personalized interventions. Further research is needed to elucidate the mechanisms of action, optimize intervention strategies, and address ethical considerations associated with these approaches in mental health settings.

Dopamine-Sensitive Anterior Cingulate Cortical Glucose-Monitoring Neurons as Potential Therapeutic Targets for Gustatory and Other Behavior Alterations

Background: The anterior cingulate cortex (ACC) is known for its involvement in various regulatory functions, including in the central control of feeding. Activation of local elements of the central glucose-monitoring (GM) neuronal network appears to be indispensable in these regulatory processes. Destruction of these type 2 glucose transporter protein (GLUT2)-equipped chemosensory cells results in multiple feeding-associated functional alterations. Methods: In order to examine this complex symptomatology, (1) dopamine sensitivity was studied in laboratory rats by means of the single-neuron-recording multibarreled microelectrophoretic technique, and (2) after local bilateral microinjection of the selective type 2 glucose transporter proteindemolishing streptozotocin (STZ), open-field, elevated plus maze, two-bottle and taste reactivity tests were performed. Results: A high proportion of the anterior cingulate cortical neurons changed their firing rate in response to microelectrophoretic administration of D-glucose, thus verifying them as local elements of the central glucose-monitoring network. Approximately 20% of the recorded cells displayed activity changes in response to microelectrophoretic application of dopamine, and almost 50% of the glucose-monitoring units here proved to be dopamine-sensitive. Moreover, taste stimulation experiments revealed even higher (80%) gustatory sensitivity dominance of these chemosensory cells. The anterior cingulate cortical STZ microinjections resulted in extensive behavioral and taste-associated functional deficits. Conclusions: The present findings provided evidence for the selective loss of glucose-monitoring neurons in the anterior cingulate cortex leading to motivated behavioral and gustatory alterations. This complex dataset also underlines the varied significance of the type 2 glucose transporter protein-equipped, dopamine-sensitive glucose-monitoring neurons as potential therapeutic targets. These units appear to be indispensable in adaptive control mechanisms of the homeostatic-motivational-emotional-cognitive balance for the overall well-being of the organism.

Integrated omics analysis reveals the alteration of gut microbiota and fecal metabolites in Cervus elaphus kansuensis

The gut microbiota is the largest and most complex microecosystem in animals. It is influenced by the host's dietary habits and living environment, and its composition and diversity play irreplaceable roles in animal nutrient metabolism, immunity, and adaptation to the environment. Although the gut microbiota of red deer has been studied, the composition and function of the gut microbiota in Gansu red deer (Cervus elaphus kansuensis), an endemic subspecies of red deer in China, has not been reported. In this study, the composition and diversity of the gut microbiome and fecal metabolomics of C. elaphus kansuensis were identified and compared for the first time by using 16S rDNA sequencing, metagenomic sequencing, and LC-MS/MS. There were significant differences in gut microbiota structure and diversity between wild and farmed C. elaphus kansuensis. The 16S rDNA sequencing results showed that the genus UCRD-005 was dominant in both captive red deer (CRD) and wild red deer (WRD). Metagenomic sequencing showed similar results to those of 16S rDNA sequencing for gut microbiota in CRD and WRD at the phylum and genus levels. 16S rDNA and metagenomics sequencing data suggested that Bacteroides and Bacillus might serve as marker genera for CRD and WRD, respectively. Fecal metabolomics results showed that 520 metabolites with significant differences were detected between CRD and WRD and most differential metabolites were involved in lipid metabolism. The results suggested that large differences in gut microbiota composition and fecal metabolites between CRD and WRD, indicating that different dietary habits and living environments over time have led to the development of stable gut microbiome characteristics for CRD and WRD to meet their respective survival and reproduction needs. KEY POINTS: • Environment and food affected the gut microbiota and fecal metabolites in red deer • Genera Bacteroides and Bacillus may play important roles in CRD and WRD, respectively • Flavonoids and ascorbic acid in fecal metabolites may influence health of red deer.

Effects of Ficus exasperata on neurotransmission and expression of BDNF, tau, ACHE and BACE in diabetic rats

Diabetes mellitus, a chronic metabolic disorder, has significant global health implications, particularly due to its neurological complications, such as diabetic neuropathy. This condition increases the risk of neurodegenerative diseases by affecting peripheral nerves and cognition. Ficus exasperata, known for its neuroprotective properties, shows promise as a therapeutic option for addressing these complications. This study evaluates the effects of methanol extract of Ficus exasperata (MEFE) on neurotransmission and the expression of Tau, brain-derived neurotrophic factor (BDNF), acetylcholinesterase (ACHE), and Beta-Site Amyloid Precursor Protein Cleaving Enzyme (BACE) in alloxan-induced diabetic Wistar rats. The controlled experimental design involved 20 Wistar rats divided into four groups (n = 5): control, diabetic untreated, diabetes + MEFE (200 mg/kg), and diabetes + insulin (0.3 IU). The methanol extract was prepared using cold maceration, and an aliquot was subjected to gas chromatography-mass spectrometry. Constituents of MEFE were docked with neurologic receptors. Blood glucose levels were measured using the glucose oxidase method, and neurotransmitter levels, antioxidants, oxidative stress markers, and the expression of Tau, BDNF, ACHE, and BACE were assessed using standard procedures and qRT-PCR. Data were analyzed using one-way ANOVA at P < 0.05. Results indicated that MEFE significantly reduced fasting blood glucose levels compared to untreated diabetic rats. In silico docking identified kaur-16-ene, a constituent of MEFE, as having the highest binding affinity for NMDA, TrkB, mAchR and nAchR receptors, indicating its neuroprotective potential. MEFE also enhanced antioxidant enzyme levels (SOD, GPx, catalase) while reducing oxidative stress markers (MDA, 8-OHdG). Gene expression analysis revealed that MEFE modulates the expression of Tau, BDNF, ACHE, and BACE, suggesting its potential to influence neurodegenerative pathways associated with diabetic neuropathy. Ficus exasperata demonstrates significant therapeutic potential in managing diabetic neuropathy and related cognitive impairments by modulating neurotransmission, protein expression, and antioxidant defenses.

Doing good well (Karma Yoga, the path of selfless action): Psychotherapeutic lessons from the East

The tripartite classification of mental faculties into cognition, affect, and conation (motivation and action) continues to be the edifice on which the mind and the methods to address mental afflictions are studied. Eastern spiritual traditions offer insights into mental health as it pertains to each of these domains. Following up on our previous paper on the cognition path to psychotherapy (Knowing oneself, or Jnana Yoga), we herein focus on the path of selfless action (Karma Yoga). We review eastern concepts on the nature of karma and the approaches to optimal action (the will to do things, doing the right things, and doing them well). We then place these eastern insights in the context of emerging concepts in psychology on motivation and action. Current psychological concepts such as autonomy and intrinsic motivation, mastery, flow and growth mindset, higher purpose and value driven self-less action, equanimity and balance are convergent with ancient eastern concepts. We also review current neuroscientific underpinnings (such as neural circuitries, neurotransmitter systems and epigenetics and how these facilitate neural plasticity) relevant to karma, including free will, focused action, prosocial behaviors, extrinsic and intrinsic and motivation. These concepts have significant implications for psychotherapeutic models, especially in the areas of positive psychology and preventive psychiatry.

A unified theoretical framework underlying the regulation of motivated behavior

To orchestrate behaviors for survival, multiple psychological components have evolved. The current theories do not clearly distinguish the distinct components. In this article, we provide a unified theoretical framework. To optimize survival, there should be four components; (1) "need", an alarm based on a predicted deficiency. (2) "motivation", a direct behavior driver. (3) "pleasure", a teacher based on immediate outcomes. (4) "utility", a teacher based on final delayed outcomes. For behavior stability, need should be accumulated into motivation to drive behavior. Based on the immediate outcome of the behavior, the pleasure should teach whether to continue the current behavior. Based on the final delay outcome, the utility should teach whether to increase future behavior by reshaping the other three components. We provide several neural substrate candidates in the food context. The proposed theoretical framework, in combination with appropriate experiments, will unravel the neural components responsible for each theoretical component.

Distinct dopaminergic spike-timing-dependent plasticity rules are suited to different functional roles

Various mathematical models have been formulated to describe the changes in synaptic strengths resulting from spike-timing-dependent plasticity (STDP). A subset of these models include a third factor, dopamine, which interacts with spike timing to contribute to plasticity at specific synapses, notably those from cortex to striatum at the input layer of the basal ganglia. Theoretical work to analyze these plasticity models has largely focused on abstract issues, such as the conditions under which they may promote synchronization and the weight distributions induced by inputs with simple correlation structures, rather than on scenarios associated with specific tasks, and has generally not considered dopamine-dependent forms of STDP. In this paper we introduce three forms of dopamine-modulated STDP adapted from previously proposed plasticity rules. We then analyze, mathematically and with simulations, their performance in three biologically relevant scenarios. We test the ability of each of the three models to maintain its weights in the face of noise and to complete simple reward prediction and action selection tasks, studying the learned weight distributions and corresponding task performance in each setting. Interestingly, we find that each plasticity rule is well suited to a subset of the scenarios studied but falls short in others. Different tasks may therefore require different forms of synaptic plasticity, yielding the prediction that the precise form of the STDP mechanism present may vary across regions of the striatum, and other brain areas impacted by dopamine, that are involved in distinct computational functions.

Individual differences in decision-making shape how mesolimbic dopamine regulates choice confidence and change-of-mind

Nucleus accumbens dopamine signaling is an important neural substrate for decision-making. Dominant theories generally discretize and homogenize decision-making, when it is in fact a continuous process, with evaluation and re-evaluation components that extend beyond simple outcome prediction into consideration of past and future value. Extensive work has examined mesolimbic dopamine in the context of reward prediction error, but major gaps persist in our understanding of how dopamine regulates volitional and self-guided decision-making. Moreover, there is little consideration of individual differences in value processing that may shape how dopamine regulates decision-making. Here, using an economic foraging task in mice, we found that dopamine dynamics in the nucleus accumbens core reflected decision confidence during evaluation of decisions, as well as both past and future value during re-evaluation and change-of-mind. Optogenetic manipulations of mesolimbic dopamine release selectively altered evaluation and re-evaluation of decisions in mice whose dopamine dynamics and behavior reflected future value.

Distinct cholinergic circuits underlie discrete effects of reward on attention

Attention and reward are functions that are critical for the control of behavior, and massive multi-region neural systems have evolved to support the discrete computations associated with each. Previous research has also identified that attention and reward interact, though our understanding of the neural mechanisms that mediate this interplay is incomplete. Here, we review the basic neuroanatomy of attention, reward, and cholinergic systems. We then examine specific contexts in which attention and reward computations interact. Building on this work, we propose two discrete neural circuits whereby acetylcholine, released from cell groups located in different parts of the brain, mediates the impact of stimulus-reward associations as well as motivation on attentional control. We conclude by examining these circuits as a potential shared loci of dysfunction across diseases states associated with deficits in attention and reward.

Dopamine Release in the Nucleus Accumbens Core Encodes the General Excitatory Components of Learning

Dopamine release in the nucleus accumbens core (NAcC) is generally considered to be a proxy for phasic firing of the ventral tegmental area dopamine (VTADA) neurons. Thus, dopamine release in NAcC is hypothesized to reflect a unitary role in reward prediction error signaling. However, recent studies reveal more diverse roles of dopamine neurons, which support an emerging idea that dopamine regulates learning differently in distinct circuits. To understand whether the NAcC might regulate a unique component of learning, we recorded dopamine release in NAcC while male rats performed a backward conditioning task where a reward is followed by a neutral cue. We used this task because we can delineate different components of learning, which include sensory-specific inhibitory and general excitatory components. Furthermore, we have shown that VTADA neurons are necessary for both the specific and general components of backward associations. Here, we found that dopamine release in NAcC increased to the reward across learning while reducing to the cue that followed as it became more expected. This mirrors the dopamine prediction error signal seen during forward conditioning and cannot be accounted for temporal-difference reinforcement learning. Subsequent tests allowed us to dissociate these learning components and revealed that dopamine release in NAcC reflects the general excitatory component of backward associations, but not their sensory-specific component. These results emphasize the importance of examining distinct functions of different dopamine projections in reinforcement learning.

Model of a striatal circuit exploring biological mechanisms underlying decision-making during normal and disordered states

Decision-making requires continuous adaptation to internal and external contexts. Changes in decision-making are reliable transdiagnostic symptoms of neuropsychiatric disorders. We created a computational model demonstrating how the striosome compartment of the striatum constructs a mathematical space for decision-making computations depending on context, and how the matrix compartment defines action value depending on the space. The model explains multiple experimental results and unifies other theories like reward prediction error, roles of the direct versus indirect pathways, and roles of the striosome versus matrix, under one framework. We also found, through new analyses, that striosome and matrix neurons increase their synchrony during difficult tasks, caused by a necessary increase in dimensionality of the space. The model makes testable predictions about individual differences in disorder susceptibility, decision-making symptoms shared among neuropsychiatric disorders, and differences in neuropsychiatric disorder symptom presentation. The model reframes the role of the striosomal circuit in neuroeconomic and disorder-affected decision-making.

Highlights

Striosomes prioritize decision-related data used by matrix to set action values. Striosomes and matrix have different roles in the direct and indirect pathways. Abnormal information organization/valuation alters disorder presentation. Variance in data prioritization may explain individual differences in disorders.

eTOC

Beck et al. developed a computational model of how a striatal circuit functions during decision-making. The model unifies and extends theories about the direct versus indirect pathways. It further suggests how aberrant circuit function underlies decision-making phenomena observed in neuropsychiatric disorders.

Stress-induced anxiety-related behavior in mice is driven by enhanced excitability of ventral tegmental area GABA neurons

Introduction

Stress and trauma are significant risk factors for many neuropsychiatric disorders and diseases, including anxiety disorders. Stress-induced anxiety symptoms have been attributed to enhanced excitability in circuits controlling fear, anxiety, and aversion. A growing body of evidence has implicated GABAergic neurons of the ventral tegmental area (VTA) in aversion processing and affective behavior.

Methods

We used an unpredictable footshock (uFS) model, together with electrophysiological and behavioral approaches, to investigate the role of VTA GABA neurons in anxiety-related behavior in mice.

Results

One day after a single uFS session, C57BL/6J mice exhibited elevated anxiety-related behavior and VTA GABA neuron excitability. The enhanced excitability of VTA GABA neurons was correlated with increased glutamatergic input and a reduction in postsynaptic signaling mediated via GABAA and GABAB receptors. Chemogenetic activation of VTA GABA neurons was sufficient to increase anxiety-related behavior in stress-naïve mice. In addition, chemogenetic inhibition of VTA GABA neurons suppressed anxiety-related behavior in mice exposed to uFS.

Discussion

These data show that VTA GABA neurons are an early substrate for stress-induced anxiety-related behavior in mice and suggest that approaches mitigating enhanced excitability of VTA GABA neurons may hold promise for the treatment of anxiety provoked by stress and trauma.

Arousal, interindividual differences and temporal binding a psychophysiological study

The sense of agency varies as a function of arousal in negative emotional contexts. As yet, it is unknown whether the same is true for positive affect, and how inter-individual characteristics might predict these effects. Temporal binding, an implicit measure of the sense of agency, was measured in 59 participants before and after watching either an emotionally neutral film clip or a positive film clip with high or low arousal. Analyses included participants' individual differences in subjective affective ratings, physiological arousal (pupillometry, skin conductance, heart rate), striatal dopamine levels via eye blink rates, and psychopathy. Linear mixed models showed that sexual arousal decreased temporal binding whereas calm pleasure had no facilitation effect on binding. Striatal dopamine levels were positively linked whereas subjective and physiological arousal may be negatively associated with binding towards actions. Psychopathic traits reduced the effect of high arousal on binding towards actions. These results provide evidence that individual differences influence the extent to which the temporal binding is affected by high arousing states with positive valence.

Mate choice in the brain: species differ in how male traits 'turn on' gene expression in female brains

Mate choice plays a fundamental role in speciation, yet we know little about the molecular mechanisms that underpin this crucial decision-making process. Stickleback fish differentially adapted to limnetic and benthic habitats are reproductively isolated and females of each species use different male traits to evaluate prospective partners and reject heterospecific males. Here, we integrate behavioural data from a mate choice experiment with gene expression profiles from the brains of females actively deciding whether to mate. We find substantial gene expression variation between limnetic and benthic females, regardless of behavioural context, suggesting general divergence in constitutive gene expression patterns, corresponding to their genetic differentiation. Intriguingly, female gene co-expression modules covary with male display traits but in opposing directions for sympatric populations of the two species, suggesting male displays elicit a dynamic neurogenomic response that reflects known differences in female preferences. Furthermore, we confirm the role of numerous candidate genes previously implicated in female mate choice in other species, suggesting evolutionary tinkering with these conserved molecular processes to generate divergent mate preferences. Taken together, our study adds important new insights to our understanding of the molecular processes underlying female decision-making critical for generating sexual isolation and speciation.

Depression promotes breast cancer progression by regulating amino acid neurotransmitter metabolism and gut microbial disturbance

INTRODUCTION

Breast cancer (BC) is the most prevalent type of cancer and has the highest mortality among women worldwide. BC patients have a high risk of depression, which has been recognized as an independent factor in the progression of BC. However, the potential mechanism has not been clearly demonstrated.

METHODS

To explore the correlation and mechanism between depression and BC progression, we induced depression and tumor in BC mouse models. Depression was induced via chronic unpredictable mild stress (CUMS) and chronic restraint stress (CRS). Amino acid (AA) neurotransmitter-targeted metabonomics and gut microbiota 16S rDNA gene sequencing were employed in the mouse model after evaluation with behavioral tests and pathological analysis.

RESULTS

The tumors in cancer-depression (CD) mice grew faster than those in cancer (CA) mice, and lung metastasis was observed in CD mice. Metabonomics revealed that the neurotransmitters and plasma AAs in CD mice were dysregulated, namely the tyrosine and tryptophan pathways and monoamine neurotransmitters in the brain. Gut microbiota analysis displayed an increased ratio of Firmicutes/Bacteroides. In detail, the abundance of f_Lachnospiraceae and s_Lachnospiraceae increased, whereas the abundance of o_Bacteroidales and s_Bacteroides_caecimuris decreased. Moreover, the gut microbiota was more closely associated with AA neurotransmitters than with plasma AA.

CONCLUSION

Depression promoted the progression of BC by modulating the abundance of s_Lachnospiraceae and s_Bacteroides_caecimuris, which affected the metabolism of monoamine neurotransmitters in the brain and AA in the blood.

Ventral pallidum GABA and glutamate neurons drive approach and avoidance through distinct modulation of VTA cell types

The ventral pallidum (VP) contains GABA and glutamate neurons projecting to ventral tegmental area (VTA) whose stimulation drives approach and avoidance, respectively. Yet little is known about the mechanisms by which VP cell types shape VTA activity and drive behavior. Here, we found that both VP GABA and glutamate neurons were activated during approach to reward or by delivery of an aversive stimulus. Stimulation of VP GABA neurons inhibited VTA GABA, but activated dopamine and glutamate neurons. Remarkably, stimulation-evoked activation was behavior-contingent such that VTA recruitment was inhibited when evoked by the subject's own action. Conversely, VP glutamate neurons activated VTA GABA, as well as dopamine and glutamate neurons, despite driving aversion. However, VP glutamate neurons evoked dopamine in aversion-associated ventromedial nucleus accumbens (NAc), but reduced dopamine release in reward-associated dorsomedial NAc. These findings show how heterogeneous VP projections to VTA can be engaged to shape approach and avoidance behaviors.

Distributional coding of associative learning in discrete populations of midbrain dopamine neurons

Midbrain dopamine neurons are thought to play key roles in learning by conveying the difference between expected and actual outcomes. Recent evidence suggests diversity in dopamine signaling, yet it remains poorly understood how heterogeneous signals might be organized to facilitate the role of downstream circuits mediating distinct aspects of behavior. Here, we investigated the organizational logic of dopaminergic signaling by recording and labeling individual midbrain dopamine neurons during associative behavior. Our findings show that reward information and behavioral parameters are not only heterogeneously encoded but also differentially distributed across populations of dopamine neurons. Retrograde tracing and fiber photometry suggest that populations of dopamine neurons projecting to different striatal regions convey distinct signals. These data, supported by computational modeling, indicate that such distributional coding can maximize dynamic range and tailor dopamine signals to facilitate specialized roles of different striatal regions.

Dopamine across timescales and cell types: Relevance for phenotypes in Parkinson's disease progression

Dopamine neurons in the substantia nigra pars compacta (SNc) synthesize and release dopamine, a critical neurotransmitter for movement and learning. SNc dopamine neurons degenerate in Parkinson's Disease (PD), causing a host of motor and non-motor symptoms. Here, we review recent conceptual advances in our basic understanding of the dopamine system - including our rapidly advancing knowledge of dopamine neuron heterogeneity - with special attention to their importance for understanding PD. In PD patients, dopamine neuron degeneration progresses from lateral SNc to medial SNc, suggesting clinically relevant heterogeneity in dopamine neurons. With technical advances in dopamine system interrogation, we can understand the relevance of this heterogeneity for PD progression and harness it to develop new treatments.

Ventral pallidum neurons projecting to the ventral tegmental area reinforce but do not invigorate reward-seeking behavior

Reward-predictive cues acquire motivating and reinforcing properties that contribute to the escalation and relapse of drug use in addiction. The ventral pallidum (VP) and ventral tegmental area (VTA) are two key nodes in brain reward circuitry implicated in addiction and cue-driven behavior. In the current study, we use in vivo fiber photometry and optogenetics to record from and manipulate VP→VTA in rats performing a discriminative stimulus task to determine the role these neurons play in invigoration and reinforcement by reward cues. We find that VP→VTA neurons are active during reward consumption and that optogenetic stimulation of these neurons biases choice behavior and is reinforcing. Critically, we find no encoding of reward-seeking vigor, and optogenetic stimulation does not enhance the probability or vigor of reward seeking in response to cues. Our results suggest that VP→VTA activity is more important for reinforcement than for invigoration of reward seeking by cues.

Inhibitory Pedunculopontine Neurons Gate Dopamine-Mediated Motor Actions of Unsigned Valence

BACKGROUND

The pedunculopontine nucleus (PPN) maintains a bidirectional connectivity with the basal ganglia that supports their shared roles in the selection and execution of motor actions. Previous studies identified a role for PPN neurons in goal-directed behavior, but the cellular substrates underlying this function have not been elucidated. We recently revealed the existence of a monosynaptic GABAergic input from the PPN that inhibits dopamine neurons of the substantia nigra. Activation of this pathway interferes with the execution of learned motor sequences when the actions are rewarded, even though the inhibition of dopamine neurons did not shift the value of the action, hence suggesting executive control over the gating of behavior.

OBJECTIVE

To test the attributes of the inhibition of dopamine neurons by the PPN in the context of goal-directed behavior regardless of whether the outcome is positively or negatively reinforced.

METHODS

We delivered optogenetic stimulation to PPN GABAergic axon terminals in the substantia nigra during a battery of behavioral tasks with positive and negative valence.

RESULTS

Inhibition of dopamine neurons by PPN optogenetic activation during an appetitive task impaired the initiation and overall execution of the behavioral sequence without affecting the consumption of reward. During an active avoidance task, the same activation impaired the ability of mice to avoid a foot shock, but their escape response was unaffected. In addition, responses to potential threats were significantly attenuated.

CONCLUSION

Our results show that PPN GABAergic neurons modulate learned, goal-directed behavior of unsigned valence without affecting overall motor behavior.

Striatal response to negative feedback in a stop signal task operates as a multi-value learning signal

Background and aim

We examined error-driven learning in fMRI activity of 217 subjects in a stop signal task to obtain a more robust characterization of the relation between behavioral measures of learning and corresponding neural learning signals than previously possible.

Methods

The stop signal task is a two-alternative forced choice in which participants respond to an arrow by pressing a left or right button but must inhibit that response on 1 in 7 trials when cued by an auditory "stop signal." We examined post-error learning by comparing brain activity (BOLD signal) and behavioral responses on trials preceded by successful (correct stop) vs. failed (failed stop) inhibition.

Results

There was strong evidence of greater bilateral striatal activity in the period immediately following correct (vs. failed) stop trials (most evident in the putamen; peak MNI coordinates [-26 8 -2], 430 voxels, p < 0.001; [24 14 0], 527 voxels, p < 0.001). We measured median activity in the bilateral striatal cluster following every failed stop and correct stop trial and correlated it with learning signals for (a) probability and (b) latency of the stop signal. In a mixed-effects model predicting activity 5-10 s after the stop signal, both reaction time (RT) change (B = -0.05, t = 3.0, χ2 = 11.3, p < 0.001) and probability of stop trial change (B = 1.53, t = 6.0, χ2 = 43.0, p < 0.001) had significant within-subjects effects on median activity. In a similar mixed model predicting activity 1-5 s after the stop signal, only probability of stop trial change was predictive.

Conclusions

A mixed-effects model indicates the striatal activity might be a learning signal that encodes reaction time change and the current expected probability of a stop trial occuring. This extends existing evidence that the striatum encodes a reward prediction error signal for learning within the stop signal task, and demonstrates for the first time that this signal seems to encode both change in stop signal probability and in stop signal delay.

Inhibition of Dopamine Neurons Prevents Incentive Value Encoding of a Reward Cue: With Revelations from Deep Phenotyping

The survival of an organism is dependent on its ability to respond to cues in the environment. Such cues can attain control over behavior as a function of the value ascribed to them. Some individuals have an inherent tendency to attribute reward-paired cues with incentive motivational value, or incentive salience. For these individuals, termed sign-trackers, a discrete cue that precedes reward delivery becomes attractive and desirable in its own right. Prior work suggests that the behavior of sign-trackers is dopamine-dependent, and cue-elicited dopamine in the NAc is believed to encode the incentive value of reward cues. Here we exploited the temporal resolution of optogenetics to determine whether selective inhibition of ventral tegmental area (VTA) dopamine neurons during cue presentation attenuates the propensity to sign-track. Using male tyrosine hydroxylase (TH)-Cre Long Evans rats, it was found that, under baseline conditions, ∼84% of TH-Cre rats tend to sign-track. Laser-induced inhibition of VTA dopamine neurons during cue presentation prevented the development of sign-tracking behavior, without affecting goal-tracking behavior. When laser inhibition was terminated, these same rats developed a sign-tracking response. Video analysis using DeepLabCutTM revealed that, relative to rats that received laser inhibition, rats in the control group spent more time near the location of the reward cue even when it was not present and were more likely to orient toward and approach the cue during its presentation. These findings demonstrate that cue-elicited dopamine release is critical for the attribution of incentive salience to reward cues.SIGNIFICANCE STATEMENT Activity of dopamine neurons in the ventral tegmental area (VTA) during cue presentation is necessary for the development of a sign-tracking, but not a goal-tracking, conditioned response in a Pavlovian task. We capitalized on the temporal precision of optogenetics to pair cue presentation with inhibition of VTA dopamine neurons. A detailed behavioral analysis with DeepLabCutTM revealed that cue-directed behaviors do not emerge without dopamine neuron activity in the VTA. Importantly, however, when optogenetic inhibition is lifted, cue-directed behaviors increase, and a sign-tracking response develops. These findings confirm the necessity of dopamine neuron activity in the VTA during cue presentation to encode the incentive value of reward cues.

Alterations in Serotonin Neurotransmission in Hyperdopaminergic Rats Lacking the Dopamine Transporter

Biogenic amines dopamine (DA) and serotonin (5-HT) are among the most significant monoaminergic neurotransmitters in the central nervous system (CNS). Separately, the physiological roles of DA and 5-HT have been studied in detail, and progress has been made in understanding their roles in normal and various pathological conditions (Parkinson's disease, schizophrenia, addiction, depression, etc.). In this article we showed that knockout of the gene encoding DAT leads not only to a profound dysregulation of dopamine neurotransmission in the striatum but also in the midbrain, prefrontal cortex, hippocampus, medulla oblongata and spinal cord. Furthermore, significant changes were observed in the production of mRNA of enzymes of monoamine metabolism, as well as to a notable alteration in the tissue level of serotonin, most clearly manifested in the cerebellum and the spinal cord. The observed region-specific changes in the tissue levels of serotonin and in the expression of dopamine and serotonergic metabolism enzymes in rats with an excess of dopamine can indicate important consequences for the pharmacotherapy of drugs that modulate the dopaminergic system. The drugs that affect the dopaminergic system could potently affect the serotonergic system, and this fact is important to consider when predicting their possible therapeutic or side effects.

The dopamine hypothesis of autism spectrum disorder: A comprehensive analysis of the evidence

Despite intensive research into the etiopathogenesis of autism spectrum disorder (ASD), limited progress has been achieved so far. Among the plethora of models seeking to clarify how ASD arises, a coherent dopaminergic model was lacking until recently. In 2017, we provided a theoretical framework that we designated "the dopamine hypothesis of ASD". In the meantime, numerous studies yielded empirical evidence for this model. 4 years later, we provided a second version encompassing a refined and reconceptualized framework that accounted for these novel findings. In this chapter, we will review the evidence backing the previous versions of our model and add the most recent developments to the picture. Along these lines, we intend to lay out a comprehensive analysis of the supporting evidence for the dopamine hypothesis of ASD.

Prediction errors drive dynamic changes in neural patterns that guide behavior

Learning describes the process by which our internal expectation models of the world are updated by surprising outcomes (prediction errors [PEs]) to improve predictions of future events. However, the mechanisms through which error signals dynamically influence existing neural representations are unknown. Here, we use functional magnetic resonance imaging (fMRI) in humans solving a two-step Markov decision task to investigate changes in neural activation patterns following PEs. Using a dynamic multivariate pattern analysis, we can show that PE-related fMRI responses in error-coding regions predict trial-by-trial changes in multivariate neural patterns in the orbitofrontal cortex, the precuneus, and the ventromedial prefrontal cortex (vmPFC). Importantly, the dynamics of these pattern changes in the vmPFC also predicted upcoming changes in choice strategies and thus highlight the importance of these pattern changes for behavior.

Reward prediction error in learning-related behaviors

Learning is a complex process, during which our opinions and decisions are easily changed due to unexpected information. But the neural mechanism underlying revision and correction during the learning process remains unclear. For decades, prediction error has been regarded as the core of changes to perception in learning, even driving the learning progress. In this article, we reviewed the concept of reward prediction error, and the encoding mechanism of dopaminergic neurons and the related neural circuities. We also discussed the relationship between reward prediction error and learning-related behaviors, including reversal learning. We then demonstrated the evidence of reward prediction error signals in several neurological diseases, including Parkinson's disease and addiction. These observations may help to better understand the regulatory mechanism of reward prediction error in learning-related behaviors.

The study of plasticity has always been about gradients

The experimental study of learning and plasticity has always been driven by an implicit question: how can physiological changes be adaptive and improve performance? For example, in Hebbian plasticity only synapses from presynaptic neurons that were active are changed, avoiding useless changes. Similarly, in dopamine-gated learning synapse changes depend on reward or lack thereof and do not change when everything is predictable. Within machine learning we can make the question of which changes are adaptive concrete: performance improves when changes correlate with the gradient of an objective function quantifying performance. This result is general for any system that improves through small changes. As such, physiology has always implicitly been seeking mechanisms that allow the brain to approximate gradients. Coming from this perspective we review the existing literature on plasticity-related mechanisms, and we show how these mechanisms relate to gradient estimation. We argue that gradients are a unifying idea to explain the many facets of neuronal plasticity.

Ventral pallidum GABA and glutamate neurons drive approach and avoidance through distinct modulation of VTA cell types

The ventral pallidum (VP) contains GABA and glutamate (Glut) neurons projecting to ventral tegmental area (VTA) whose stimulation drives approach and avoidance, respectively. Yet little is known about the cell-type-specific mechanisms by which VP projections to VTA drive behavior. Here, we found that both VP GABA and Glut neurons were activated during approach to reward or delivery of an aversive stimulus. Stimulation of VP GABA neurons inhibited VTA GABA, but activated dopamine (DA) and glutamate neurons. Remarkably, this cell-type-specific recruitment was behavior-contingent such that VTA recruitment was inhibited when evoked by the subject's own action. Conversely, VP Glut neurons activated VTA GABA, as well as DA and Glut neurons, despite driving aversion. However, VP Glut neurons evoked DA in reward-associated ventromedial nucleus accumbens (NAc), but reduced DA in aversion-associated dorsomedial NAc. These findings show how heterogeneous VP cell types can engage VTA cell types to shape approach and avoidance behaviors.

Neurochemistry of the mammillary body

The mammillary body (MB) is a component of the extended hippocampal system and many studies have shown that its functions are vital for mnemonic processes. Together with other subcortical structures, such as the anterior thalamic nuclei and tegmental nuclei of Gudden, the MB plays a crucial role in the processing of spatial and working memory, as well as navigation in rats. The aim of this paper is to review the distribution of various substances in the MB of the rat, with a description of their possible physiological roles. The following groups of substances are reviewed: (1) classical neurotransmitters (glutamate and other excitatory transmitters, gamma-aminobutyric acid, acetylcholine, serotonin, and dopamine), (2) neuropeptides (enkephalins, substance P, cocaine- and amphetamine-regulated transcript, neurotensin, neuropeptide Y, somatostatin, orexins, and galanin), and (3) other substances (calcium-binding proteins and calcium sensor proteins). This detailed description of the chemical parcellation may facilitate a better understanding of the MB functions and its complex relations with other structures of the extended hippocampal system.

Inhibition of dopamine neurons prevents incentive value encoding of a reward cue: With revelations from deep phenotyping

The survival of an organism is dependent on their ability to respond to cues in the environment. Such cues can attain control over behavior as a function of the value ascribed to them. Some individuals have an inherent tendency to attribute reward-paired cues with incentive motivational value, or incentive salience. For these individuals, termed sign-trackers, a discrete cue that precedes reward delivery becomes attractive and desirable in its own right. Prior work suggests that the behavior of sign-trackers is dopamine-dependent, and cue-elicited dopamine in the nucleus accumbens is believed to encode the incentive value of reward cues. Here we exploited the temporal resolution of optogenetics to determine whether selective inhibition of ventral tegmental area (VTA) dopamine neurons during cue presentation attenuates the propensity to sign-track. Using male tyrosine hydroxylase (TH)-Cre Long Evans rats it was found that, under baseline conditions, ∼84% of TH-Cre rats tend to sign-track. Laser-induced inhibition of VTA dopamine neurons during cue presentation prevented the development of sign-tracking behavior, without affecting goal-tracking behavior. When laser inhibition was terminated, these same rats developed a sign-tracking response. Video analysis using DeepLabCut revealed that, relative to rats that received laser inhibition, rats in the control group spent more time near the location of the reward cue even when it was not present and were more likely to orient towards and approach the cue during its presentation. These findings demonstrate that cue-elicited dopamine release is critical for the attribution of incentive salience to reward cues.

Significance Statement

Activity of dopamine neurons in the ventral tegmental area (VTA) during cue presentation is necessary for the development of a sign-tracking, but not a goal-tracking, conditioned response in a Pavlovian task. We capitalized on the temporal precision of optogenetics to pair cue presentation with inhibition of VTA dopamine neurons. A detailed behavioral analysis with DeepLabCut revealed that cue-directed behaviors do not emerge without VTA dopamine. Importantly, however, when optogenetic inhibition is lifted, cue-directed behaviors increase, and a sign-tracking response develops. These findings confirm the necessity of VTA dopamine during cue presentation to encode the incentive value of reward cues.

The effects of propofol anaesthesia on molecular-enriched networks during resting-state and naturalistic listening

Placing a patient in a state of anaesthesia is crucial for modern surgical practice. However, the mechanisms by which anaesthetic drugs, such as propofol, impart their effects on consciousness remain poorly understood. Propofol potentiates GABAergic transmission, which purportedly has direct actions on cortex as well as indirect actions via ascending neuromodulatory systems. Functional imaging studies to date have been limited in their ability to unravel how these effects on neurotransmission impact the system-level dynamics of the brain. Here, we leveraged advances in multi-modal imaging, Receptor-Enriched Analysis of functional Connectivity by Targets (REACT), to investigate how different levels of propofol-induced sedation alter neurotransmission-related functional connectivity (FC), both at rest and when individuals are exposed to naturalistic auditory stimulation. Propofol increased GABA-A- and noradrenaline transporter-enriched FC within occipital and somatosensory regions respectively. Additionally, during auditory stimulation, the network related to the dopamine transporter showed reduced FC within bilateral regions of temporal and mid/posterior cingulate cortices, with the right temporal cluster showing an interaction between auditory stimulation and level of consciousness. In bringing together these micro- and macro-scale systems, we provide support for both direct GABAergic and indirect noradrenergic and dopaminergic-related network changes under propofol sedation. Further, we delineate a cognition-related reconfiguration of the dopaminergic network, highlighting the utility of REACT to explore the molecular substrates of consciousness and cognition.

A dopaminergic reward prediction error signal shapes maternal behavior in mice

How social contact is perceived as rewarding and subsequently modifies interactions is unclear. Dopamine (DA) from the ventral tegmental area (VTA) regulates sociality, but the ongoing, unstructured nature of free behavior makes it difficult to ascertain how. Here, we tracked the emergence of a repetitive stereotyped parental retrieval behavior and conclude that VTA DA neurons incrementally refine it by reinforcement learning (RL). Trial-by-trial performance was correlated with the history of DA neuron activity, but DA signals were inconsistent with VTA directly influencing the current trial. We manipulated the subject's expectation of imminent pup contact and show that DA signals convey reward prediction error, a fundamental component of RL. Finally, closed-loop optogenetic inactivation of DA neurons at the onset of pup contact dramatically slowed emergence of parental care. We conclude that this component of maternal behavior is shaped by an RL mechanism in which social contact itself is the primary reward.

Digitally embodied lifespan neurocognitive development and Tactile Internet: Transdisciplinary challenges and opportunities

Mechanisms underlying perceptual processing and inference undergo substantial changes across the lifespan. If utilized properly, technologies could support and buffer the relatively more limited neurocognitive functions in the still developing or aging brains. Over the past decade, a new type of digital communication infrastructure, known as the "Tactile Internet (TI)," is emerging in the fields of telecommunication, sensor and actuator technologies and machine learning. A key aim of the TI is to enable humans to experience and interact with remote and virtual environments through digitalized multimodal sensory signals that also include the haptic (tactile and kinesthetic) sense. Besides their applied focus, such technologies may offer new opportunities for the research tapping into mechanisms of digitally embodied perception and cognition as well as how they may differ across age cohorts. However, there are challenges in translating empirical findings and theories about neurocognitive mechanisms of perception and lifespan development into the day-to-day practices of engineering research and technological development. On the one hand, the capacity and efficiency of digital communication are affected by signal transmission noise according to Shannon's (1949) Information Theory. On the other hand, neurotransmitters, which have been postulated as means that regulate the signal-to-noise ratio of neural information processing (e.g., Servan-Schreiber et al., 1990), decline substantially during aging. Thus, here we highlight neuronal gain control of perceptual processing and perceptual inference to illustrate potential interfaces for developing age-adjusted technologies to enable plausible multisensory digital embodiments for perceptual and cognitive interactions in remote or virtual environments.

Combined EEG and immersive virtual reality unveil dopaminergic modulation of error monitoring in Parkinson's Disease

Detecting errors in your own and others' actions is associated with discrepancies between intended and expected outcomes. The processing of salient events is associated with dopamine release, the balance of which is altered in Parkinson's disease (PD). Errors in observed actions trigger various electrocortical indices (e.g. mid-frontal theta, error-related delta, and error positivity [oPe]). However, the impact of dopamine depletion to observed errors in the same individual remains unclear. Healthy controls (HCs) and PD patients observed ecological reach-to-grasp-a-glass actions performed by a virtual arm from a first-person perspective. PD patients were tested under their dopaminergic medication (on-condition) and after dopaminergic withdrawal (off-condition). Analyses of oPe, delta, and theta-power increases indicate that while the formers were elicited after incorrect vs. correct actions in all groups, the latter were observed in on-condition but altered in off-condition PD. Therefore, different EEG error signatures may index the activity of distinct mechanisms, and error-related theta power is selectively modulated by dopamine depletion. Our findings may facilitate discovering dopamine-related biomarkers for error-monitoring dysfunctions that may have crucial theoretical and clinical implications.

A multi-faceted role of dual-state dopamine signaling in working memory, attentional control, and intelligence

Genetic evidence strongly suggests that individual differences in intelligence will not be reducible to a single dominant cause. However, some of those variations/changes may be traced to tractable, cohesive mechanisms. One such mechanism may be the balance of dopamine D1 (D₁R) and D2 (D₂R) receptors, which regulate intrinsic currents and synaptic transmission in frontal cortical regions. Here, we review evidence from human, animal, and computational studies that suggest that this balance (in density, activity state, and/or availability) is critical to the implementation of executive functions such as attention and working memory, both of which are principal contributors to variations in intelligence. D1 receptors dominate neural responding during stable periods of short-term memory maintenance (requiring attentional focus), while D2 receptors play a more specific role during periods of instability such as changing environmental or memory states (requiring attentional disengagement). Here we bridge these observations with known properties of human intelligence. Starting from theories of intelligence that place executive functions (e.g., working memory and attentional control) at its center, we propose that dual-state dopamine signaling might be a causal contributor to at least some of the variation in intelligence across individuals and its change by experiences/training. Although it is unlikely that such a mechanism can account for more than a modest portion of the total variance in intelligence, our proposal is consistent with an array of available evidence and has a high degree of explanatory value. We suggest future directions and specific empirical tests that can further elucidate these relationships.

Cerebellar Contributions to the Basal Ganglia Influence Motor Coordination, Reward Processing, and Movement Vigor

Both the cerebellum and the basal ganglia are known for their roles in motor control and motivated behavior. These two systems have been classically considered as independent structures that coordinate their contributions to behavior via separate cortico-thalamic loops. However, recent evidence demonstrates the presence of a rich set of direct connections between these two regions. Although there is strong evidence for connections in both directions, for brevity we limit our discussion to the better-characterized connections from the cerebellum to the basal ganglia. We review two sets of such connections: disynaptic projections through the thalamus and direct monosynaptic projections to the midbrain dopaminergic nuclei, the VTA and the SNc. In each case, we review the evidence for these pathways from anatomic tracing and physiological recordings, and discuss their potential functional roles. We present evidence that the disynaptic pathway through the thalamus is involved in motor coordination, and that its dysfunction contributes to motor deficits, such as dystonia. We then discuss how cerebellar projections to the VTA and SNc influence dopamine release in the respective targets of these nuclei: the NAc and the dorsal striatum. We argue that the cerebellar projections to the VTA may play a role in reward-based learning and therefore contribute to addictive behavior, whereas the projection to the SNc may contribute to movement vigor. Finally, we speculate how these projections may explain many of the observations that indicate a role for the cerebellum in mental disorders, such as schizophrenia.