From Prediction to Action: Dissociable Roles of Ventral Tegmental Area and Substantia Nigra Dopamine Neurons in Instrumental Reinforcement

Developing algorithmic psychiatry via multi-level spanning computational models

Modern psychiatry faces challenges in translating neurobiological insights into treatments for severe illnesses. The mid-20th century witnessed the rise of molecular mechanisms as pathophysiological and treatment models, with recent holistic proposals keeping this focus unaltered. In this perspective, we explore how psychiatry can utilize systems neuroscience to develop a vertically integrated understanding of brain function to inform treatment. Using schizophrenia as a case study, we discuss scale-related challenges faced by researchers studying molecules, circuits, networks, and cognition and clinicians operating within existing frameworks. We emphasize computation as a bridging language, with algorithmic models like hierarchical predictive processing offering explanatory potential for targeted interventions. Developing such models will not only facilitate new interventions but also optimize combining existing treatments by predicting their multi-level effects. We conclude with the prognosis that the future is bright, but that continued investment in research closely driven by clinical realities will be critical.

20 Years of Aberrant Salience in Psychosis: What Have We Learned?

Twenty years ago Shitij Kapur's "Psychosis as a state of aberrant salience" captured the attention of clinicians and cognitive and behavioral neuroscientists. It has become the de facto way of talking about delusion formation in labs and clinics. Here, evidence for this theory is critically evaluated in consideration of evolving data since its publication. A particular focus is placed on its specific predictions regarding the neural and behavioral loci of dopamine dysfunction in psychosis and finds them lacking. This examination is informed by recent advances in the understanding of the function of the dopamine system and its impacts on behavior following the explosion of new tools and probes for precise measurement and manipulation of dopaminergic circuits. Contemporary theories that have developed since Kapur-which suggest a role for dopamine in belief formation, belief updating under uncertainty, and abductive inference to the best explanation for some set of circumstances-are argued to form a more cogent theory that fits better with the work in patients with delusions and hallucinations, how they behave, and what is known about the function of their dopamine system. The original salience hypothesis has been influential as it attempted to unite neurochemical dysfunction with clinical phenomenology through computational cognitive neuroscience, which has led to the development of novel predictions that the authors highlight as future directions for the field.

Contextual cues facilitate dynamic value encoding in the mesolimbic dopamine system

Adaptive behavior in a dynamic environmental context often requires rapid revaluation of stimuli that deviates from well-learned associations. The divergence between stable value-encoding and appropriate behavioral output remains a critical component of theories of dopamine's function in learning, motivation, and motor control. Yet, how dopamine neurons are involved in the revaluation of cues when the world changes, to alter our behavior, remains unclear. Here, we make use of a complementary set of in vivo approaches to clarify the contributions of the mesolimbic dopamine system to the dynamic reorganization of reward- seeking behavior. Male and female rats were trained to discriminate when a conditioned stimulus would be followed by a sucrose reward by exploiting the prior, non-overlapping presentation of a another discrete cue-an occasion setter. Only when the occasion setter's presentation preceded the conditioned stimulus did the conditioned stimulus predict sucrose delivery, dissociating the average value of the conditioned stimulus from its immediate value, on a trial-to-trial basis. Activity of ventral tegmental area dopamine neurons was essential for rats to successfully update behavioral response to the occasion setter. Moreover, dopamine release in the nucleus accumbens following the conditioned stimulus only occurred when the occasion setter indicated it would predict reward and did not reflect its average expected value. Downstream of dopamine release, we found that neurons in the nucleus accumbens dynamically tracked the value of the conditioned stimulus. Together, these results help refine notions of dopamine function, revealing a prominent contribution of the mesolimbic dopamine system to the rapid revaluation of motivation.

Unconscious will as a neurobehavioral mechanism against adversity

Incentive salience theory both explains the directional component of motivation (in terms of cue attraction or "wanting") and its energetic component, as a function of the strength of cue attraction. This theory characterizes cue- and reward-triggered approach behavior. But it does not tell us how behavior can show enhanced vigor under reward uncertainty, when cues are inconsistent or resources hidden. Reinforcement theory is also ineffective in explaining enhanced vigor in case reward expectation is low or nil. This paper provides a neurobehavioral interpretation of effort in situations of adversity (which always include some uncertainty about outcomes) that is complementary to the attribution of incentive salience to environmental cues. It is argued that manageable environmental challenges activate an unconscious process of self-determination to achieve "wanted" actions. This unconscious process is referred to as incentive effort, which involves the hypothalamo-pituitary-adrenal (HPA) axis, noradrenaline, as well as striatal dopamine. Concretely, HPA-induced dopamine release would have the function to make effort-or effortful actions-"wanted" in a challenging context, in which the environmental cues are poorly predictive of reward-i.e., unattractive. Stress would only emerge in the presence of unmanageable challenges. It is hypothesized that incentive effort is the core psychological basis of will-and is, for this reason, termed "willing."

Sequence termination cues drive habits via dopamine-mediated credit assignment

Mesolimbic dopamine (DA) neurons are central to sequence learning and habit formation. Yet, the mechanisms by which cue-induced DA neural activity drives goal-directed or habitual sequence execution remain unknown. We designed two novel tasks to investigate how sequence initiation and termination cues influence DA-driven behavioral strategies and learning. We found that sequence initiation and termination cues differentially affect reward expectation during action sequences, with only the termination cue contributing to greater outcome devaluation insensitivity, automaticity and behavioral chunking. Mesolimbic fiber photometry recording revealed that this habit-like behavior was associated with a rapid backpropagation in DA signals from the reward to the immediately preceding cue and with attenuated DA reward prediction error signals, which reflected greater behavioral inflexibility. Finally, in absence of external cues, brief optogenetic stimulation of VTA DA neurons at sequence termination was sufficient to drive automaticity and behavioral chunking. Our results highlight the critical role of cue-evoked DA signals at sequence termination in mediating credit assignment and driving the development of habitual action sequence execution.

The Incentive-Sensitization Theory of Addiction 30 Years On

The incentive-sensitization theory (IST) of addiction was first published in 1993, proposing that (a) brain mesolimbic dopamine systems mediate incentive motivation ("wanting") for addictive drugs and other rewards, but not their hedonic impact (liking) when consumed; and (b) some individuals are vulnerable to drug-induced long-lasting sensitization of mesolimbic systems, which selectively amplifies their "wanting" for drugs without increasing their liking of the same drugs. Here we describe the origins of IST and evaluate its status 30 years on. We compare IST to other theories of addiction, including opponent-process theories, habit theories of addiction, and prefrontal cortical dysfunction theories of impaired impulse control. We also address critiques of IST that have been raised over the years, such as whether craving is important in addiction and whether addiction can ever be characterized as compulsive. Finally, we discuss several contemporary phenomena, including the potential role of incentive sensitization in behavioral addictions, the emergence of addiction-like dopamine dysregulation syndrome in medicated Parkinson's patients, the role of attentional capture and approach tendencies, and the role of uncertainty in incentive motivation.

Impact of supplementation with omega-3 fatty acids after maternal dietary deficiency on adolescent anxiety and microglial morphology

Dietary maternal deficiency in omega-3 polyunsaturated fatty acids (n-3 PUFA) is a potential risk factor for the development of anxiety and other mood disorders in children and adolescents. Here, we used a previously characterized maternal n-3 PUFA dietary deficiency model in rats to determine the impact of postweaning supplementation on adolescent anxiety-like behaviors. We focused on two models of anxiety: innate anxiety tested by the elevated plus maze and a novel operant model of learned anxiety where animals learn that actions may be associated with a variable probability of harm. Given that recent basic and clinical studies have associated anxiety and other adverse effects of n-3 PUFA deficiency on inflammatory processes and microglial structure and function, we also assessed the impact of our dietary deficiency model and supplementation on adolescent microglial morphology in multiple brain regions. We found that the male and female adolescent n-3 PUFA-deficient groups exhibit increased innate anxiety, but only females showed enhanced learned anxiety. Supplementation after weaning did not significantly affect innate anxiety but ameliorated learned anxiety in females. Thus, the beneficial effects of supplementation on adolescent anxiety may be sex-specific and depend on the type of anxiety. We also found that n-3 PUFA deficiency influences microglia function in adolescents in the amygdala and nigrostriatal, but not mesolimbic, brain regions. Collectively, these data suggest that while n-3 PUFA dietary supplementation may be effective in reducing adolescent anxiety, this effect is context-, sex-, and brain network-specific. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning

Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward rewards. Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) are crucial for this process, via engagement of a reciprocally connected network with their striatal targets. Critically, it remains unknown how dopamine neuron activity itself engages dopamine signals throughout the striatum, across learning. Here, we investigated how optogenetic Pavlovian cue conditioning of VTA or SNc dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. Although VTA stimulation-evoked robust dopamine release in NAc core, shell, and DMS, predictive cues preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS. Despite the development of vigorous movement late in training, conditioned dopamine signals did not emerge in the DLS, even during Pavlovian conditioning with SNc dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show a broad dissociation in the fundamental prediction and reward-related information generated by VTA and SNc dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale adaptations across the striatal network emerge during learning to coordinate behavior.

Cognitive representations of intracranial self-stimulation of midbrain dopamine neurons depend on stimulation frequency

Dopamine neurons in the ventral tegmental area support intracranial self-stimulation (ICSS), yet the cognitive representations underlying this phenomenon remain unclear. Here, 20-Hz stimulation of dopamine neurons, which approximates a physiologically relevant prediction error, was not sufficient to support ICSS beyond a continuously reinforced schedule and did not endow cues with a general or specific value. However, 50-Hz stimulation of dopamine neurons was sufficient to drive robust ICSS and was represented as a specific reward to motivate behavior. The frequency dependence of this effect is due to the rate (not the number) of action potentials produced by dopamine neurons, which differently modulates dopamine release downstream.

Rethinking dopamine-guided action sequence learning

As opposed to those requiring a single action for reward acquisition, tasks necessitating action sequences demand that animals learn action elements and their sequential order and sustain the behaviour until the sequence is completed. With repeated learning, animals not only exhibit precise execution of these sequences but also demonstrate enhanced smoothness and efficiency. Previous research has demonstrated that midbrain dopamine and its major projection target, the striatum, play crucial roles in these processes. Recent studies have shown that dopamine from the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA) serve distinct functions in action sequence learning. The distinct contributions of dopamine also depend on the striatal subregions, namely the ventral, dorsomedial and dorsolateral striatum. Here, we have reviewed recent findings on the role of striatal dopamine in action sequence learning, with a focus on recent rodent studies.

Mesostriatal dopamine is sensitive to changes in specific cue-reward contingencies

Learning causal relationships relies on understanding how often one event precedes another. To investigate how dopamine neuron activity and neurotransmitter release change when a retrospective relationship is degraded for a specific pair of events, we used outcome-selective Pavlovian contingency degradation in rats. Conditioned responding was attenuated for the cue-reward contingency that was degraded, as was dopamine neuron activity in the midbrain and dopamine release in the ventral striatum in response to the cue and subsequent reward. Contingency degradation also abolished the trial-by-trial history dependence of the dopamine responses at the time of trial outcome. This profile of changes in cue- and reward-evoked responding is not easily explained by a standard reinforcement learning model. An alternative model based on learning causal relationships was better able to capture dopamine responses during contingency degradation, as well as conditioned behavior following optogenetic manipulations of dopamine during noncontingent rewards. Our results suggest that mesostriatal dopamine encodes the contingencies between meaningful events during learning.

Dopamine across timescales and cell types: Relevance for phenotypes in Parkinson's disease progression

Dopamine neurons in the substantia nigra pars compacta (SNc) synthesize and release dopamine, a critical neurotransmitter for movement and learning. SNc dopamine neurons degenerate in Parkinson's Disease (PD), causing a host of motor and non-motor symptoms. Here, we review recent conceptual advances in our basic understanding of the dopamine system - including our rapidly advancing knowledge of dopamine neuron heterogeneity - with special attention to their importance for understanding PD. In PD patients, dopamine neuron degeneration progresses from lateral SNc to medial SNc, suggesting clinically relevant heterogeneity in dopamine neurons. With technical advances in dopamine system interrogation, we can understand the relevance of this heterogeneity for PD progression and harness it to develop new treatments.

Sensory Cues Potentiate VTA Dopamine Mediated Reinforcement

Sensory cues are critical for shaping decisions and invigorating actions during reward seeking. Dopamine neurons in the ventral tegmental area (VTA) are central in this process, supporting associative learning in Pavlovian and instrumental settings. Studies of intracranial self-stimulation (ICSS) behavior, which show that animals will work hard to receive stimulation of dopamine neurons, support the notion that dopamine transmits a reward or value signal to support learning. Recent studies have begun to question this, however, emphasizing dopamine's value-free functions, leaving its contribution to behavioral reinforcement somewhat muddled. Here, we investigated the role of sensory stimuli in dopamine-mediated reinforcement, using an optogenetic ICSS paradigm in tyrosine hydroxylase (TH)-Cre rats. We find that while VTA dopamine neuron activation in the absence of explicit external cues is sufficient to maintain robust self-stimulation, the presence of cues dramatically potentiates ICSS behavior. Our results support a framework where dopamine can have some base value as a reinforcer, but the impact of this signal is modulated heavily by the sensory learning context.

Nucleus accumbens and dorsal medial striatal dopamine and neural activity are essential for action sequence performance

Separable striatal circuits have unique functions in Pavlovian and instrumental behaviors but how these roles relate to performance of sequences of actions with and without associated cues are less clear. Here, we tested whether dopamine transmission and neural activity more generally in three striatal subdomains are necessary for performance of an action chain leading to reward delivery. Male and female Long-Evans rats were trained to press a series of three spatially distinct levers to receive reward. We assessed the contribution of neural activity or dopamine transmission within each striatal subdomain when progression through the action sequence was explicitly cued and in the absence of cues. Behavior in both task variations was substantially impacted following microinfusion of the dopamine antagonist, flupenthixol, into nucleus accumbens core (NAc) or dorsomedial striatum (DMS), with impairments in sequence timing and numbers of rewards earned after NAc flupenthixol. In contrast, after pharmacological inactivation to suppress overall activity, there was minimal impact on total rewards earned. Instead, inactivation of both NAc and DMS impaired sequence timing and led to sequence errors in the uncued, but not cued task. There was no impact of dopamine antagonism or reversible inactivation of dorsolateral striatum on either cued or uncued action sequence completion. These results highlight an essential contribution of NAc and DMS dopamine systems in motivational and performance aspects of chains of actions, whether cued or internally generated, as well as the impact of intact NAc and DMS function for correct sequence performance.

Prefrontal Dopamine in Flexible Adaptation to Environmental Changes: A Game for Two Players

Deficits in cognitive flexibility have been characterized in affective, anxiety, and neurodegenerative disorders. This paper reviews data, mainly from studies on animal models, that support the existence of a cortical-striatal brain circuit modulated by dopamine (DA), playing a major role in cognitive/behavioral flexibility. Moreover, we reviewed clinical findings supporting misfunctioning of this circuit in Parkinson's disease that could be responsible for some important non-motoric symptoms. The reviewed findings point to a role of catecholaminergic transmission in the medial prefrontal cortex (mpFC) in modulating DA's availability in the nucleus accumbens (NAc), as well as a role of NAc DA in modulating the motivational value of natural and conditioned stimuli. The review section is accompanied by a preliminary experiment aimed at testing weather the extinction of a simple Pavlovian association fosters increased DA transmission in the mpFC and inhibition of DA transmission in the NAc.