Latent inhibition of conditioned dopamine release in rat nucleus accumbens

65 years of research on dopamine's role in classical fear conditioning and extinction: A systematic review

Dopamine, a catecholamine neurotransmitter, has historically been associated with the encoding of reward, whereas its role in aversion has received less attention. Here, we systematically gathered the vast evidence of the role of dopamine in the simplest forms of aversive learning: classical fear conditioning and extinction. In the past, crude methods were used to augment or inhibit dopamine to study its relationship with fear conditioning and extinction. More advanced techniques such as conditional genetic, chemogenic and optogenetic approaches now provide causal evidence for dopamine's role in these learning processes. Dopamine neurons encode conditioned stimuli during fear conditioning and extinction and convey the signal via activation of D1-4 receptor sites particularly in the amygdala, prefrontal cortex and striatum. The coordinated activation of dopamine receptors allows for the continuous formation, consolidation, retrieval and updating of fear and extinction memory in a dynamic and reciprocal manner. Based on the reviewed literature, we conclude that dopamine is crucial for the encoding of classical fear conditioning and extinction and contributes in a way that is comparable to its role in encoding reward.

From safety to frustration: The neural substrates of inhibitory learning in aversive and appetitive conditioning procedures

Inhibitory associative learning counters the effects of excitatory learning, whether appetitively or aversively motivated. Moreover, the affective responses accompanying the inhibitory associations are of opponent valence to the excitatory conditioned responses. Inhibitors for negative aversive outcomes (e.g. shock) signal safety, while inhibitors for appetitive outcomes (e.g. food reward) elicit frustration and/or disappointment. This raises the question as to whether studies using appetitive and aversive conditioning procedures should demonstrate the same neural substrates for inhibitory learning. We review the neural substrates of appetitive and aversive inhibitory learning as measured in different procedural variants and in the context of the underpinning excitatory conditioning on which it depends. The mesocorticolimbic dopamine pathways, retrosplenial cortex and hippocampus are consistently implicated in inhibitory learning. Further neural substrates identified in some procedural variants may be related to the specific motivation of the learning task and modalities of the learning cues. Finally, we consider the translational implications of our understanding of the neural substrates of inhibitory learning, for obesity and addictions as well as for anxiety disorders.

Nucleus accumbens dopamine tracks aversive stimulus duration and prediction but not value or prediction error

There is active debate on the role of dopamine in processing aversive stimuli, where inferred roles range from no involvement at all, to signaling an aversive prediction error (APE). Here, we systematically investigate dopamine release in the nucleus accumbens core (NAC), which is closely linked to reward prediction errors, in rats exposed to white noise (WN, a versatile, underutilized, aversive stimulus) and its predictive cues. Both induced a negative dopamine ramp, followed by slow signal recovery upon stimulus cessation. In contrast to reward conditioning, this dopamine signal was unaffected by WN value, context valence, or probabilistic contingencies, and the WN dopamine response shifted only partially toward its predictive cue. However, unpredicted WN provoked slower post-stimulus signal recovery than predicted WN. Despite differing signal qualities, dopamine responses to simultaneous presentation of rewarding and aversive stimuli were additive. Together, our findings demonstrate that instead of an APE, NAC dopamine primarily tracks prediction and duration of aversive events.

Neural signals implicated in the processing of appetitive and aversive events in social and non-social contexts

In 2014, we participated in a special issue of Frontiers examining the neural processing of appetitive and aversive events. Specifically, we reviewed brain areas that contribute to the encoding of prediction errors and value versus salience, attention and motivation. Further, we described how we disambiguated these cognitive processes and their neural substrates by using paradigms that incorporate both appetitive and aversive stimuli. We described a circuit in which the orbitofrontal cortex (OFC) signals expected value and the basolateral amygdala (BLA) encodes the salience and valence of both appetitive and aversive events. This information is integrated by the nucleus accumbens (NAc) and dopaminergic (DA) signaling in order to generate prediction and prediction error signals, which guide decision-making and learning via the dorsal striatum (DS). Lastly, the anterior cingulate cortex (ACC) is monitoring actions and outcomes, and signals the need to engage attentional control in order to optimize behavioral output. Here, we expand upon this framework, and review our recent work in which within-task manipulations of both appetitive and aversive stimuli allow us to uncover the neural processes that contribute to the detection of outcomes delivered to a conspecific and behaviors in social contexts. Specifically, we discuss the involvement of single-unit firing in the ACC and DA signals in the NAc during the processing of appetitive and aversive events in both social and non-social contexts.

Dopamine signaling in the nucleus accumbens core mediates latent inhibition

Studies investigating the neural mechanisms by which associations between cues and predicted outcomes control behavior often use associative learning frameworks to understand the neural control of behavior. These frameworks do not always account for the full range of effects that novelty can have on behavior and future associative learning. Here, in mice, we show that dopamine in the nucleus accumbens core is evoked by novel, neutral stimuli, and the trajectory of this response over time tracked habituation to these stimuli. Habituation to novel cues before associative learning reduced future associative learning, a phenomenon known as latent inhibition. Crucially, trial-by-trial dopamine response patterns tracked this phenomenon. Optogenetic manipulation of dopamine responses to the cue during the habituation period bidirectionally influenced future associative learning. Thus, dopamine signaling in the nucleus accumbens core has a causal role in novelty-based learning in a way that cannot be predicted based on purely associative factors.

Dopamine and fear memory formation in the human amygdala

Learning which environmental cues that predict danger is crucial for survival and accomplished through Pavlovian fear conditioning. In humans and rodents alike, fear conditioning is amygdala-dependent and rests on similar neurocircuitry. Rodent studies have implicated a causative role for dopamine in the amygdala during fear memory formation, but the role of dopamine in aversive learning in humans is unclear. Here, we show dopamine release in the amygdala and striatum during fear learning in humans. Using simultaneous positron emission tomography and functional magnetic resonance imaging, we demonstrate that the amount of dopamine release is linked to strength of conditioned fear responses and linearly coupled to learning-induced activity in the amygdala. Thus, like in rodents, formation of amygdala-dependent fear memories in humans seems to be facilitated by endogenous dopamine release, supporting an evolutionary conserved neurochemical mechanism for aversive memory formation.

Dopamine release in the nucleus accumbens core signals perceived saliency

A large body of work has aimed to define the precise information encoded by dopaminergic projections innervating the nucleus accumbens (NAc). Prevailing models are based on reward prediction error (RPE) theory, in which dopamine updates associations between rewards and predictive cues by encoding perceived errors between predictions and outcomes. However, RPE cannot describe multiple phenomena to which dopamine is inextricably linked, such as behavior driven by aversive and neutral stimuli. We combined a series of behavioral tasks with direct, subsecond dopamine monitoring in the NAc of mice, machine learning, computational modeling, and optogenetic manipulations to describe behavior and related dopamine release patterns across multiple contingencies reinforced by differentially valenced outcomes. We show that dopamine release only conforms to RPE predictions in a subset of learning scenarios but fits valence-independent perceived saliency encoding across conditions. Here, we provide an extended, comprehensive framework for accumbal dopamine release in behavioral control.

Dopamine modulates subcortical responses to surprising sounds

Dopamine guides behavior and learning through pleasure, according to classic understanding. Dopaminergic neurons are traditionally thought to signal positive or negative prediction errors (PEs) when reward expectations are, respectively, exceeded or not matched. These signed PEs are quite different from the unsigned PEs, which report surprise during sensory processing. But mounting theoretical accounts from the predictive processing framework postulate that dopamine, as a neuromodulator, could potentially regulate the postsynaptic gain of sensory neurons, thereby scaling unsigned PEs according to their expected precision or confidence. Despite ample modeling work, the physiological effects of dopamine on the processing of surprising sensory information are yet to be addressed experimentally. In this study, we tested how dopamine modulates midbrain processing of unexpected tones. We recorded extracellular responses from the rat inferior colliculus to oddball and cascade sequences, before, during, and after the microiontophoretic application of dopamine or eticlopride (a D₂-like receptor antagonist). Results demonstrate that dopamine reduces the net neuronal responsiveness exclusively to unexpected sensory input without significantly altering the processing of expected input. We conclude that dopaminergic projections from the thalamic subparafascicular nucleus to the inferior colliculus could encode the expected precision of unsigned PEs, attenuating via D₂-like receptors the postsynaptic gain of sensory inputs forwarded by the auditory midbrain neurons. This direct dopaminergic modulation of sensory PE signaling has profound implications for both the predictive coding framework and the understanding of dopamine function.

Information about unexpected stimuli is encoded in the form of prediction error signals. The earliest prediction error signals identified in the auditory brain emerge subcortically in the inferior colliculus. This study reveals the essential role of dopamine in encoding the precision of prediction errors at the auditory midbrain.

Distinct signals in medial and lateral VTA dopamine neurons modulate fear extinction at different times

Dopamine (DA) neurons are thought to encode reward prediction error (RPE), in addition to other signals, such as salience. While RPE is known to support learning, the role of salience in learning remains less clear. To address this, we recorded and manipulated VTA DA neurons in mice during fear extinction. We applied deep learning to classify mouse freezing behavior, eliminating the need for human scoring. Our fiber photometry recordings showed DA neurons in medial and lateral VTA have distinct activity profiles during fear extinction: medial VTA activity more closely reflected RPE, while lateral VTA activity more closely reflected a salience-like signal. Optogenetic inhibition of DA neurons in either region slowed fear extinction, with the relevant time period for inhibition differing across regions. Our results indicate salience-like signals can have similar downstream consequences to RPE-like signals, although with different temporal dependencies.

Drug instrumentalization

Psychoactive drugs with addiction potential are widely used by people of virtually all cultures in a non-addictive way. In order to understand this behaviour, its population penetrance, and its persistence, drug instrumentalization was suggested as a driving force for this consumption. Drug instrumentalization theory holds that psychoactive drugs are consumed in a very systematic way in order to make other, non-drug-related behaviours more efficient. Here, we review the evolutionary origin of this behaviour and its psychological mechanisms and explore the neurobiological and neuropharmacological mechanisms underlying them. Instrumentalization goals are discussed, for which an environmentally selective and mental state-dependent consumption of psychoactive drugs can be learned and maintained in a non-addictive way. A small percentage of people who regularly instrumentalize psychoactive drugs make a transition to addiction, which often starts with qualitative and quantitative changes in the instrumentalization goals. As such, addiction is proposed to develop from previously established long-term drug instrumentalization. Thus, preventing and treating drug addiction in an individualized medicine approach may essentially require understanding and supporting personal instrumentalization goals.

Non-pharmacological factors that determine drug use and addiction

Based on their pharmacological properties, psychoactive drugs are supposed to take control of the natural reward system to finally drive compulsory drug seeking and consumption. However, psychoactive drugs are not used in an arbitrary way as pure pharmacological reinforcement would suggest, but rather in a highly specific manner depending on non-pharmacological factors. While pharmacological effects of psychoactive drugs are well studied, neurobiological mechanisms of non-pharmacological factors are less well understood. Here we review the emerging neurobiological mechanisms beyond pharmacological reinforcement which determine drug effects and use frequency. Important progress was made on the understanding of how the character of an environment and social stress determine drug self-administration. This is expanded by new evidence on how behavioral alternatives and opportunities for drug instrumentalization generate different patterns of drug choice. Emerging evidence suggests that the neurobiology of non-pharmacological factors strongly determines pharmacological and behavioral drug action and may, thus, give rise for an expanded system's approach of psychoactive drug use and addiction.

Believing in dopamine

Midbrain dopamine signals are widely thought to report reward prediction errors that drive learning in the basal ganglia. However, dopamine has also been implicated in various probabilistic computations, such as encoding uncertainty and controlling exploration. Here, we show how these different facets of dopamine signalling can be brought together under a common reinforcement learning framework. The key idea is that multiple sources of uncertainty impinge on reinforcement learning computations: uncertainty about the state of the environment, the parameters of the value function and the optimal action policy. Each of these sources plays a distinct role in the prefrontal cortex-basal ganglia circuit for reinforcement learning and is ultimately reflected in dopamine activity. The view that dopamine plays a central role in the encoding and updating of beliefs brings the classical prediction error theory into alignment with more recent theories of Bayesian reinforcement learning.

Dopamine signals related to appetitive and aversive events in paradigms that manipulate reward and avoidability

Using environmental cues to acquire good and avoid harmful things is critical for survival. Rewarding and aversive outcomes both drive behavior through reinforcement learning and sometimes occur together in the environment, but it remains unclear how these signals are encoded within the brain and if signals for positive and negative reinforcement are encoded similarly. Recent studies demonstrate that the dopaminergic system and interconnected brain regions process both positive and negative reinforcement necessary for approach and avoidance behaviors, respectively. Here, we review these data with a special focus on behavioral paradigms that manipulate both expected reward and the avoidability of aversive events to reveal neural correlates related to value, prediction error encoding, motivation, and salience.

Understanding the role of dopamine in conditioned and unconditioned fear

Pharmacological and molecular imaging studies in anxiety disorders have primarily focused on the serotonin system. In the meantime, dopamine has been known as the neurotransmitter of reward for 60 years, particularly for its action in the nervous terminals of the mesocorticolimbic system. Interest in the mediation by dopamine of the well-known brain aversion system has grown recently, particularly given recent evidence obtained on the role of D2 dopamine receptors in unconditioned fear. However, it has been established that excitation of the mesocorticolimbic pathway, originating from dopaminergic (DA) neurons from the ventral tegmental area (VTA), is relevant for the development of anxiety. Among the forebrain regions innervated by this pathway, the amygdala is an essential component of the neural circuitry of conditioned fear. Current findings indicate that the dopamine D2 receptor-signaling pathway connecting the VTA to the basolateral amygdala modulates fear and anxiety, whereas neural circuits in the midbrain tectum underlie the expression of innate fear. The A13 nucleus of the zona incerta is proposed as the origin of these DA neurons projecting to caudal structures of the brain aversion system. In this article we review data obtained in studies showing that DA receptor-mediated mechanisms on ascending or descending DA pathways play opposing roles in fear/anxiety processes. Dopamine appears to mediate conditioned fear by acting at rostral levels of the brain and regulate unconditioned fear at the midbrain level.

Ventral hippocampal overexpression of Cannabinoid Receptor Interacting Protein 1 (CNRIP1) produces a schizophrenia-like phenotype in the rat

Adolescent cannabis use has been implicated as a risk factor for schizophrenia; however, it is neither necessary nor sufficient. Previous studies examining this association have focused primarily on the role of the cannabinoid receptor 1 (CB1R) with relatively little known about a key regulatory protein, the cannabinoid receptor interacting protein 1 (CNRIP1). CNRIP1 is an intracellular protein that interacts with the C-terminal tail of CB1R and regulates its intrinsic activity. Previous studies have demonstrated aberrant CNRIP1 DNA promoter methylation in post-mortem in human patients with schizophrenia, and we have recently reported decreased methylation of the CNRIP1 DNA promoter in the ventral hippocampus (vHipp) of a rodent model of schizophrenia susceptibility. To examine whether augmented CNRIP1 expression could contribute to the pathology of schizophrenia, we performed viral-mediated overexpression of CNRIP1 in the vHipp of Sprague Dawley rats. We then tested these rats for behavioral correlates of schizophrenia symptoms, followed by electrophysiology to determine the effects on the dopamine system, known to underlie psychosis. Here, we report that overexpression of vHipp CNRIP1 induces impairments in latent inhibition and social interaction, similar to those observed in individuals with schizophrenia and in rodent models of the disease. Furthermore, rats overexpressing vHipp CNRIP1 displayed a significant increase in ventral tegmental area (VTA) dopamine neuron population activity, a putative correlate of psychosis. These data provide evidence that alterations in CNRIP1 may contribute to the pathophysiology of schizophrenia, as overexpression is sufficient to produce neurophysiological and behavioral correlates consistently observed in rodent models of the disease.

Individual differences in learning and biogenic amine levels influence the behavioural division between foraging honeybee scouts and recruits

Animals must effectively balance the time they spend exploring the environment for new resources and exploiting them. One way that social animals accomplish this balance is by allocating these two tasks to different individuals. In honeybees, foraging is divided between scouts, which tend to explore the landscape for novel resources, and recruits, which tend to exploit these resources. Exploring the variation in cognitive and physiological mechanisms of foraging behaviour will provide a deeper understanding of how the division of labour is regulated in social insect societies. Here, we uncover how honeybee foraging behaviour may be shaped by predispositions in performance of latent inhibition (LI), which is a form of non-associative learning by which individuals learn to ignore familiar information. We compared LI between scouts and recruits, hypothesizing that differences in learning would correlate with differences in foraging behaviour. Scouts seek out and encounter many new odours while locating novel resources, while recruits continuously forage from the same resource, even as its quality degrades. We found that scouts show stronger LI than recruits, possibly reflecting their need to discriminate forage quality. We also found that scouts have significantly elevated tyramine compared to recruits. Furthermore, after associative odour training, recruits have significantly diminished octopamine in their brains compared to scouts. These results suggest that individual variation in learning behaviour shapes the phenotypic behavioural differences between different types of honeybee foragers. These differences in turn have important consequences for how honeybee colonies interact with their environment. Uncovering the proximate mechanisms that influence individual variation in foraging behaviour is crucial for understanding the ecological context in which societies evolve.

Social defeat stress before pregnancy induces depressive-like behaviours and cognitive deficits in adult male offspring: correlation with neurobiological changes

BACKGROUND

Epidemiological surveys and studies with animal models have established a relationship between maternal stress and affective disorders in their offspring. However, whether maternal depression before pregnancy influences behaviour and related neurobiological mechanisms in the offspring has not been studied.

RESULTS

A social defeat stress (SDS) maternal rat model was established using the resident-intruder paradigm with female specific pathogen-free Wistar rats and evaluated with behavioural tests. SDS maternal rats showed a significant reduction in sucrose preference and locomotor and exploratory activities after 4 weeks of stress. In the third week of the experiment, a reduction in body weight gain was observed in SDS animals. Sucrose preference, open field, the elevated-plus maze, light-dark box, object recognition, the Morris water maze, and forced swimming tests were performed using the 2-month-old male offspring of the female SDS rats. Offspring subjected to pre-gestational SDS displayed enhanced anxiety-like behaviours, reduced exploratory behaviours, reduced sucrose preference, and atypical despair behaviours. With regard to cognition, the offspring showed significant impairments in the retention phase of the object recognition test, but no effect was observed in the acquisition phase. These animals also showed impairments in recognition memory, as the discrimination index in the Morris water maze test in this group was significantly lower for both 1 h and 24 h memory retention compared to controls. Corticosterone, adrenocorticotropic hormone, and monoamine neurotransmitters levels were determined using enzyme immunoassays or radioimmunoassays in plasma, hypothalamus, left hippocampus, and left prefrontal cortex samples from the offspring of the SDS rats. These markers of hypothalamic-pituitary-adrenal axis responsiveness and the monoaminergic system were significantly altered in pre-gestationally stressed offspring. Brain-derived neurotrophic factor (BDNF), cyclic adenosine monophosphate response element binding protein (CREB), phosphorylated CREB (pCREB), and serotonin transporter (SERT) protein levels were evaluated using western blotting with right hippocampus and right prefrontal cortex samples. Expression levels of BDNF, pCREB, and SERT in the offspring were also altered in the hippocampus and in the prefrontal cortex; however, there was no effect on CREB.

CONCLUSION

We conclude that SDS before pregnancy might induce depressive-like behaviours, cognitive deficits, and neurobiological alterations in the offspring.

Dopamine, Inference, and Uncertainty

The hypothesis that the phasic dopamine response reports a reward prediction error has become deeply entrenched. However, dopamine neurons exhibit several notable deviations from this hypothesis. A coherent explanation for these deviations can be obtained by analyzing the dopamine response in terms of Bayesian reinforcement learning. The key idea is that prediction errors are modulated by probabilistic beliefs about the relationship between cues and outcomes, updated through Bayesian inference. This account can explain dopamine responses to inferred value in sensory preconditioning, the effects of cue preexposure (latent inhibition), and adaptive coding of prediction errors when rewards vary across orders of magnitude. We further postulate that orbitofrontal cortex transforms the stimulus representation through recurrent dynamics, such that a simple error-driven learning rule operating on the transformed representation can implement the Bayesian reinforcement learning update.

Drugs as instruments: a new framework for non-addictive psychoactive drug use

Most people who are regular consumers of psychoactive drugs are not drug addicts, nor will they ever become addicts. In neurobiological theories, non-addictive drug consumption is acknowledged only as a "necessary" prerequisite for addiction, but not as a stable and widespread behavior in its own right. This target article proposes a new neurobiological framework theory for non-addictive psychoactive drug consumption, introducing the concept of "drug instrumentalization." Psychoactive drugs are consumed for their effects on mental states. Humans are able to learn that mental states can be changed on purpose by drugs, in order to facilitate other, non-drug-related behaviors. We discuss specific "instrumentalization goals" and outline neurobiological mechanisms of how major classes of psychoactive drugs change mental states and serve non-drug-related behaviors. We argue that drug instrumentalization behavior may provide a functional adaptation to modern environments based on a historical selection for learning mechanisms that allow the dynamic modification of consummatory behavior. It is assumed that in order to effectively instrumentalize psychoactive drugs, the establishment of and retrieval from a drug memory is required. Here, we propose a new classification of different drug memory subtypes and discuss how they interact during drug instrumentalization learning and retrieval. Understanding the everyday utility and the learning mechanisms of non-addictive psychotropic drug use may help to prevent abuse and the transition to drug addiction in the future.

Ventral tegmental area dopamine revisited: effects of acute and repeated stress

Aversive events rapidly and potently excite certain dopamine neurons in the ventral tegmental area (VTA), promoting phasic increases in the medial prefrontal cortex and nucleus accumbens. This is in apparent contradiction to a wealth of literature demonstrating that most VTA dopamine neurons are strongly activated by reward and reward-predictive cues while inhibited by aversive stimuli. How can these divergent processes both be mediated by VTA dopamine neurons? The answer may lie within the functional and anatomical heterogeneity of the VTA. We focus on VTA heterogeneity in anatomy, neurochemistry, electrophysiology, and afferent/efferent connectivity. Second, recent evidence for a critical role of VTA dopamine neurons in response to both acute and repeated stress will be discussed. Understanding which dopamine neurons are activated by stress, the neural mechanisms driving the activation, and where these neurons project will provide valuable insight into how stress can promote psychiatric disorders associated with the dopamine system, such as addiction and depression.

A role for phasic dopamine release within the nucleus accumbens in encoding aversion: a review of the neurochemical literature

Survival is dictated by an organism's fitness in approaching positive stimuli and avoiding harm. While a rich literature outlines a role for mesolimbic dopamine in reward and appetitive behaviors, dopamine's involvement in aversion and avoidance behaviors remains controversial. Debate surrounding dopamine's function in the processing of negative stimuli likely stems from conflicting results reported by single-unit electrophysiological studies. Indeed, a number of studies suggest that midbrain dopaminergic cells are inhibited by the presentation of negative or fearful stimuli, while others report no change, or even an increase, in their activity. These disparate results may be due to population heterogeneity. Recent evidence demonstrates that midbrain dopamine neurons are heterogeneous in their projection targets, responses to environmental stimuli, pharmacology, and influences on motivated behavior. Thus, in order to assemble an accurate account of dopamine function during aversive stimulus experience and related behavior, it is necessary to examine the functional output of dopamine neural activity at mesolimbic terminal regions. This Review presents a growing body of evidence that dopamine release in the nucleus accumbens encodes not only reward, but also aversion. For example, our laboratory recently utilized fast-scan cyclic voltammetry to show that real-time changes in accumbal dopamine release are detected when animals are presented with predictors of aversion and its avoidance. These data, along with other reports, support a considerably more nuanced view of dopamine neuron function, wherein accumbal dopamine release is differentially modulated by positive and negative affective stimuli to promote adaptive behaviors.

Endogenous nociceptin system involvement in stress responses and anxiety behavior

The mechanisms underpinning stress-related behavior and dysfunctional events leading to the expression of neuropsychiatric disorders remain incompletely understood. Novel candidates involved in the neuromodulation of stress, mediated both peripherally and centrally, provide opportunities for improved understanding of the neurobiological basis of stress disorders and may represent targets for novel therapeutic development. This chapter provides an overview of the mechanisms by which the opioid-related peptide, nociceptin, regulates the neuroendocrine stress response and stress-related behavior. In our research, we have employed nociceptin receptor antagonists to investigate endogenous nociceptin function in tonic control over stress-induced activity of the hypothalamo-pituitary-adrenal axis. Nociceptin demonstrates a wide range of functions, including modulation of psychological and inflammatory stress responses, modulation of neurotransmitter release, immune homeostasis, in addition to anxiety and cognitive behaviors. Greater appreciation of the complexity of limbic-hypothalamic neuronal networks, together with attention toward gender differences and the roles of steroid hormones, provides an opportunity for deeper understanding of the importance of the nociceptin system in the context of the neurobiology of stress and behavior.

Effects of dietary choline availability on latent inhibition of flavor aversion learning

OBJECTIVE

It has been previously reported that dietary choline supplementation might affect latent inhibition (LI) using a conditioned suppression procedure in rats. We have assessed the effect of dietary choline on LI of flavor aversion learning.

METHOD

Adult male Wistar rats received a choline supplemented (5 g/kg), deficient (0 g/kg), or standard (1.1 g/kg) diet for 3 months. After this supplementation period, all rats went through a conditioned taste aversion (CTA) procedure, half of them being pre-exposed to the conditioned stimulus before the conditioning.

RESULTS

The results indicated that choline deficiency prevents LI of conditioned flavor aversion to cider vinegar (3%) induced by a LiCl (0.15 M; 2% body weight) intraperitoneal injection, while choline supplementation enhances CTA leading to slower extinction.

DISCUSSION

The role of the brain systems modulating attentional processes is discussed.

Sensory regulation of dopaminergic cell activity: Phenomenology, circuitry and function

Dopaminergic neurons in a range of species are responsive to sensory stimuli. In the anesthetized preparation, responses to non-noxious and noxious sensory stimuli are usually tonic in nature, although long-duration changes in activity have been reported in the awake preparation as well. However, in the awake preparation, short-latency, phasic changes in activity are most common. These phasic responses can occur to unconditioned aversive and non-aversive stimuli, as well as to the stimuli which predict them. In both the anesthetized and awake preparations, not all dopaminergic neurons are responsive to sensory stimuli, however responsive neurons tend to respond to more than a single stimulus modality. Evidence suggests that short-latency sensory information is provided to dopaminergic neurons by relatively primitive subcortical structures - including the midbrain superior colliculus for vision and the mesopontine parabrachial nucleus for pain and possibly gustation. Although short-latency visual information is provided to dopaminergic neurons by the relatively primitive colliculus, dopaminergic neurons can discriminate between complex visual stimuli, an apparent paradox which can be resolved by the recently discovered route of information flow through to dopaminergic neurons from the cerebral cortex, via a relay in the colliculus. Given that projections from the cortex to the colliculus are extensive, such a relay potentially allows the activity of dopaminergic neurons to report the results of complex stimulus processing from widespread areas of the cortex. Furthermore, dopaminergic neurons could acquire their ability to reflect stimulus value by virtue of reward-related modification of sensory processing in the cortex. At the forebrain level, sensory-related changes in the tonic activity of dopaminergic neurons may regulate the impact of the cortex on forebrain structures such as the nucleus accumbens. In contrast, the short latency of the phasic responses to sensory stimuli in dopaminergic neurons, coupled with the activation of these neurons by non-rewarding stimuli, suggests that phasic responses of dopaminergic neurons may provide a signal to the forebrain which indicates that a salient event has occurred (and possibly an estimate of how salient that event is). A stimulus-related salience signal could be used by downstream systems to reinforce behavioral choices.

Stratified medicine for mental disorders

There is recognition that biomedical research into the causes of mental disorders and their treatment needs to adopt new approaches to research. Novel biomedical techniques have advanced our understanding of how the brain develops and is shaped by behaviour and environment. This has led to the advent of stratified medicine, which translates advances in basic research by targeting aetiological mechanisms underlying mental disorder. The resulting increase in diagnostic precision and targeted treatments may provide a window of opportunity to address the large public health burden, and individual suffering associated with mental disorders. While mental health and mental disorders have significant representation in the "health, demographic change and wellbeing" challenge identified in Horizon 2020, the framework programme for research and innovation of the European Commission (2014-2020), and in national funding agencies, clear advice on a potential strategy for mental health research investment is needed. The development of such a strategy is supported by the EC-funded "Roadmap for Mental Health Research" (ROAMER) which will provide recommendations for a European mental health research strategy integrating the areas of biomedicine, psychology, public health well being, research integration and structuring, and stakeholder participation. Leading experts on biomedical research on mental disorders have provided an assessment of the state of the art in core psychopathological domains, including arousal and stress regulation, affect, cognition social processes, comorbidity and pharmacotherapy. They have identified major advances and promising methods and pointed out gaps to be addressed in order to achieve the promise of a stratified medicine for mental disorders.

On the role of subsecond dopamine release in conditioned avoidance

Using shock avoidance procedures to study conditioned behavioral responses has a rich history within the field of experimental psychology. Such experiments led to the formulation of the general concept of negative reinforcement and specific theories attempting to explain escape and avoidance behavior, or why animals choose to either terminate or prevent the presentation of an aversive event. For example, the two-factor theory of avoidance holds that cues preceding an aversive event begin to evoke conditioned fear responses, and these conditioned fear responses reinforce the instrumental avoidance response. Current neuroscientific advances are providing new perspectives into this historical literature. Due to its well-established role in reinforcement processes and behavioral control, the mesolimbic dopamine system presented itself as a logical starting point in the search for neural correlates of avoidance and escape behavior. We recently demonstrated that phasic dopamine release events are inhibited by stimuli associated with aversive events but increased by stimuli preceding the successful avoidance of the aversive event. The latter observation is inconsistent with the second component of the two-factor theory of avoidance and; therefore, led us propose a new theoretical explanation of conditioned avoidance: (1) fear is initially conditioned to the warning signal and dopamine computes this fear association as a decrease in release, (2) the warning signal, now capable of producing a negative emotional state, suppresses dopamine release and behavior, (3) over repeated trials the warning signal becomes associated with safety rather than fear; dopaminergic neurons already compute safety as an increase in release and begin to encode the warning signal as the earliest predictor of safety (4) the warning signal now promotes conditioned avoidance via dopaminergic modulation of the brain's incentive-motivational circuitry.

D(1)-like receptors in the nucleus accumbens shell regulate the expression of contextual fear conditioning and activity of the anterior cingulate cortex in rats

Although dopamine-related circuits are best known for their roles in appetitive motivation, consistent data have implicated this catecholamine in some forms of response to stressful situations. In fact, projection areas of the ventral tegmental area, such as the amygdala and hippocampus, are well established to be involved in the acquisition and expression of fear conditioning, while less is known about the role of the medial prefrontal cortex (mPFC) and nucleus accumbens (NAc) in these processes. In the present study, we initially investigated the involvement of the mPFC and NAc in the expression of conditioned fear, assessing freezing behaviour and Fos protein expression in the brains of rats exposed to a context, light or tone previously paired with footshocks. Contextual and cued stimuli were able to increase the time of the freezing response while only the contextual fear promoted a significant increase in Fos protein expression in the mPFC and caudal NAc. We then examined the effects of specific dopaminergic agonists and antagonists injected bilaterally into the posterior medioventral shell subregion of the NAc (NAcSh) on the expression of contextual fear. SKF38393, quinpirole and sulpiride induced no behavioural changes, but the D1-like receptor antagonist SCH23390 increased the freezing response of the rats and selectively reduced Fos protein expression in the anterior cingulate cortex and rostral NAcSh. These findings confirm the involvement of the NAcSh in the expression of contextual fear memories and indicate the selective role of NAcSh D1-like receptors and anterior cingulate cortex in this process.

Aversive stimuli differentially modulate real-time dopamine transmission dynamics within the nucleus accumbens core and shell

Although fear directs adaptive behavioral responses, how aversive cues recruit motivational neural circuitry is poorly understood. Specifically, while it is known that dopamine (DA) transmission within the nucleus accumbens (NAc) is imperative for mediating appetitive motivated behaviors, its role in aversive behavior is controversial. It has been proposed that divergent phasic DA transmission following aversive events may correspond to segregated mesolimbic dopamine pathways; however, this prediction has never been tested. Here, we used fast-scan cyclic voltammetry to examine real-time DA transmission within NAc core and shell projection systems in response to a fear-evoking cue. In male Sprague Dawley rats, we first demonstrate that a fear cue results in decreased DA transmission within the NAc core, but increased transmission within the NAc shell. We examined whether these changes in DA transmission could be attributed to modulation of phasic transmission evoked by cue presentation. We found that cue presentation decreased the probability of phasic DA release in the core, while the same cue enhanced the amplitude of release events in the NAc shell. We further characterized the relationship between freezing and both changes in DA as well as local pH. Although we found that both analytes were significantly correlated with freezing in the NAc across the session, changes in DA were not strictly associated with freezing while basic pH shifts in the core more consistently followed behavioral expression. Together, these results provide the first real-time neurochemical evidence that aversive cues differentially modulate distinct DA projection systems.

The role of dopamine in the context of aversive stimuli with particular reference to acoustically signaled avoidance learning

Learning from punishment is a powerful means for behavioral adaptation with high relevance for various mechanisms of self-protection. Several studies have explored the contribution of released dopamine (DA) or responses of DA neurons on reward seeking using rewards such as food, water, and sex. Phasic DA signals evoked by rewards or conditioned reward predictors are well documented, as are modulations of these signals by such parameters as reward magnitude, probability, and deviation of actually occurring from expected rewards. Less attention has been paid to DA neuron firing and DA release in response to aversive stimuli, and the prediction and avoidance of punishment. In this review, we first focus on DA changes in response to aversive stimuli as measured by microdialysis and voltammetry followed by the change in electrophysiological signatures by aversive stimuli and fearful events. We subsequently focus on the role of DA and effect of DA manipulations on signaled avoidance learning, which consists of learning the significance of a warning cue through Pavlovian associations and the execution of an instrumental avoidance response. We present a coherent framework utilizing the data on microdialysis, voltammetry, electrophysiological recording, electrical brain stimulation, and behavioral analysis. We end by outlining current gaps in the literature and proposing future directions aimed at incorporating technical and conceptual progress to understand the involvement of reward circuit on punishment based decisions.

Modularity of mind and the role of incentive motivation in representing novelty

Animal and human brains contain a myriad of mental representations that have to be successfully tracked within fractions of a second in a large number of situations. This retrieval process is hard to explain without postulating the massive modularity of cognition. Assuming that the mind is massively modular, it is then necessary to understand how cognitive modules can efficiently represent dynamic environments-in which some modules may have to deal with change-induced novelty and uncertainty. Novelty of a stimulus is a problem for a module when unknown, significant stimuli do not satisfy the module's processing criteria-or domain specificity-and cannot therefore be included in its database. It is suggested that the brain mechanisms of incentive motivation, recruited when faced with novelty and uncertainty, induce transient variations in the domain specificity of cognitive modules in order to allow them to process information they were not prepared to learn. It is hypothesised that the behavioural transitions leading from exploratory activity to habit formation are correlated with (and possibly caused by) the organism's ability to counter novelty-induced uncertainty.

Gender differences in the effects of presynaptic and postsynaptic dopamine agonists on latent inhibition in rats

The present study investigated gender differences in the effects of presynaptic and postsynaptic DA agonists on latent inhibition in the passive avoidance paradigm. During the preexposure phase, 32 male and 32 female Wistar rats were exposed to a passive avoidance box (or a different context) and received drug injections in three trials: the control group received an injection of 10% ascorbic acid in a different context. The experimental groups received injections of 10% ascorbic acid (latent inhibition [LI] group), 1mg/kg of the postsynaptic DA D(1)/D(2) agonist apomorphine (APO group), and 1.5mg/kg of the presynaptic DA agonist methamphetamine (METH group) in a passive avoidance box. All experimental groups were placed in the light compartment of the passive avoidance box and were allowed to enter into the dark compartment to receive a footshock (1mA, 2s) in five trials over 5 days. The latency to enter into the dark compartment was recorded in these five trials. The latent inhibition occurred in the female LI group but not in the male LI group. Regardless of gender, the APO group exhibited an increase in latent inhibition. Male rats in the METH group exhibited a decrease in latent inhibition, but female rats in the METH group exhibited an increase in latent inhibition, indicating that the METH group exhibited sexual dimorphism. The gender factor interacted only with the METH group and not the LI or APO group. The present paper discusses whether gender, the postsynaptic DA D(1)/D(2) agonist APO, and presynaptic DA agonist METH may be related to schizophrenia.

Catecholaminergic depletion in nucleus accumbens enhances trace conditioning

PURPOSE

To examine the effect of dopamine depletion in nucleus accumbens on trace conditioning; to distinguish the role of core and shell sub-regions, as far as possible.

MATERIAL/METHODS

6-hydroxydopamine was used to lesion dopamine terminals within the core and shell accumbens. Experiment 1 assessed conditioning to a tone conditioned stimulus that had previously been paired with footshock (unconditioned stimulus) at a 30s trace interval. Experiment 2 subsequently assessed contiguous conditioning (at 0s trace) using a light conditioned stimulus directly followed by the unconditioned stimulus.

RESULTS

Both sham and shell-lesioned animals showed the normal trace effect of reduced conditioning to the trace conditioned stimulus but 6-hydroxydopamine injections targeted on the core subregion of the nucleus accumbens abolished this effect and enhanced conditioning to the trace conditioned stimulus. However, the depletion produced by this lesion placement extended to the shell. In Experiment 2 (at 0s trace), there was no effect of either lesion placement as all animals showed comparable levels of conditioning to the light conditioned stimulus. Neurochemical analysis across core, shell and comparison regions showed some effects on noradrenalin as well as dopamine.

CONCLUSIONS

The pattern of changes in noradrenalin did not systematically relate to the observed behavioural changes after core injections. The pattern of changes in dopamine suggested that depletion in core mediated the increased conditioning to the trace conditioned stimulus seen in the present study. However, the comparison depletion restricted to the shell subregion was less substantial, and a role for secondarily affected brain regions cannot be excluded.

Emerging, reemerging, and forgotten brain areas of the reward circuit: Notes from the 2010 Motivational Neural Networks conference

On April 24-27, 2010, the Motivational Neuronal Networks meeting took place in Wrightsville Beach, North Carolina. The conference was devoted to "Emerging, re-emerging, and forgotten brain areas" of the reward circuit. A central feature of the conference was four scholarly discussions of cutting-edge topics related to the conference's theme. These discussions form the basis of the present review, which summarizes areas of consensus and controversy, and serves as a roadmap for the next several years of research.

Latent inhibition-related dopaminergic responses in the nucleus accumbens are disrupted following neonatal transient inactivation of the ventral subiculum

Schizophrenia would result from a defective connectivity between several integrative regions as a consequence of neurodevelopmental failure. Various anomalies reminiscent of early brain development disturbances have been observed in patients' left ventral subiculum of the hippocampus (SUB). Numerous data support the hypothesis of a functional dopaminergic dysregulation in schizophrenia. The common target structure for the action of antipsychotics appears to be a subregion of the ventral striatum, the dorsomedial shell part of the nucleus accumbens. Latent inhibition, a cognitive marker of interest for schizophrenia, has been found to be disrupted in acute patients. The present study set out to investigate the consequences of a neonatal functional inactivation of the left SUB by tetrodotoxin (TTX) in 8-day-old rats for the latent inhibition-related dopaminergic responses, as monitored by in vivo voltammetry in freely moving adult animals (11 weeks) in the left core and dorsomedial shell parts of the nucleus accumbens in an olfactory aversion procedure. Results obtained during the retention session of a three-stage latent inhibition protocol showed that the postnatal unilateral functional blockade of the SUB was followed in pre-exposed TTX-conditioned adult rats by a disruption of the behavioral expression of latent inhibition and induced a total and a partial reversal of the latent inhibition-related dopaminergic responses in the dorsomedial shell and core parts of the nucleus accumbens, respectively. The present data suggest that neonatal inactivation of the SUB has more marked consequences for the dopaminergic responses recorded in the dorsomedial shell part, than in the core part of the nucleus accumbens. These findings may provide new insight into the pathophysiology of schizophrenia.

Dopaminergic modulation of appetitive and aversive predictive learning

Temporal contiguity between two stimuli is insufficient for the establishment of a predictive relation between those stimuli. Rather, learning about predictive relations is influenced by a prediction error mechanism: the discrepancy between actual and expected outcomes. Although the neural substrates of contiguous stimuli presentation have been the focus of research for decades, relatively little empirical evidence exists with regard to the neural mechanisms of prediction error. Recent work has implicated the neurotransmitter dopamine in regulation of predictive learning. If dopamine modulates prediction error then it should do so despite the nature (appetitive or aversive) of the biological stimuli that serve to drive learning. The exact role of dopamine in appetitive and aversive predictive learning, however, remains the focus of continuous debate. This review focuses on the behavioural, neuropharmacological and electrophysiological evidence implicating dopamine in prediction error in appetitive and aversive predictive learning. In addition, recent work in the area of fear conditioning implicating other neurochemical substrates, namely opioids, in the process of prediction error is discussed. Finally, some predictions are made with regard to the neurochemical circuitry involved in modulating learning and behaviour based on prediction error.

Predicting value of pain and analgesia: nucleus accumbens response to noxious stimuli changes in the presence of chronic pain

VIDEO ABSTRACT

We compared brain activations in response to acute noxious thermal stimuli in controls and chronic back pain (CBP) patients. Pain perception and related cortical activation patterns were similar in the two groups. However, nucleus accumbens (NAc) activity differentiated the groups at a very high accuracy, exhibiting phasic and tonic responses with distinct properties. Positive phasic NAc activations at stimulus onset and offset tracked stimulus salience and, in normal subjects, predicted reward (pain relief) magnitude at stimulus offset. In CBP, NAc activity correlated with different cortical circuitry from that of normals and phasic activity at stimulus offset was negative in polarity, suggesting that the acute pain relieves the ongoing back pain. The relieving effect was confirmed in a separate psychophysical study in CBP. Therefore, in contrast to somatosensory pathways, which reflect sensory properties of acute noxious stimuli, NAc activity in humans encodes its predicted value and anticipates its analgesic potential on chronic pain.

Blockade of dopamine activity in the nucleus accumbens impairs learning extinction of conditioned fear

Three experiments used rats to investigate the role of dopamine activity in learning to inhibit conditioned fear responses (freezing) in extinction. In Experiment 1, rats systemically injected with the D2 dopamine antagonist, haloperidol, froze more across multiple extinction sessions and on a drug-free retention test than control rats. In Experiment 2, rats extinguished under an intracerebroventricular (ICV) infusion of haloperidol suppressed fear responses across extinction but froze more on a subsequent drug-free retention test than control rats. In Experiment 3, rats extinguished under an infusion of haloperidol in the nucleus accumbens were impaired in suppressing fear responses across extinction and froze more on subsequent drug-free retention test than control rats. These results show that learning to inhibit fear responses in extinction requires dopamine activity in the nucleus accumbens. They were interpreted to mean that dopaminergic activity in the nucleus accumbens regulates the prediction error required for learning to inhibit fear responses in extinction.

On giving a more active and selective role to consciousness

An active role for conscious processes in the production of behaviour is proposed, involving top level controls in a hierarchy of behavioural control. It is suggested that by inhibiting or sensitizing lower levels in the hierarchy conscious processes can play a role in the organization of ongoing behaviour. Conscious control can be more or less evident, according to prevailing circumstances.

The control of consciousness via a neuropsychological feedback loop

Gray's neuropsychological model of consciousness uses a hierarchical feedback loop framework that has been extensively discussed by many others in psychology. This commentary therefore urges Gray to integrate with, or at least acknowledge previous models. It also points out flaws in his feedback model and suggests directions for further theoretical work.

Don't leave the “un” off “consciousness”

Gray extrapolates from circuit models of psychopathology to propose neural substrates for the contents of consciousness. I raise three concerns: (1) knowledge of synaptic arrangements may be inadequate to fully support his model; (2) latent inhibition deficits in schizophrenia, a focus of this and related models, are complex and deserve replication; and (3) this conjecture omits discussion of the neuropsychological basis for the contents of the unconscious.

Consciousness is for other people

Gray has expanded his account of schizophrenia to explain consciousness as well. His theory explains neither phenomenon adequately because he treats individual minds (and brains) in isolation. The primary function of consciousness is to permit high level interactions with other conscious beings. The key symptoms of schizophrenia reflect a failure of this mechanism.

Comparators, functions, and experiences

The comparator model is insufficient for three reasons. First, consciousness is involved in the process of comparison as well as in the output. Second, we still do not have enough neurophysiological information to match the events of consciousness, although such knowledge is growing. Third, the anatomical localisation proposed can be damaged bilaterally but consciousness will persist.

Possible roles for a predictor plus comparator mechanism in human episodic recognition memory and imitative learning

This commentary is divided into two parts. The first considers a possible role for Gray's predictor plus comparator mechanism in human episodic recognition memory. It draws on the computational specifications of recognition outlined in Humphreys et al. (1994) to demonstrate how the logically necessary components of recognition tasks might be mapped onto the mechanism. The second part demonstrates how the mechanism outlined by Gray might be implicated in a form of imitative learning suitable for the acquisition of complex tasks.