Updating dopamine reward signals

Dopaminergic action prediction errors serve as a value-free teaching signal

Choice behaviour of animals is characterized by two main tendencies: taking actions that led to rewards and repeating past actions1,2. Theory suggests that these strategies may be reinforced by different types of dopaminergic teaching signals: reward prediction error to reinforce value-based associations and movement-based action prediction errors to reinforce value-free repetitive associations3-6. Here we use an auditory discrimination task in mice to show that movement-related dopamine activity in the tail of the striatum encodes the hypothesized action prediction error signal. Causal manipulations reveal that this prediction error serves as a value-free teaching signal that supports learning by reinforcing repeated associations. Computational modelling and experiments demonstrate that action prediction errors alone cannot support reward-guided learning, but when paired with the reward prediction error circuitry they serve to consolidate stable sound-action associations in a value-free manner. Together we show that there are two types of dopaminergic prediction errors that work in tandem to support learning, each reinforcing different types of association in different striatal areas.

The devilish details affecting TDRL models in dopamine research

Over recent decades, temporal difference reinforcement learning (TDRL) models have successfully explained much dopamine (DA) activity. This success has invited heightened scrutiny of late, with many studies challenging the validity of TDRL models of DA function. Yet, when evaluating the validity of these models, the devil is truly in the details. TDRL is a broad class of algorithms sharing core ideas but differing greatly in implementation and predictions. Thus, it is important to identify the defining aspects of the TDRL framework being tested and to use state spaces and model architectures that capture the known complexity of the behavioral representations and neural systems involved. Here, we discuss several examples that illustrate the importance of these considerations.

Dissociable effects of psilocybin and escitalopram for depression on processing of musical surprises

Psilocybin therapy (PT) is emerging as an effective intervention for Major Depressive Disorder (MDD), offering comparable efficacy to conventional treatments like selective serotonin reuptake inhibitors (SSRIs). Music, an emotionally evocative stimulus, provides a valuable tool to explore changes in hedonic and predictive processing mechanisms via expectancy violations, or 'surprises'. This study sought to compare behavioural and functional magnetic resonance imaging (fMRI) responses to musical surprises in MDD patients treated with either PT or the SSRI, escitalopram. In this secondary analysis of a trial, 41 MDD patients (with usable fMRI data) were randomly assigned to either PT (n = 22) or escitalopram (n = 19) treatment groups. Participants listened to music during fMRI and tracked their emotional experience, both before and after a 6-week intervention. Surprise-related valence and arousal indices were calculated. Musical surprises were entered as regressors for whole-brain and region of interest fMRI analyses. PT caused a greater decrease in anhedonia scores compared with escitalopram. While escitalopram led to reductions in surprise-related affective responses, PT showed no significant change. Escitalopram was associated with increased activation in memory and emotional processing areas during musical surprises (versus control events) when compared with PT. Following PT, there was greater activation in the ventromedial prefrontal cortex and sensory regions, and reduced activation in the angular gyrus. PT may allow for the subjective response to musical surprises to be maintained through a lasting reduction in the salience of prediction errors, or, alternatively, by increasing hedonic priors. Contrastingly, escitalopram may diminish hedonic priors, highlighting fundamental differences in treatment mechanisms.

Primate thalamic nuclei select abstract rules and shape prefrontal dynamics

Flexible behavior depends on abstract rules to generalize beyond specific instances and outcome monitoring to adjust actions. Cortical circuits are posited to read out rules from high-dimensional representations of task-relevant variables in prefrontal cortex (PFC). We instead hypothesized that converging inputs from PFC, directly or via basal ganglia (BGs), enable the thalamus to select rules. We measured activity across PFC and connected thalamic nuclei of monkeys applying rules. Abstract rule information first appeared in ventroanterior thalamus (VA)-the main thalamic hub between BG and PFC. Mediodorsal thalamus (MD) also represented rule information before PFC, persisting to help maintain activation of relevant PFC cell ensembles. MD, a major recipient of midbrain dopamine input, was the first to represent information about behavioral outcomes. A PFC-BG-thalamus model reproduced key findings, and thalamic-lesion modeling disrupted PFC rule representations. This suggests that the thalamus selects high-level cognitive information from PFC and monitors behavioral outcomes of these selections.

Eligibility traces as a synaptic substrate for learning

Animals can learn to associate a behavior or a stimulus with a delayed reward, this is essential for survival. A mechanism proposed for bridging this gap are synaptic eligibility traces, which are slowly decaying tags, which can lead to synaptic plasticity if followed by rewards. Recently, experiments have demonstrated the existence of synaptic eligibility traces in diverse neural systems, depending on either neuromodulators or plateau potentials. Evidence for both eligibility trace-dependent potentiation and depression of synaptic efficacies has emerged. We discuss the commonalities and differences of these different results. We show why the existence of both potentiation and depression is important because these opposing forces can lead to a synaptic stopping rule. Without a stopping rule, synapses would saturate at their upper bound thus leading to a loss of selectivity and representational power. We discuss the possible underlying mechanisms of the eligibility traces as well as their functional and theoretical significance.

Establishing In-vivo brain microdialysis for comparing concentrations of a variety of cortical neurotransmitters in the awake rhesus macaque between different cognitive states

BACKGROUND

Neuronal activity is modulated by behavior and cognitive processes. The combination of several neurotransmitter systems, acting directly or indirectly on specific populations of neurons, underlie such modulations. Most studies with non-human primates (NHPs) fail to capture this complexity, partly due to the lack of adequate methods for reliably and simultaneously measuring a broad spectrum of neurotransmitters while the animal engages in behavioral tasks.

NEW METHOD

To address this gap, we introduce a novel implementation of brain microdialysis (MD), employing semi-chronically implanted guides and probes in awake, behaving NHPs facilitated by removable insets within a standard recording chamber over extrastriate visual cortex (here, the visual middle temporal area (MT)). This approach allows flexible access to diverse brain regions, including areas deep within the sulcus.

RESULTS

Reliable concentration measurements of GABA, glutamate, norepinephrine, epinephrine, dopamine, serotonin, and choline were achieved from small sample volumes (<20 µl) using ultra-performance liquid chromatography with electrospray ionization-mass spectrometry (UPLC-ESI-MS). Comparing two behavioral states - 'active' and 'inactive', we observe subtle concentration variations between the two behavioral states and a greater variability of concentrations in the active state. Additionally, we find positively and negatively correlated concentration changes for neurotransmitter pairs between the behavioral states.

CONCLUSIONS

Therefore, this MD setup allows insights into the neurochemical dynamics in awake primates, facilitating comprehensive investigations into the roles and the complex interplay of neurotransmitters in cognitive and behavioral functions.

Cognitive effects of early life exposure to PCBs in rats: Sex-specific behavioral, hormonal and neuromolecular mechanisms involving the brain dopamine system

Endocrine-disrupting chemicals (EDCs) are environmental toxicants that disrupt hormonal and neurodevelopmental processes. Among these chemicals, polychlorinated biphenyls (PCBs) are particularly concerning due to their resistance to biodegradation and tendency to bioaccumulate. PCBs affect neurodevelopmental function and disrupt the brain's dopamine (DA) system, which is crucial for attentional, affective, and reward processing. These disruptions may contribute to the rising prevalence of DA-mediated neuropsychiatric disorders such as ADHD, depression, and substance use disorders. Notably, these behaviors are sexually dimorphic in part due to differences in sex hormones and their receptors, which are targets of estrogenic PCBs. Therefore, this study determined effects of early life PCB exposure on behaviors and neurochemistry related to potential disruption of dopaminergic signaling. Male and female Sprague Dawley rats were exposed to the PCB mixture Aroclor 1221 (A1221) or vehicle perinatally and then underwent a series of behavioral tests in adulthood, including the sucrose preference test to measure anhedonia, conditioned orienting to assess incentive-motivational phenotype, and attentional set-shifting to evaluate cognitive flexibility and response latency. Following these tests, rats were euthanized, and serum estradiol (E2), DA cells in the midbrain ventral tegmental area (VTA) and substantia nigra (SN), and gene expression from those combined midbrain nuclei were measured. Female rats exposed perinatally to A1221 exhibited decreased sucrose preference, and both male and female A1221 rats had reduced response latency in the attentional set-shifting task compared to vehicle counterparts. Conditioned orienting and serum estradiol (E2)were not affected in either sex; however, A1221-exposed rats of both sexes displayed higher TH+ cell numbers in the VTA and increased expression of dopamine receptor 1 (Drd1) in the combined midbrain nuclei. Additionally, E2 uniquely predicted behavioral outcomes and VTA DAergic cell numbers in A1221-exposed female rats, whereas DA signaling genes were predictive of behavioral outcomes in males. These data highlight sex-specific effects of A1221 on neuromolecular and behavioral phenotypes.

Substantial Heritability Underlies Fairness Norm Adaptation Capability and its Neural Basis

The present research uncovers the shared genetic underpinnings of fairness norm adaptation capability, its neural correlates, and long-term mental health outcomes. One hundred and eighty-six twins are recruited and played as responders in the Ultimatum Game (UG) while undergoing fMRI scanning in their early adulthood (Study-1) and are measured on depressive symptoms eight years later (Study-2). With computational modeling, the process of norm adaptation is differentiated from that of fairness valuation in UG. The two processes both have moderate levels of heritability. The anterior insula has a significant phenotypic correlation, whereas the Supplementary Motor Area/Medial Frontal Gyrus (SMA/mSFG) shows both a significant phenotypic correlation and a shared genetic influence with the learning rate, an index for norm adaptation. The dopaminergic DRD2 polymorphisms correlate with both the learning rate and the SMA/mSFG encoding of prediction error, constituting of their common genetic basis. Regional gene expression analysis reveals the high expression of dopamine-related genes in the SMA/mSFG. Moreover, the learning rate can predict depressive symptom severity eight years later, with the DRD2 polymorphisms constituting their shared genetic basis. This suggests that heritability is a non-negligible driving force behind norm adaptation, which facilitates the learning of social norms in changing environments and preserves long-term mental health.

Binding of HCN channels to GABAB receptors in dopamine neurons of the VTA limits synaptic inhibition and prevents the development of anxiety

During GABAergic synaptic transmission, G protein-coupled GABAB receptors (GBRs) activate K+ channels that prolong the duration of inhibitory postsynaptic potentials (IPSPs). We now show that KCTD16, an auxiliary GBR subunit, anchors hyperpolarization-activated cyclic nucleotide-gated (HCN) channels containing HCN2/HCN3 subunits to GBRs. In dopamine neurons of the VTA (DAVTA neurons), this interaction facilitates activation of HCN channels via hyperpolarization during IPSPs, counteracting the GBR-mediated late phase of these IPSPs. Consequently, disruption of the GBR/HCN complex in KCTD16-/- mice leads to prolonged optogenetic inhibition of DAVTA neuron firing. KCTD16-/- mice exhibit increased anxiety-like behavior in response to stress - a behavior replicated by CRISPR/Cas9-mediated KCTD16 ablation in DAVTA neurons or by intra-VTA infusion of an HCN antagonist in wild-type mice. Our findings support that the retention of HCN channels at GABAergic synapses by GBRs in DAVTA neurons provides a negative feedback mechanism that restricts IPSP duration and mitigates the development of anxiety.

Remission from addiction: erasing the wrong circuits or making new ones?

Chronic relapse is a hallmark of substance-use disorders (SUDs), but many people with SUDs do recover and eventually enter remission. Many preclinical studies in this field aim to identify interventions that can precipitate recovery by reversing or erasing the neuronal circuit changes caused by chronic drug use. A better understanding of remission from SUDs can also come from preclinical studies that model factors known to influence recovery in humans, such as the negative consequences of drug use and positive environmental influences. In this Perspective we discuss human neuroimaging studies that have provided information about recovery from SUDs and highlight mechanisms identified in preclinical studies - such as the reconfiguration of neuronal circuits - that could contribute to remission. We also analyse how studies of memory and forgetting can provide insights into the mechanisms of remission. Overall, we propose that remission can be driven by the introduction of new neuronal changes (which outcompete those induced by drugs) as well as by the erasure of drug-induced changes.

Self-prioritization in working memory gating

Working memory (WM) involves a dynamic interplay between temporary maintenance and updating of goal-relevant information. The balance between maintenance and updating is regulated by an input-gating mechanism that determines which information should enter WM (gate opening) and which should be kept out (gate closing). We investigated whether updating and gate opening/closing are differentially sensitive to the kind of information to be encoded and maintained in WM. Specifically, since the social salience of a stimulus is known to affect cognitive performance, we investigated if self-relevant information differentially impacts maintenance, updating, or gate opening/closing. Participants first learned to associate two neutral shapes with two social labels (i.e., "you" vs. "stranger"), respectively. Subsequently they performed the reference-back paradigm, a well-established WM task that disentangles WM updating, gate opening, and gate closing. Crucially, the shapes previously associated with the self or a stranger served as target stimuli in the reference-back task. We replicated the typical finding of a repetition benefit when consecutive trials require opening the gate to WM. In Study 1 (N = 45) this advantage disappeared when self-associated stimuli were recently gated into WM and immediately needed to be replaced by stranger-associated stimuli. However, this was not replicated in a larger sample (Study 2; N = 90), where a repetition benefit always occurred on consecutive gate-opening trials. Overall, our results do not provide evidence that the self-relevance of stimuli modulates component processes of WM. We discuss possible reasons for this null finding, including the importance of continuous reinstatement and task-relevance of the shape-label associations.

Implicit and explicit learning strategies and fatigue: an evaluation of throwing task performance

Introduction

This study aimed to determine the effects of implicit (errorless) and explicit (errorful) training strategies on a throwing task under physiological and mental fatigue conditions.

Methods

Thirty-two participants, equally divided between the explicit and implicit learning groups, participated in a throwing task. The explicit learning group began at a significant distance from the target and gradually moved closer. In contrast, the implicit learning group started close to the target and progressively increased their distance. The initial session referred to as the acquisition phase, comprised 150 throws from five different distances. Subsequent sessions included a retention test and two transfer tests conducted under conditions of both physiological and mental fatigue. Mental fatigue was induced using a 30-minute color-word Stroop task, while physical fatigue was elicited by requiring subjects to maintain 50% of their maximum voluntary isometric contraction (MVC) in elbow extension for a 2-minute duration.

Results

The results revealed that the implicit learning group exhibited improved performance under fatigue conditions and outperformed the explicit learning group significantly, regardless of the type of fatigue.

Conclusion

This results suggests that implicit learning may improve motor performance even under fatigue conditions.

Addiction and stress: Exploring the reward pathways in brain affected by different drugs

This chapter delves into the complex interplay among addiction, stress, and the reward pathways in the brain, emphasizing the ways in which various drugs affect these systems and exacerbate SUD. Drugs have physiological effects that can be both pleasurable and unpleasant. These effects change behavior through both positive and negative reinforcement. A person's genetic predisposition to addiction is mostly determined by factors such as biological sex, age of first usage, and dopamine receptor density. Drug use behaviors are also greatly influenced by environmental stressors, media exposure, and substance accessibility; nevertheless, protective variables including social support, participation in healthy activities, and preventative programs serve to reduce the dangers associated with drug use. The reinforcement of addictive behaviors is mostly dependent on the brain's reward circuits, which include the nucleus accumbens, ventral tegmental region, and prefrontal cortex, in addition to neurotransmitters such as dopamine, serotonin, and endorphins. Stress makes addiction worse by intensifying cravings and raising the possibility of relapsing. Examined are the impacts of several drug types, such as opioids, stimulants, depressants, and hallucinogens, emphasizing the long-term consequences on brain function and susceptibility to addiction. In order to create individualized interventions that target the environmental and neurological components of addiction and eventually improve treatment results, a thorough understanding of these elements is important.

Resting-state voxel-wise dynamic effective connectivity predicts risky decision-making in patients with bipolar disorder type I

Patients with Bipolar Disorder type I (BD-I) exhibit maladaptive risky decision-making, which is related to impulsivity, suicide attempts, and aggressive behavior. Currently, there is a lack of effective predictive methods for early intervention in risky behaviors for patients with BD-I. This study aimed to predict risky behavior in patients with BD-I using resting-state functional magnetic resonance imaging (rs-fMRI). We included 48 patients with BD-I and 124 healthy controls (HC) and constructed voxel-wise functional connectivity (FC), dynamic FC (dFC), effective connectivity (EC), and dynamic EC (dEC) for each subject. The Balloon Analogue Risk Task (BART) was employed to measure the risky decision-making of all participants. We applied connectome-based predictive modeling (CPM) with five regression algorithms to predict risky behaviors as well as Barratt Impulsivity Scale (BIS) scores. Results showed that the BD-I had significantly lower risky adjusted pump scores compared to HC. The dEC-based linear regression-CPM model exhibited significant predictive ability for the adjusted pump scores in BD-I, while no significant predictive power was observed in HC. Furthermore, this model successfully predicted non-planning impulsiveness, motor impulsiveness, and BIS total score, but failed for attentional impulsiveness in BD-I. These findings provide a foundation for future work in predicting risky behaviors of psychiatric patients by using voxel-wise dEC underlying resting state.

The effect of reward and voluntary choice on the motor learning of serial reaction time task

Objective

Reward and voluntary choice facilitate motor skill learning through motivation. However, it remains unclear how their combination influences motor skill learning. The purpose of the present study is to investigate the effects of reward and voluntary choice on motor skill learning in a serial reaction time task (SRTT).

Methods

Participants completed six parts of SRTT, including pre-test, training phase, immediate post-test, a random session, delayed post-test, and retention test on the following day. During the training phase, participants were divided into four groups (reward_choice, reward_no-choice, no-reward_choice, no-reward_no-choice). In the reward condition, participants received reward for correct and faster (than a baseline) responses while those in the no-reward groups did not. For the choice manipulation, participants in the voluntary choice groups chose the color of the target, whereas in the forced choice groups, the same color was assigned by the computer.

Results

The results showed that the four groups did not exhibit any significant differences in reaction time and error rate in the pre-test phase. Importantly, both reward and voluntary choice significantly enhanced sequence-specific learning effects, while no interaction was found. No significant effects of reward and voluntary choice were observed in the retention test.

Conclusions

These findings suggest that reward and voluntary choice enhance motor skill performance and training independently, potentially at the action-selection level, which implies different mechanisms underlying the influences of reward and voluntary choice.

The pleasantness of foods

Food pleasantness is largely based on the palatability of food and is linked to taste. Along with homeostatic and cognitive control, it forms part of the control of food intake (hedonic control), and does not only correspond to the pleasure that can be described of food intake. There are many factors that cause variations in eating pleasantness between individuals, such as age, sex, culture, co-morbidities, treatments, environmental factors or the specific characteristics of foods. The control of food intake is based on four determinants: conditioned satiety, the reward system, sensory specific satiety and alliesthesia. These four determinants follow one another over time, in the per-prandial and inter-prandial periods, and complement one another. There are many cerebral areas involved in the hedonic control of food intake. The most involved brain areas are the orbitofrontal and anterior cingulate cortices, which interact with deep neural structures (amygdala, striatum, substantia nigra) for the reward circuit, with the hippocampi for memorising pleasant foods, and even with the hypothalamus and insula, brain areas more recently involved in the physiology of food pleasantness. Changes in brain activity secondary to modulation of food pleasantness can be measured objectively by recording taste-evoked potentials, an electroencephalography technique with very good temporal resolution.

Adverse effects of thimerosal on the early life stages of zebrafish

Thimerosal (THI) is an organic mercury compound that is widely used in drugs, vaccines and antibacterial products. Its extensive production and use have resulted in significant environmental contamination, posing a considerable threat to aquatic life. However, the knowledge of the toxicity of THI to aquatic organisms is still insufficient. In this study, we conducted a 5-day THI exposure experiment using zebrafish, from 0 to 5 days post fertilization (dpf). The possible adverse effects of THI on the early-life stages of zebrafish were explored by investigating variations in their physiological parameters, behavioral traits, and neurotransmitter levels. The results showed THI exhibited significant developmental toxicity to aquatic organisms. Exposure to THI significantly induced serious malformation (at 50 μg/L), accelerated hatching, and elevated heart rate (at 5 and 50 μg/L). The behavioral traits of zebrafish larvae had an increased first and then decreased relationship with increasing concentration of THI, which induced hyperactivity at 0.5 μg/L but opposite at 50 μg/L. Furthermore, exposure to 50 μg/L THI significantly raised levels of 5-HT, 5-HIAA, DA, DOPAC and ACH in zebrafish larvae. In addition, several significant correlations between behavioral traits and the neurotransmitter contents were detected, which seemed to reveal an important mechanism of the neurobehavioral toxicity of THI to fish.

Stress-induced overeating behaviors explained from a (transitory) relief-learning perspective

People use various behaviors to cope with stressful events. These behaviors are mostly adaptive, as they allow a successful release of stress without impacting other aspects of life: How nice is it to have a break with a few spoons of that favorite ice cream during a hectic working day? However, when excessive consumption of high-sugar/salt ultra-processed food becomes the gateway to find relief from stress, eating loses this adaptive function and may escalate to binge eating, lead to obesity, and other medical conditions linked to overweight. Several etiological models attempt to explain stress-induced eating and excessive overeating behaviors characterizing these clinical conditions. The popular Emotional Eating Theory proposes that stress-related (over-)eating, a major predictor of obesity and diagnosed binge eating disorders, develops based on negative reinforcement learning since food consumption regulates the negative affective state associated with stressful circumstances. Differently, the prominent Incentive Sensitization Theory explains overeating, binge eating disorders (including bulimia), and obesity in terms of excessive amplification of reward 'wanting', which is thought to emerge from overexposure to obesogenic (food)cues. The several studies oriented by these theories have paved the way to better understand stress-related (over-)eating and its clinical excesses. However, a deep mechanistic understanding of how and why stress-induced (over-)eating can escalate till clinical forms of overeating remain elusive. A well-funded connection of the mechanisms proposed by the Emotional Eating Theory and the Incentive Sensitization Theory might address this etiological open question. To avoid erroneous arguments, it is however essential to first address the internal theoretical and methodological shortcomings of each theory and connected studies. These shortcomings stem from conceptual fallacies and poorly implemented designs, which might partially explain the 'high variability and low replicability' problem of empirical findings. Next, the formulation of a new integrative model could provide fresh insight into the deep learning and biological mechanisms of this escalation. A successful formalization of this model could then create the much-needed impact in clinical and preventive research since excessive overeating is a behavior hard to change once established. In this opinion paper, I propose to apply recent insights we gathered on the role of relief from the field of safety learning to stress (over-)eating. I will present a new relief-based model that, as a starting point, has the potential to connect the Emotional Eating Theory with the Incentive Sensitization Theory, setting the base for more integrative science.

Imaging of impulse control disorders in Parkinson's disease

Impulse control disorders (ICD) are frequent and cumbersome behavioral disorders in patients with Parkinson's disease (PD). Understanding their pathophysiological underpinnings is crucial. Molecular imaging using positron emission tomography (PET) and single-photon emission computed tomography (SPECT) clearly indicates preexisting vulnerability and abnormal sensitization of the pre- and postsynaptic dopaminergic system. Functional magnetic resonance imaging (fMRI) studies reveal abnormal connectivity within the reward system involving the ventral striatum and orbitofrontal cortex. These alterations pinpoint the dysfunction of reinforcement learning in ICD, which is biased toward the overvaluation of reward and underestimation of risk, and the deficit in inhibitory control mechanisms related to abnormal connectivity within and between the limbic and the associative and motor networks.

Dopamine and Norepinephrine Differentially Mediate the Exploration-Exploitation Tradeoff

Dopamine (DA) and norepinephrine (NE) have been repeatedly implicated in neuropsychiatric vulnerability, in part via their roles in mediating the decision-making processes. Although two neuromodulators share a synthesis pathway and are coactivated under states of arousal, they engage in distinct circuits and modulatory roles. However, the specific role of each neuromodulator in decision-making, in particular the exploration-exploitation tradeoff, remains unclear. Revealing how each neuromodulator contributes to exploration-exploitation tradeoff is important in guiding mechanistic hypotheses emerging from computational psychiatric approaches. To understand the differences and overlaps of the roles of these two catecholamine systems in regulating exploration, a direct comparison using the same dynamic decision-making task is needed. Here, we ran male and female mice in a restless two-armed bandit task, which encourages both exploration and exploitation. We systemically administered a nonselective DA antagonist (flupenthixol), a nonselective DA agonist (apomorphine), a NE beta-receptor antagonist (propranolol), and a NE beta-receptor agonist (isoproterenol) and examined changes in exploration within subjects across sessions. We found a bidirectional modulatory effect of dopamine on exploration. Increasing dopamine activity decreased exploration and decreasing dopamine activity increased exploration. The modulatory effect of beta-noradrenergic receptor activity on exploration was mediated by sex. Reinforcement learning model parameters suggested that dopamine modulation affected exploration via decision noise and norepinephrine modulation affected exploration via sensitivity to outcome. Together, these findings suggested that the mechanisms that govern the exploration-exploitation transition are sensitive to changes in both catecholamine functions and revealed differential roles for NE and DA in mediating exploration.

Consensus Paper: Cerebellum and Reward

Cerebellum is a key-structure for the modulation of motor, cognitive, social and affective functions, contributing to automatic behaviours through interactions with the cerebral cortex, basal ganglia and spinal cord. The predictive mechanisms used by the cerebellum cover not only sensorimotor functions but also reward-related tasks. Cerebellar circuits appear to encode temporal difference error and reward prediction error. From a chemical standpoint, cerebellar catecholamines modulate the rate of cerebellar-based cognitive learning, and mediate cerebellar contributions during complex behaviours. Reward processing and its associated emotions are tuned by the cerebellum which operates as a controller of adaptive homeostatic processes based on interoceptive and exteroceptive inputs. Lobules VI-VII/areas of the vermis are candidate regions for the cortico-subcortical signaling pathways associated with loss aversion and reward sensitivity, together with other nodes of the limbic circuitry. There is growing evidence that the cerebellum works as a hub of regional dysconnectivity across all mood states and that mental disorders involve the cerebellar circuitry, including mood and addiction disorders, and impaired eating behaviors where the cerebellum might be involved in longer time scales of prediction as compared to motor operations. Cerebellar patients exhibit aberrant social behaviour, showing aberrant impulsivity/compulsivity. The cerebellum is a master-piece of reward mechanisms, together with the striatum, ventral tegmental area (VTA) and prefrontal cortex (PFC). Critically, studies on reward processing reinforce our view that a fundamental role of the cerebellum is to construct internal models, perform predictions on the impact of future behaviour and compare what is predicted and what actually occurs.

Functional connectivity between the nucleus accumbens and amygdala underlies avoidance learning during adolescence: Implications for developmental psychopathology

BACKGROUND

Reward and threat processes work together to support adaptive learning during development. Adolescence is associated with increasing approach behavior (e.g., novelty-seeking, risk-taking) but often also coincides with emerging internalizing symptoms, which are characterized by heightened avoidance behavior. Peaking engagement of the nucleus accumbens (NAcc) during adolescence, often studied in reward paradigms, may also relate to threat mechanisms of adolescent psychopathology.

METHODS

47 typically developing adolescents (9.9-22.9 years) completed an aversive learning task during functional magnetic resonance imaging, wherein visual cues were paired with an aversive sound or no sound. Task blocks involved an escapable aversively reinforced stimulus (CS+r), the same stimulus without reinforcement (CS+nr), or a stimulus that was never reinforced (CS-). Parent-reported internalizing symptoms were measured using Revised Child Anxiety and Depression Scales.

RESULTS

Functional connectivity between the NAcc and amygdala differentiated the stimuli, such that connectivity increased for the CS+r (p = .023) but not for the CS+nr and CS-. Adolescents with greater internalizing symptoms demonstrated greater positive functional connectivity for the CS- (p = .041).

CONCLUSIONS

Adolescents show heightened NAcc-amygdala functional connectivity during escape from threat. Higher anxiety and depression symptoms are associated with elevated NAcc-amygdala connectivity during safety, which may reflect poor safety versus threat discrimination.

Music Listening as Exploratory Behavior: From Dispositional Reactions to Epistemic Interactions with the Sonic World

Listening to music can span a continuum from passive consumption to active exploration, relying on processes of coping with the sounds as well as higher-level processes of sense-making. Revolving around the major questions of "what" and "how" to explore, this paper takes a naturalistic stance toward music listening, providing tools to objectively describe the underlying mechanisms of musical sense-making by weakening the distinction between music and non-music. Starting from a non-exclusionary conception of "coping" with the sounds, it stresses the exploratory approach of treating music as a sound environment to be discovered by an attentive listener. Exploratory listening, in this view, is an open-minded and active process, not dependent on simply recalling pre-existing knowledge or information that reduces cognitive processing efforts but having a high cognitive load due to the need for highly focused attention and perceptual readiness. Music, explored in this way, is valued for its complexity, surprisingness, novelty, incongruity, puzzlingness, and patterns, relying on processes of selection, differentiation, discrimination, and identification.

Cognitive effects of early life exposure to PCBs: Sex-specific behavioral, hormonal and neuromolecular mechanisms involving the brain dopamine system

Endocrine-disrupting chemicals (EDCs) are environmental toxicants that disrupt hormonal and neurodevelopmental processes. Among these chemicals, polychlorinated biphenyls (PCBs) are particularly concerning due to their resistance to biodegradation and tendency to bioaccumulate. PCBs affect neurodevelopmental function and disrupt the brain's dopamine (DA) system, which is crucial for attentional, affective, and reward processing. These disruptions may contribute to the rising prevalence of DA-mediated neuropsychiatric disorders such as ADHD, depression, and substance use disorders. Notably, these behaviors are sexually dimorphic, in part due to differences in sex hormones and their receptors, which are targets of estrogenic PCBs. Therefore, this study determined effects of early life PCB exposure on behaviors and neurochemistry related to potential disruption of dopaminergic signaling. Male and female Sprague Dawley rats were exposed to PCBs or vehicle perinatally and then underwent a series of behavioral tests, including the sucrose preference test to measure affect, conditioned orienting to assess incentive-motivational phenotype, and attentional set-shifting to evaluate cognitive flexibility and response latency. Following these tests, rats were euthanized, and we measured serum estradiol (E2), midbrain DA cells, and gene expression in the midbrain. Female rats exposed perinatally to A1221 exhibited decreased sucrose preference, and both male and female A1221 rats had reduced response latency in the attentional set-shifting task compared to vehicle counterparts. Conditioned orienting, serum estradiol (E2), and midbrain DA cell numbers were not affected in either sex; however, A1221-exposed male rats displayed higher expression of estrogen receptor alpha ( Esr1 ) in the midbrain and non-significant effects on other DA-signaling genes. Additionally, E2 uniquely predicted behavioral outcomes and DAergic cell numbers in A1221-exposed female rats, whereas DA signaling genes were predictive of behavioral outcomes in males. These data highlight sex-specific effects of A1221 on neuromolecular and behavioral phenotypes.

The effect of value on context and target recollection in memory for truth and falsity

Memory for truth and falsity has recently been investigated from the perspective of the dual-recollection theory, showing better context and target recollection for truth than falsity. In this paper, we examine whether these memory effects obtained for true statements are similar to the value effect, whereby true statements are given higher priority in encoding. For this purpose, we implemented value-directed remembering (VDR) into the conjoint-recognition paradigm. In our first experiment, the primary goal was to verify how VDR influences the processes defined by dual-recollection theory. At study, prioritized/important items were linked to higher numerical values (e.g., 10), while unimportant ones had lower values (e.g., 1). At test, the participants' task was to recognize whether a particular sentence was important, unimportant, or new. We found that both context and target recollection were better for important items. In the second experiment, the main goal was to study the combined effects of importance and veracity on memory. In the between-subjects design, participants were monetarily rewarded for memorizing true or false sentences. The results demonstrated differences in the ability to prioritize truth over falsity. Specifically, we found a substantial increase in context recollection for prioritized true information but not for prioritized false information. Moreover, we found higher context recollection for true than false sentences in the true-prioritized condition, but not in the false-prioritized condition. These results indicated that people are able to prioritize true information better than false, and suggested that memory for truth may be a special case of the value effect.

Bibliometric analysis of functional magnetic resonance imaging studies on chronic pain over the past 20 years

PURPOSE

The utilization of functional magnetic resonance imaging (fMRI) in studying the mechanisms and treatment of chronic pain has gained significant popularity. However, there is currently a dearth of literature conducting bibliometric analysis on fMRI studies focused on chronic pain.

METHODS

All the literature included in this study was obtained from the Science Citation Index Expanded of Web of Science Core Collection. We used CiteSpace and VOSviewer to analyze publications, authors, countries or regions, institutions, journals, references and keywords. Additionally, we evaluated the timeline and burst analysis of keywords, as well as the timeline and burst analysis of references. The search was conducted from 2004 to 2023 and completed within a single day on October 4th, 2023.

RESULTS

A total of 1,327 articles were retrieved. The annual publication shows an overall increasing trend. The United States has the highest number of publications and the main contributing institution is Harvard University. The journal PAIN produces the most articles. In recent years, resting-state fMRI, the prefrontal cortex, nucleus accumbens, thalamus, and migraines have been researched hotspots of fMRI studies on chronic pain.

CONCLUSIONS

This study provides an in-depth perspective on fMRI for chronic pain research, revealing key points, research hotspots and research trends, which offers valuable ideas for future research activities. It concludes with a summary of advances in clinical practice in this area, pointing out the need for critical evaluation of these findings in the light of guidelines and expert recommendations. It is anticipated that further high-quality research outputs will be generated in the future, which will facilitate the utilization of fMRI in clinical decision-making for chronic pain.

Effects of Chronic Exposure to Low Doses of Rotenone on Dopaminergic and Cholinergic Neurons in the CNS of Hemigrapsus sanguineus

Rotenone, as a common pesticide and insecticide frequently found in environmental samples, may be present in aquatic habitats worldwide. Exposure to low concentrations of this compound may cause alterations in the nervous system, thus contributing to Parkinsonian motor symptoms in both vertebrates and invertebrates. However, the effects of chronic exposure to low doses of rotenone on the activity of neurotransmitters that govern motor functions and on the specific molecular mechanisms leading to movement morbidity remain largely unknown for many aquatic invertebrates. In this study, we analyzed the effects that rotenone poisoning exerts on the activity of dopamine (DA) and acetylcholine (ACh) synthesis enzymes in the central nervous system (CNS) of Asian shore crab, Hemigrapsus sanguineus (de Haan, 1835), and elucidated the association of its locomotor behavior with Parkinson's-like symptoms. An immunocytochemistry analysis showed a reduction in tyrosine hydroxylase (TH) in the median brain and the ventral nerve cord (VNC), which correlated with the subsequent decrease in the locomotor activity of shore crabs. We also observed a variation in cholinergic neurons' activity, mostly in the ventral regions of the VNC. Moreover, the rotenone-treated crabs showed signs of damage to ChAT-lir neurons in the VNC. These data suggest that chronic treatment with low doses of rotenone decreases the DA level in the VNC and the ACh level in the brain and leads to progressive and irreversible reductions in the crab's locomotor activity, life span, and changes in behavior.

Distinct basal ganglia contributions to learning from implicit and explicit value signals in perceptual decision-making

Metacognitive evaluations of confidence provide an estimate of decision accuracy that could guide learning in the absence of explicit feedback. We examine how humans might learn from this implicit feedback in direct comparison with that of explicit feedback, using simultaneous EEG-fMRI. Participants performed a motion direction discrimination task where stimulus difficulty was increased to maintain performance, with intermixed explicit- and no-feedback trials. We isolate single-trial estimates of post-decision confidence using EEG decoding, and find these neural signatures re-emerge at the time of feedback together with separable signatures of explicit feedback. We identified these signatures of implicit versus explicit feedback along a dorsal-ventral gradient in the striatum, a finding uniquely enabled by an EEG-fMRI fusion. These two signals appear to integrate into an aggregate representation in the external globus pallidus, which could broadcast updates to improve cortical decision processing via the thalamus and insular cortex, irrespective of the source of feedback.

Either Autonomy Support or Enhanced Expectancies Delivered Via Virtual-Reality Benefits Frontal-Plane Single-Leg Squatting Kinematics

Our purpose in this study was to determine the effects of a virtual reality intervention delivering specific motivational motor learning manipulations of either autonomy support (AS) or enhanced expectancies (EE) on frontal plane single-leg squatting kinematics. We allocated 45 participants (21 male, 24 female) demonstrating knee, hip, and trunk frontal plane mechanics associated with elevated anterior cruciate ligament injury risk to one of three groups (control, AS, or EE). Participants mimicked an avatar performing five sets of eight repetitions of exemplary single-leg squats. AS participants were given the added option of choosing the color of their avatar. EE participants received real-time biofeedback in the form of green highlights on the avatar that remained on as long as the participant maintained pre-determined 'safe' frontal plane mechanics. We measured peak frontal plane knee, hip, and trunk angles before (baseline) and immediately following (post) the intervention. The control group demonstrated greater increases in knee abduction angle (Δ = +2.3°) than did the AS (Δ = +0.1°) and EE groups (Δ = -0.4°) (p = .003; η2p = .28). All groups demonstrated increased peak hip adduction (p = .01, ηp2 = .18) (control Δ = +1.5°; AS Δ = +3.2°; EE Δ = +0.7°). Hip adduction worsened in all groups. AS and EE motivation strategies appeared to mitigate maladaptive frontal plane knee mechanics.

The neural mechanism of communication between graduate students and advisers in different adviser-advisee relationships

Communication is crucial in constructing the relationship between students and advisers, ultimately bridging interpersonal interactions. Only a few studies however explore the communication between postgraduate students and advisers. To fill the gaps in the empirical researches, this study uses functional near-infrared spectroscopy (FNIRS) techniques to explore the neurophysiology differences in brain activation of postgraduates with different adviser-advise relationships during simulated communication with their advisers. Results showed significant differences in the activation of the prefrontal cortex between high-quality and the low-quality students during simulating and when communicating with advisers, specifically in the Broca's areas, the frontal pole, and the orbitofrontal and dorsolateral prefrontal cortices. This further elucidated the complex cognitive process of communication between graduate students and advisers.

Hierarchical control over foraging behavior by anterior cingulate cortex

Foraging is a natural behavior that involves making sequential decisions to maximize rewards while minimizing the costs incurred when doing so. The prevalence of foraging across species suggests that a common brain computation underlies its implementation. Although anterior cingulate cortex is believed to contribute to foraging behavior, its specific role has been contentious, with predominant theories arguing either that it encodes environmental value or choice difficulty. Additionally, recent attempts to characterize foraging have taken place within the reinforcement learning framework, with increasingly complex models scaling with task complexity. Here we review reinforcement learning foraging models, highlighting the hierarchical structure of many foraging problems. We extend this literature by proposing that ACC guides foraging according to principles of model-based hierarchical reinforcement learning. This idea holds that ACC function is organized hierarchically along a rostral-caudal gradient, with rostral structures monitoring the status and completion of high-level task goals (like finding food), and midcingulate structures overseeing the execution of task options (subgoals, like harvesting fruit) and lower-level actions (such as grabbing an apple).

Parental exposure to antidepressants has lasting effects on offspring? A case study with zebrafish

Fish have common neurotransmitter pathways with humans, exhibiting a significant degree of conservation and homology. Thus, exposure to fluoxetine makes fish potentially susceptible to biochemical and physiological changes, similarly to what is observed in humans. Over the years, several studies demonstrated the potential effects of fluoxetine on different fish species and at different levels of biological organization. However, the effects of parental exposure to unexposed offspring remain largely unknown. The consequences of 15-day parental exposure to relevant concentrations of fluoxetine (100 and 1000 ng/L) were assessed on offspring using zebrafish as a model organism. Parental exposure resulted in offspring early hatching, non-inflation of the swimming bladder, increased malformation frequency, decreased heart rate and blood flow, and reduced growth. Additionally, a significant behavioral impairment was also found (reduced startle response, basal locomotor activity, and altered non-associative learning during early stages and a negative geotaxis and scototaxis, reduced thigmotaxis, and anti-social behavior at later life stages). These behavior alterations are consistent with decreased anxiety, a significant increase in the expression of the monoaminergic genes slc6a4a (sert), slc6a3 (dat), slc18a2 (vmat2), mao, tph1a, and th2, and altered levels of monoaminergic neurotransmitters. Alterations in behavior, expression of monoaminergic genes, and neurotransmitter levels persisted until offspring adulthood. Given the high conservation of neuronal pathways between fish and humans, data show the possibility of potential transgenerational and multigenerational effects of pharmaceuticals' exposure. These results reinforce the need for transgenerational and multigenerational studies in fish, under realistic scenarios, to provide realistic insights into the impact of these pharmaceuticals.

Two what, two where, visual cortical streams in humans

ROLLS, E. T. Two What, Two Where, Visual Cortical Streams in Humans. NEUROSCI BIOBEHAV REV 2024. Recent cortical connectivity investigations lead to new concepts about 'What' and 'Where' visual cortical streams in humans, and how they connect to other cortical systems. A ventrolateral 'What' visual stream leads to the inferior temporal visual cortex for object and face identity, and provides 'What' information to the hippocampal episodic memory system, the anterior temporal lobe semantic system, and the orbitofrontal cortex emotion system. A superior temporal sulcus (STS) 'What' visual stream utilising connectivity from the temporal and parietal visual cortex responds to moving objects and faces, and face expression, and connects to the orbitofrontal cortex for emotion and social behaviour. A ventromedial 'Where' visual stream builds feature combinations for scenes, and provides 'Where' inputs via the parahippocampal scene area to the hippocampal episodic memory system that are also useful for landmark-based navigation. The dorsal 'Where' visual pathway to the parietal cortex provides for actions in space, but also provides coordinate transforms to provide inputs to the parahippocampal scene area for self-motion update of locations in scenes in the dark or when the view is obscured.

Impaired flexible reward learning in ADHD patients is associated with blunted reinforcement sensitivity and neural signals in ventral striatum and parietal cortex

Reward-based learning and decision-making are prime candidates to understand symptoms of attention deficit hyperactivity disorder (ADHD). However, only limited evidence is available regarding the neurocomputational underpinnings of the alterations seen in ADHD. This concerns flexible behavioral adaption in dynamically changing environments, which is challenging for individuals with ADHD. One previous study points to elevated choice switching in adolescent ADHD, which was accompanied by disrupted learning signals in medial prefrontal cortex. Here, we investigated young adults with ADHD (n = 17) as compared to age- and sex-matched controls (n = 17) using a probabilistic reversal learning experiment during functional magnetic resonance imaging (fMRI). The task requires continuous learning to guide flexible behavioral adaptation to changing reward contingencies. To disentangle the neurocomputational underpinnings of the behavioral data, we used reinforcement learning (RL) models, which informed the analysis of fMRI data. ADHD patients performed worse than controls particularly in trials before reversals, i.e., when reward contingencies were stable. This pattern resulted from 'noisy' choice switching regardless of previous feedback. RL modelling showed decreased reinforcement sensitivity and enhanced learning rates for negative feedback in ADHD patients. At the neural level, this was reflected in a diminished representation of choice probability in the left posterior parietal cortex in ADHD. Moreover, modelling showed a marginal reduction of learning about the unchosen option, which was paralleled by a marginal reduction in learning signals incorporating the unchosen option in the left ventral striatum. Taken together, we show that impaired flexible behavior in ADHD is due to excessive choice switching ('hyper-flexibility'), which can be detrimental or beneficial depending on the learning environment. Computationally, this resulted from blunted sensitivity to reinforcement of which we detected neural correlates in the attention-control network, specifically in the parietal cortex. These neurocomputational findings remain preliminary due to the relatively small sample size.

An excitatory projection from the basal forebrain to the ventral tegmental area that underlies anorexia-like phenotypes

Maladaptation in balancing internal energy needs and external threat cues may result in eating disorders. However, brain mechanisms underlying such maladaptations remain elusive. Here, we identified that the basal forebrain (BF) sends glutamatergic projections to glutamatergic neurons in the ventral tegmental area (VTA) in mice. Glutamatergic neurons in both regions displayed correlated responses to various stressors. Notably, in vivo manipulation of BF terminals in the VTA revealed that the glutamatergic BF → VTA circuit reduces appetite, increases locomotion, and elicits avoidance. Consistently, activation of VTA glutamatergic neurons reduced body weight, blunted food motivation, and caused hyperactivity with behavioral signs of anxiety, all hallmarks of typical anorexia symptoms. Importantly, activation of BF glutamatergic terminals in the VTA reduced dopamine release in the nucleus accumbens. Collectively, our results point to overactivation of the glutamatergic BF → VTA circuit as a potential cause of anorexia-like phenotypes involving reduced dopamine release.

Neural adaptation of the reward system in primary dysmenorrhea

Introduction: The brain's reward system (RS) reacts differently to pain and its alleviation. This study examined the correlation between RS activity and behavior during both painful and pain-free periods in individuals with primary dysmenorrhea (PDM) to elucidate their varying responses throughout the menstrual cycle. Methods: Ninety-two individuals with PDM and 90 control participants underwent resting-state functional magnetic resonance imaging (rsfMRI) scans during their menstrual and peri-ovulatory phases. Regional homogeneity (ReHo) and amplitude of low-frequency fluctuation (ALFF) analyses were used to evaluate RS responses. Psychological evaluations were conducted using the McGill Pain Questionnaire and the Pain Catastrophizing Scale. Results: ReHo analysis showed higher values in the left putamen and right amygdala of the PDM group during the peri-ovulatory phase compared to the menstrual phase. ALFF analysis revealed lower values in the putamen of the PDM group compared to controls, regardless of phase. ReHo and ALFF values in the putamen, amygdala, and nucleus accumbens were positively correlated with pain scales during menstruation, while ALFF values in the ventral tegmental area inversely correlated with pain intensity. Those with severe PDM (pain intensity ≥7) displayed distinct amygdala ALFF patterns between pain and pain-free phases. PDM participants also had lower ReHo values in the left insula during menstruation, with no direct correlation to pain compared to controls. Discussion: Our study highlights the pivotal role of the RS in dysmenorrhea management, exhibiting varied responses between menstrual discomfort and non-painful periods among individuals with PDM. During menstruation, the RS triggers mechanisms for pain avoidance and cognitive coping strategies, while it transitions to processing rewards during the peri-ovulatory phase. This demonstrates the flexibility of the RS in adapting to the recurring pain experienced by those with PDM.

Context changes judgments of liking and predictability for melodies

Predictability plays an important role in the experience of musical pleasure. By leveraging expectations, music induces pleasure through tension and surprise. However, musical predictions draw on both prior knowledge and immediate context. Similarly, musical pleasure, which has been shown to depend on predictability, may also vary relative to the individual and context. Although research has demonstrated the influence of both long-term knowledge and stimulus features in influencing expectations, it is unclear how perceptions of a melody are influenced by comparisons to other music pieces heard in the same context. To examine the effects of context we compared how listeners' judgments of two distinct sets of stimuli differed when they were presented alone or in combination. Stimuli were excerpts from a repertoire of Western music and a set of experimenter created melodies. Separate groups of participants rated liking and predictability for each set of stimuli alone and in combination. We found that when heard together, the Repertoire stimuli were more liked and rated as less predictable than if they were heard alone, with the opposite pattern being observed for the Experimental stimuli. This effect was driven by a change in ratings between the Alone and Combined conditions for each stimulus set. These findings demonstrate a context-based shift of predictability ratings and derived pleasure, suggesting that judgments stem not only from the physical properties of the stimulus, but also vary relative to other options available in the immediate context.

Approach-avoidance reinforcement learning as a translational and computational model of anxiety-related avoidance

Although avoidance is a prevalent feature of anxiety-related psychopathology, differences in the measurement of avoidance between humans and non-human animals hinder our progress in its theoretical understanding and treatment. To address this, we developed a novel translational measure of anxiety-related avoidance in the form of an approach-avoidance reinforcement learning task, by adapting a paradigm from the non-human animal literature to study the same cognitive processes in human participants. We used computational modelling to probe the putative cognitive mechanisms underlying approach-avoidance behaviour in this task and investigated how they relate to subjective task-induced anxiety. In a large online study (n = 372), participants who experienced greater task-induced anxiety avoided choices associated with punishment, even when this resulted in lower overall reward. Computational modelling revealed that this effect was explained by greater individual sensitivities to punishment relative to rewards. We replicated these findings in an independent sample (n = 627) and we also found fair-to-excellent reliability of measures of task performance in a sub-sample retested 1 week later (n = 57). Our findings demonstrate the potential of approach-avoidance reinforcement learning tasks as translational and computational models of anxiety-related avoidance. Future studies should assess the predictive validity of this approach in clinical samples and experimental manipulations of anxiety.

Human thalamic low-frequency oscillations correlate with expected value and outcomes during reinforcement learning

Reinforcement-based adaptive decision-making is believed to recruit fronto-striatal circuits. A critical node of the fronto-striatal circuit is the thalamus. However, direct evidence of its involvement in human reinforcement learning is lacking. We address this gap by analyzing intra-thalamic electrophysiological recordings from eight participants while they performed a reinforcement learning task. We found that in both the anterior thalamus (ATN) and dorsomedial thalamus (DMTN), low frequency oscillations (LFO, 4-12 Hz) correlated positively with expected value estimated from computational modeling during reward-based learning (after outcome delivery) or punishment-based learning (during the choice process). Furthermore, LFO recorded from ATN/DMTN were also negatively correlated with outcomes so that both components of reward prediction errors were signaled in the human thalamus. The observed differences in the prediction signals between rewarding and punishing conditions shed light on the neural mechanisms underlying action inhibition in punishment avoidance learning. Our results provide insight into the role of thalamus in reinforcement-based decision-making in humans.

Deficits in consummatory reward relate to severity of cocaine use

BACKGROUND AND AIMS

Identifying modifiable neuropsychological factors associated with more severe CUD could improve CUD treatment. Impairments in processing of non-drug rewards may be one such factor. This study assessed the relationship between reward functioning and cocaine use severity using multi-modal measures of three distinct reward functions: consummatory reward (pleasure or "liking"); motivational reward ("wanting") and reward learning.

METHODS

Fifty-three adults with at least moderate CUD completed self-report and behavioral measures of consummatory reward, motivational reward and reward learning, and a composite cocaine use severity measure including quantity, frequency and life impacts of cocaine use. We conducted parallel Frequentist and Bayesian multiple regressions with measures of reward functioning as predictors of cocaine use severity.

RESULTS

Less self-reported ability to experience pleasure, a hypothesized measure of consummatory reward, significantly predicted greater severity after adjustment for covariates and multiple hypothesis testing, β = 0.39, t(38) = 2.86, p = 0.007. Bayesian analyses confirmed a highly likely association between severity and ability to experience pleasure, and provided moderate evidence for associations with willingness to exert effort and reward learning.

CONCLUSIONS

Our results suggest that less experience of subjective pleasure is related to greater cocaine use severity. This cross-sectional study cannot establish whether differences in consummatory reward are pre-existing, a result of CUD, or both. However, these results suggest interventions focused on increasing subjective pleasure, such as mindful "savoring", should be investigated for CUD.

Dopamine and norepinephrine differentially mediate the exploration-exploitation tradeoff

The catecholamines dopamine (DA) and norepinephrine (NE) have been repeatedly implicated in neuropsychiatric vulnerability, in part via their roles in mediating the decision making processes. Although the two neuromodulators share a synthesis pathway and are co-activated under states of arousal, they engage in distinct circuits and roles in modulating neural activity across the brain. However, in the computational neuroscience literature, they have been assigned similar roles in modulating the latent cognitive processes of decision making, in particular the exploration-exploitation tradeoff. Revealing how each neuromodulator contributes to this explore-exploit process will be important in guiding mechanistic hypotheses emerging from computational psychiatric approaches. To understand the differences and overlaps of the roles of these two catecholamine systems in regulating exploration and exploitation, a direct comparison using the same dynamic decision making task is needed. Here, we ran mice in a restless two-armed bandit task, which encourages both exploration and exploitation. We systemically administered a nonselective DA receptor antagonist (flupenthixol), a nonselective DA receptor agonist (apomorphine), a NE beta-receptor antagonist (propranolol), and a NE beta-receptor agonist (isoproterenol), and examined changes in exploration within subjects across sessions. We found a bidirectional modulatory effect of dopamine receptor activity on the level of exploration. Increasing dopamine activity decreased exploration and decreasing dopamine activity increased exploration. Beta-noradrenergic receptor activity also modulated exploration, but the modulatory effect was mediated by sex. Reinforcement learning model parameters suggested that dopamine modulation affected exploration via decision noise and norepinephrine modulation affected exploration via outcome sensitivity. Together, these findings suggested that the mechanisms that govern the transition between exploration and exploitation are sensitive to changes in both catecholamine functions and revealed differential roles for NE and DA in mediating exploration.

Apathy in Parkinson's Disease: Clinical Patterns and Neurobiological Basis

Apathy is commonly defined as a loss of motivation leading to a reduction in goal-directed behaviors. This multidimensional syndrome, which includes cognitive, emotional and behavioral components, is one of the most prevalent neuropsychiatric features of Parkinson's disease (PD). It has been established that the prevalence of apathy increases as PD progresses. However, the pathophysiology and anatomic substrate of this syndrome remain unclear. Apathy seems to be underpinned by impaired anatomical structures that link the prefrontal cortex with the limbic system. It can be encountered in the prodromal stage of the disease and in fluctuating PD patients receiving bilateral chronic subthalamic nucleus stimulation. In these stages, apathy may be considered as a disorder of motivation that embodies amotivational behavioral syndrome, is underpinned by combined dopaminergic and serotonergic denervation and is dopa-responsive. In contrast, in advanced PD patients, apathy may be considered as cognitive apathy that announces cognitive decline and PD dementia, is underpinned by diffuse neurotransmitter system dysfunction and Lewy pathology spreading and is no longer dopa-responsive. In this review, we discuss the clinical patterns of apathy and their treatment, the neurobiological basis of apathy, the potential role of the anatomical structures involved and the pathways in motivational and cognitive apathy.

Emotion, motivation, decision-making, the orbitofrontal cortex, anterior cingulate cortex, and the amygdala

The orbitofrontal cortex and amygdala are involved in emotion and in motivation, but the relationship between these functions performed by these brain structures is not clear. To address this, a unified theory of emotion and motivation is described in which motivational states are states in which instrumental goal-directed actions are performed to obtain rewards or avoid punishers, and emotional states are states that are elicited when the reward or punisher is or is not received. This greatly simplifies our understanding of emotion and motivation, for the same set of genes and associated brain systems can define the primary or unlearned rewards and punishers such as sweet taste or pain. Recent evidence on the connectivity of human brain systems involved in emotion and motivation indicates that the orbitofrontal cortex is involved in reward value and experienced emotion with outputs to cortical regions including those involved in language, and is a key brain region involved in depression and the associated changes in motivation. The amygdala has weak effective connectivity back to the cortex in humans, and is implicated in brainstem-mediated responses to stimuli such as freezing and autonomic activity, rather than in declarative emotion. The anterior cingulate cortex is involved in learning actions to obtain rewards, and with the orbitofrontal cortex and ventromedial prefrontal cortex in providing the goals for navigation and in reward-related effects on memory consolidation mediated partly via the cholinergic system.

Attachment, behavior problems and interventions

This paper puts forward an explanation for the frequent co-occurrence of attachment and behavior problems in children and the implications of this for interventions; presents preliminary evidence that some behaviorally based parenting programs reduce child behavior problems through two separate, but mutually reinforcing, processes-improved attachment relationships and increased parental use of behavior management techniques; and suggests next steps for the field to improve outcomes for those children who, without interventions that addresses both relationship building and behavior management, are at risk of significant long-term difficulties.

Influence of Recent Trial History on Interval Timing

Interval timing is involved in a variety of cognitive behaviors such as associative learning and decision-making. While it has been shown that time estimation is adaptive to the temporal context, it remains unclear how interval timing behavior is influenced by recent trial history. Here we found that, in mice trained to perform a licking-based interval timing task, a decrease of inter-reinforcement interval in the previous trial rapidly shifted the time of anticipatory licking earlier. Optogenetic inactivation of the anterior lateral motor cortex (ALM), but not the medial prefrontal cortex, for a short time before reward delivery caused a decrease in the peak time of anticipatory licking in the next trial. Electrophysiological recordings from the ALM showed that the response profiles preceded by short and long inter-reinforcement intervals exhibited task-engagement-dependent temporal scaling. Thus, interval timing is adaptive to recent experience of the temporal interval, and ALM activity during time estimation reflects recent experience of interval.

Neuro-computational mechanisms and individual biases in action-outcome learning under moral conflict

Learning to predict action outcomes in morally conflicting situations is essential for social decision-making but poorly understood. Here we tested which forms of Reinforcement Learning Theory capture how participants learn to choose between self-money and other-shocks, and how they adapt to changes in contingencies. We find choices were better described by a reinforcement learning model based on the current value of separately expected outcomes than by one based on the combined historical values of past outcomes. Participants track expected values of self-money and other-shocks separately, with the substantial individual difference in preference reflected in a valuation parameter balancing their relative weight. This valuation parameter also predicted choices in an independent costly helping task. The expectations of self-money and other-shocks were biased toward the favored outcome but fMRI revealed this bias to be reflected in the ventromedial prefrontal cortex while the pain-observation network represented pain prediction errors independently of individual preferences.

The challenge of memory destabilisation: From prediction error to prior expectations and biomarkers

The re-ignition of memory reconsolidation research sparked by Karim Nader in the early 2000s led to great excitement that 'reconsolidation-based' interventions might be developed for mental health disorders such as post-traumatic stress disorder and substance use disorder. Two decades on, it is clear that reconsolidation-based interventions have been more challenging to translate to the clinic than initially thought. We argue that this challenge could be addressed with a better understanding of how prior expectations interact with information presented in a putative memory reactivation / cue reminder session, and through the identification of non-invasive biomarkers for memory destabilisation that would allow reminder sessions to be 'tuned' to enhance memory lability in an ad hoc manner.

Blockade of dopamine D3 receptor in ventral tegmental area attenuating contextual fear memory

The abnormal fear memory will lead to the onset of stress disorders, such as post-traumatic stress disorder (PTSD) and so on. Therefore, the intervention in the formation of abnormal fear memory will provide a new strategy for the prevention and treatment of PTSD. In our previous studies, we found that blockade of dopamine D3 receptor (DRD3) with highly selective antagonist YQA14 or knockout of DRD3 was able to attenuate the expression or retrieval of fear memory in PTSD animal models. However, the neurobiological mechanism of regulation of DRD3 in fear is unclear. In the present research, we clarified that DRD3 was expressed in the dopaminergic (DAergic) neurons in the ventral tegmental area (VTA). Then, we identified that microinjection of YQA14 (1 μg/0.2 μl/side) in VTA before the aversive stimuli in the training session or during days subsequent to the shock significantly meliorated the freezing behaviors in the inescapable electric foot-shock model. At last, using fiber photometry system, we found that microinjection of YQA14 in VTA promoted the dopamine neurotransmitter release in the basolateral amygdala (BLA), and pre-training YQA14 infusion in VTA lowered the increase of dopamine (DA) in BLA induced by shock during the training session or by context during the retrieval session. All above the results demonstrated that YQA14 attenuated the fear learning through the blockade of DRD3 in VTA decreasing the excitability of the projection to BLA. This study may provide new mechanisms and potential intervention targets for stress disorders with abnormal fear memory.

Estradiol and progesterone in female reward-learning, addiction, and therapeutic interventions

Sex steroid hormones like estradiol (E2) and progesterone (P4) guide the sexual organization and activation of the developing brain and control female reproductive behavior throughout the lifecycle; importantly, these hormones modulate functional activity of not just the endocrine system, but most of the nervous system including the brain reward system. The effects of E2 and P4 can be seen in the processing of and memory for rewarding stimuli and in the development of compulsive reward-seeking behaviors like those seen in substance use disorders. Women are at increased risk of developing substance use disorders; however, the origins of this sex difference are not well understood and therapeutic interventions targeting ovarian hormones have produced conflicting results. This article reviews the contribution of the E2 and P4 in females to functional modulation of the brain reward system, their possible roles in origins of addiction vulnerability, and the development and treatment of compulsive reward-seeking behaviors.