Dose-dependent effects of estrogen on prediction error related neural activity in the nucleus accumbens of healthy young women

Devaluing memories of reward: a case for dopamine

Midbrain dopamine cells encode differences in predictive and expected value to support learning through reward prediction error. Recent findings have questioned whether reward prediction error can fully account for dopamine function and suggest a more complex role for dopamine in encoding detailed features of the reward environment. In this series of studies, we describe a novel role for dopamine in devaluing sensory features of reward. Mesencephalic dopamine cells activated during a mediated devaluation phase were later chemogenetically reactivated. This retrieval of the devalued reward memory elicited a reduction in the hedonic evaluation of sucrose reward. Through optogenetic and chemogenetic manipulations, we confirm dopamine cells are both sufficient and necessary for mediated devaluation, and retrieval of these memories reflected dopamine release in the nucleus accumbens. Consistent with our computational modeling data, our findings indicate a critical role for dopamine in encoding predictive representations of the sensory features of reinforcement. Overall, we elucidate a novel role for dopamine function in mediated devaluation and illuminate a more elaborate framework through which dopamine encodes reinforcement signals.

Reward processes in extinction learning and applications to exposure therapy

Anxiety disorders are common and highly distressing mental health conditions. Exposure therapy is a gold-standard treatment for anxiety disorders. Mechanisms of Pavlovian fear learning, and particularly fear extinction, are central to exposure therapy. A growing body of evidence suggests an important role of reward processes during Pavlovian fear extinction. Nonetheless, predominant models of exposure therapy do not currently incorporate reward processes. Herein, we present a theoretical model of reward processes in relation to Pavlovian mechanisms of exposure therapy, including a focus on dopaminergic prediction error signaling, coinciding positive emotional experiences (i.e., relief), and unexpected positive outcomes. We then highlight avenues for further research and discuss potential strategies to leverage reward processes to maximize exposure therapy response, such as pre-exposure interventions to increase reward sensitivity or post-exposure rehearsal (e.g., savoring, imaginal recounting strategies) to enhance retrieval and retention of learned associations.

The effect of short-term increase of estradiol levels on sexual desire and orgasm frequency in women and men: A double-blind, randomized, placebo-controlled trial

Estradiol (E2) has been implicated in sexual functioning in both sexes. E2 levels change distinctively over the menstrual cycle, peaking around ovulation. Data on short-term effects of fluctuating E2 levels on sexual desire are however sparse and mostly based on observational studies. To fill this gap, we ran a double-blind, randomized, placebo-controlled study (N = 126) to investigate the effects of a short-term increase in E2 on sexual desire and orgasm frequency in healthy, young men and women. Circulating E2 levels were elevated through estradiol valerate (E2V) administered over two consecutive days to simulate the rise in E2 levels around ovulation. E2V had no effect on orgasm frequency and only minor effects on sexual desire. On average, the administered E2V dampened change in sexual desire compared to untreated participants with comparable baseline sexual desire in such a way that sexual desire was slightly reduced even in those with higher baseline sexual desire. These findings suggest that short-term increases in E2 have little effect on sexual function and are unlikely to explain the increase in sexual desire around ovulation.

Effects of 24-hour oral estradiol-valerate administration on hormone levels in men and pre-menopausal women

In order to translate the findings from the vast animal literature on the effect of 17β-estradiol (E2) on brain and behavior to humans, a placebo-controlled pharmacological enhancement of E2 levels for at least 24 h is necessary. However, an exogenous increase in E2 for such a prolonged period might affect the endogenous secretion of other (neuroactive) hormones. Such effects would be of relevance for the interpretation of the effects of this pharmacological regimen on cognition and its neural correlates as well as be of basic scientific interest. We therefore administered a double dose of 12 mg of estradiol-valerate (E2V) to men and of 8 mg to naturally cycling women in their low-hormone phase, and assessed the concentration of two steroids critical to hormone regulation: follicle stimulating hormone (FSH) and luteinizing hormone (LH). We also assessed any changes in concentration of the neuroactive hormones progesterone (P4), testosterone (TST), dihydrotestosterone (DHT) and immune-like growth factor 1 (IGF-1). This regimen resulted in similar E2 levels in both sexes (saliva and serum). FSH and LH levels in both sexes were down-regulated to the same degree. P4 concentration decreased in both sexes only in serum but not saliva. TST and DHT levels dropped only in men whereas sex-hormone binding globulin was not affected. Finally, the concentration of IGF-1 decreased in both sexes. Based on previous studies on the effects of these neuroactive hormones, only the degree of downregulation of TST and DHT levels in men might have an impact on brain and behavior, which should be considered when interpreting the effects of the presented E2V regimes.

Regulation of sex differences in risk-based decision making by gonadal hormones: Insights from rodent models

Men and women differ in their ability to evaluate options that vary in their rewards and the risks that are associated with these outcomes. Most studies have shown that women are more risk averse than men and that gonadal hormones significantly contribute to this sex difference. Gonadal hormones can influence risk-based decision making (i.e., risk taking) by modulating the neurobiological substrates underlying this cognitive process. Indeed, estradiol, progesterone and testosterone modulate activity in the prefrontal cortex, amygdala and nucleus accumbens associated with reward and risk-related information. The use of animal models of decision making has advanced our understanding of the intersection between the behavioral, neural and hormonal mechanisms underlying sex differences in risk taking. This review will outline the current state of this literature, identify the current gaps in knowledge and suggest the neurobiological mechanisms by which hormones regulate risky decision making. Collectively, this knowledge can be used to understand the potential consequences of significant hormonal changes, whether endogenously or exogenously induced, on risk-based decision making as well as the neuroendocrinological basis of neuropsychiatric diseases that are characterized by impaired risk taking, such as substance use disorder and schizophrenia.

Sex Differences and Exogenous Estrogen Influence Learning and Brain Responses to Prediction Errors

Animal studies show marked sex differences as well as effects of estrogen (E2) in the mesocorticolimbic dopaminergic (DA) pathways, which play a critical role in reward processing and reinforcement learning and are also implicated in drug addiction. In this computational pharmacological fMRI study, we investigate the effects of both factors, sex and estrogen, on reinforcement learning and the dopaminergic system in humans; 67 male and 64 naturally cycling female volunteers, the latter in their low-hormone phase, were randomly assigned, double-blind, to take E2 or placebo. They completed a reinforcement learning task in the MRI scanner for which we have previously shown reward prediction error (RPE)-related activity to be dopaminergic. We found RPE-related brain activity to be enhanced in women compared with men and to a greater extent when E2 levels were elevated in both sexes. However, both factors, female sex and E2, slowed adaptation to RPEs (smaller learning rate). This discrepancy of larger RPE-related activity yet smaller learning rates can be explained by organizational sex differences and activational effects of circulating E2, which both affect DA release differently to DA receptor binding capacities.

Estradiol administration modulates neural emotion regulation

Variations of sex hormones during the menstrual cycle can lead to changes in emotion processing. The ability to successfully regulate one's emotions is associated with better social abilities and mental health. While women show better performance in fear extinction learning under high estradiol (E2) compared to women under low E2 levels, little is known about the effect of E2 on emotion regulation. We explored whether E2 modulates emotion regulation in a functional magnetic resonance imaging paradigm and administered E2 valerate to 32 young naturally cycling women during their early follicular phase in a double-blind, placebo-controlled within-subject design. This standardized experimental control allowed us to explore the specific effect of E2 on emotion regulation while controlling for other hormones varying throughout the menstrual cycle. Behaviorally, no difference between conditions appeared. However, on the neural level, E2 administration was associated with lower activation in the right lingual- and left calcarine gyrus, right orbitofrontal cortex and left hippocampus relative to placebo. With respect to the main effect of down-regulation higher activation of the right superior frontal gyrus and left dorsomedial prefrontal cortex was seen; which is in accordance to previous literature. An interaction between drug condition and emotion regulation appeared for the left inferior frontal gyrus extending into the middle frontal gyrus indicating lower activation during down-regulation in the E2 condition than the placebo condition. On the behavioral level, women reported less negative affect in the E2 condition. The results fit well to a previously described psychoneuroendocrinological model in which E2 plays an important modulatory role on emotional processes and risk factors of mental health in women.

A causal role of estradiol in human reinforcement learning

The sex hormone estradiol is hypothesized to play a key role in human cognition, and reward processing specifically, via increased dopamine D1-receptor signalling. However, the effect of estradiol on reward processing in men has never been established. To fill this gap, we performed a double-blind placebo-controlled study in which men (N = 100) received either a single dose of estradiol (2 mg) or a placebo. Subjects performed a probabilistic reinforcement learning task where they had to choose between two options with varying reward probabilities to maximize monetary reward. Results showed that estradiol administration increased reward sensitivity compared to placebo. This effect was observed in subjects' choices, how much weight they assigned to their previous choices, and subjective reports about the reward probabilities. Furthermore, effects of estradiol were moderated by reward sensitivity, as measured through the BIS/BAS questionnaire. Using reinforcement learning models, we found that behavioral effects of estradiol were reflected in increased learning rates. These results demonstrate a causal role of estradiol within the framework of reinforcement learning, by enhancing reward sensitivity and learning. Furthermore, they provide preliminary evidence for dopamine-related genetic variants moderating the effect of estradiol on reward processing.