Distinct medial temporal networks encode surprise during motivation by reward versus punishment

More widespread and rigid neuronal representation of reward expectation underlies impulsive choices

Impulsive choices prioritize smaller, more immediate rewards over larger, delayed, or potentially uncertain rewards. Impulsive choices are a critical aspect of substance use disorders and maladaptive decision-making across the lifespan. Here, we sought to understand the neuronal underpinnings of expected reward and risk estimation on a trial-by-trial basis during impulsive choices. To do so, we acquired electrical recordings from the human brain while participants carried out a risky decision-making task designed to measure choice impulsivity. Behaviorally, we found a reward-accuracy tradeoff, whereby more impulsive choosers were more accurate at the task, opting for a more immediate reward while compromising overall task performance. We then examined how neuronal populations across frontal, temporal, and limbic brain regions parametrically encoded reinforcement learning model variables, namely reward and risk expectation and surprise, across trials. We found more widespread representations of reward value expectation and prediction error in more impulsive choosers, whereas less impulsive choosers preferentially represented risk expectation. A regional analysis of reward and risk encoding highlighted the anterior cingulate cortex for value expectation, the anterior insula for risk expectation and surprise, and distinct regional encoding between impulsivity groups. Beyond describing trial-by-trial population neuronal representations of reward and risk variables, these results suggest impaired inhibitory control and model-free learning underpinnings of impulsive choice. These findings shed light on neural processes underlying reinforced learning and decision-making in uncertain environments and how these processes may function in psychiatric disorders.

Electome network factors: Capturing emotional brain networks related to health and disease

Therapeutic development for mental disorders has been slow despite the high worldwide prevalence of illness. Unfortunately, cellular and circuit insights into disease etiology have largely failed to generalize across individuals that carry the same diagnosis, reflecting an unmet need to identify convergent mechanisms that would facilitate optimal treatment. Here, we discuss how mesoscale networks can encode affect and other cognitive processes. These networks can be discovered through electrical functional connectome (electome) analysis, a method built upon explainable machine learning models for analyzing and interpreting mesoscale brain-wide signals in a behavioral context. We also outline best practices for identifying these generalizable, interpretable, and biologically relevant networks. Looking forward, translational electome analysis can span species and various moods, cognitive processes, or other brain states, supporting translational medicine. Thus, we argue that electome analysis provides potential translational biomarkers for developing next-generation therapeutics that exhibit high efficacy across heterogeneous disorders.

The Emerging Therapeutic Potential of Kisspeptin and Neurokinin B

Kisspeptin (KP) and neurokinin B (NKB) are neuropeptides that govern the reproductive endocrine axis through regulating hypothalamic gonadotropin-releasing hormone (GnRH) neuronal activity and pulsatile GnRH secretion. Their critical role in reproductive health was first identified after inactivating variants in genes encoding for KP or NKB signaling were shown to result in congenital hypogonadotropic hypogonadism and a failure of pubertal development. Over the past 2 decades since their discovery, a wealth of evidence from both basic and translational research has laid the foundation for potential therapeutic applications. Beyond KP's function in the hypothalamus, it is also expressed in the placenta, liver, pancreas, adipose tissue, bone, and limbic regions, giving rise to several avenues of research for use in the diagnosis and treatment of pregnancy, metabolic, liver, bone, and behavioral disorders. The role played by NKB in stimulating the hypothalamic thermoregulatory center to mediate menopausal hot flashes has led to the development of medications that antagonize its action as a novel nonsteroidal therapeutic agent for this indication. Furthermore, the ability of NKB antagonism to partially suppress (but not abolish) the reproductive endocrine axis has supported its potential use for the treatment of various reproductive disorders including polycystic ovary syndrome, uterine fibroids, and endometriosis. This review will provide a comprehensive up-to-date overview of the preclinical and clinical data that have paved the way for the development of diagnostic and therapeutic applications of KP and NKB.

Long-term, multi-event surprise correlates with enhanced autobiographical memory

Neurobiological and psychological models of learning emphasize the importance of prediction errors (surprises) for memory formation. This relationship has been shown for individual momentary surprising events; however, it is less clear whether surprise that unfolds across multiple events and timescales is also linked with better memory of those events. We asked basketball fans about their most positive and negative autobiographical memories of individual plays, games and seasons, allowing surprise measurements spanning seconds, hours and months. We used advanced analytics on National Basketball Association play-by-play data and betting odds spanning 17 seasons, more than 22,000 games and more than 5.6 million plays to compute and align the estimated surprise value of each memory. We found that surprising events were associated with better recall of positive memories on the scale of seconds and months and negative memories across all three timescales. Game and season memories could not be explained by surprise at shorter timescales, suggesting that long-term, multi-event surprise correlates with memory. These results expand notions of surprise in models of learning and reinforce its relevance in real-world domains.

Contributions of transient and sustained reward to memory formation

Reward benefits to memory formation have been robustly linked to dopaminergic activity. Despite the established characterization of dopaminergic mechanisms as operating at multiple timescales, potentially supporting distinct functional outcomes, the temporal dynamics by which reward might modulate memory encoding are just beginning to be investigated. In the present study, we leveraged a mixed block/event experimental design to disentangle transient and sustained reward influences on task engagement and subsequent recognition memory in an adapted monetary-incentive-encoding (MIE) paradigm. Across three behavioral experiments, transient and sustained reward modulation of item and context memory was probed, at both 24-h and ~ 15-min retention intervals, to investigate the importance of overnight consolidation. In general, we observed that transient reward was associated with enhanced item memory encoding, while sustained reward modulated response speed but did not appear to benefit subsequent recognition accuracy. Notably, reward effects on item memory performance and response speed were somewhat inconsistent across the three experiments, with suggestions that RT speeding might also be related to time on task, and we did not observe reward modulation of context memory performance or amplification of reward benefits to memory by overnight consolidation. Taken together, the observed pattern of behavior is consistent with potentially distinct roles for transient and sustained reward in memory encoding and cognitive performance and suggests that further investigation of the temporal dynamics of dopaminergic contributions to memory formation will advance the understanding of motivated memory.

Switching between External and Internal Attention in Hippocampal Networks

Everyday experience requires processing external signals from the world around us and internal information retrieved from memory. To do both, the brain must fluctuate between states that are optimized for external versus internal attention. Here, we focus on the hippocampus as a region that may serve at the interface between these forms of attention and ask how it switches between prioritizing sensory signals from the external world versus internal signals related to memories and thoughts. Pharmacological, computational, and animal studies have identified input from the cholinergic basal forebrain as important for biasing the hippocampus toward processing external information, whereas complementary research suggests the dorsal attention network (DAN) may aid in allocating attentional resources toward accessing internal information. We therefore tested the hypothesis that the basal forebrain and DAN drive the hippocampus toward external and internal attention, respectively. We used data from 29 human participants (17 female) who completed two attention tasks during fMRI. One task (memory-guided) required proportionally more internal attention, and proportionally less external attention, than the other (explicitly instructed). We discovered that background functional connectivity between the basal forebrain and hippocampus was stronger during the explicitly instructed versus memory-guided task. In contrast, DAN-hippocampus background connectivity was stronger during the memory-guided versus explicitly instructed task. Finally, the strength of DAN-hippocampus background connectivity was correlated with performance on the memory-guided but not explicitly instructed task. Together, these results provide evidence that the basal forebrain and DAN may modulate the hippocampus to switch between external and internal attention.SIGNIFICANCE STATEMENT How does the brain balance the need to pay attention to internal thoughts and external sensations? We focused on the human hippocampus, a region that may serve at the interface between internal and external attention, and asked how its functional connectivity varies based on attentional states. The hippocampus was more strongly coupled with the cholinergic basal forebrain when attentional states were guided by the external world rather than retrieved memories. This pattern flipped for functional connectivity between the hippocampus and dorsal attention network, which was higher for attention tasks that were guided by memory rather than external cues. Together, these findings show that distinct networks in the brain may modulate the hippocampus to switch between external and internal attention.

Instructed motivational states bias reinforcement learning and memory formation

Motivation influences goals, decisions, and memory formation. Imperative motivation links urgent goals to actions, narrowing the focus of attention and memory. Conversely, interrogative motivation integrates goals over time and space, supporting rich memory encoding for flexible future use. We manipulated motivational states via cover stories for a reinforcement learning task: The imperative group imagined executing a museum heist, whereas the interrogative group imagined planning a future heist. Participants repeatedly chose among four doors, representing different museum rooms, to sample trial-unique paintings with variable rewards (later converted to bonus payments). The next day, participants performed a surprise memory test. Crucially, only the cover stories differed between the imperative and interrogative groups; the reinforcement learning task was identical, and all participants had the same expectations about how and when bonus payments would be awarded. In an initial sample and a preregistered replication, we demonstrated that imperative motivation increased exploitation during reinforcement learning. Conversely, interrogative motivation increased directed (but not random) exploration, despite the cost to participants' earnings. At test, the interrogative group was more accurate at recognizing paintings and recalling associated values. In the interrogative group, higher value paintings were more likely to be remembered; imperative motivation disrupted this effect of reward modulating memory. Overall, we demonstrate that a prelearning motivational manipulation can bias learning and memory, bearing implications for education, behavior change, clinical interventions, and communication.

A meta-analysis of the neural substrates of monetary reward anticipation and outcome in alcohol use disorder

The capacity to anticipate and detect rewarding outcomes is fundamental for the development of adaptive decision-making and goal-oriented behavior. Delineating the neural correlates of different stages of reward processing is imperative for understanding the neurobiological mechanism underlying alcohol use disorder (AUD). To examine the neural correlates of monetary anticipation and outcome in AUD patients, we performed two separate voxel-wise meta-analyses of functional neuroimaging studies, including 12 studies investigating reward anticipation and 7 studies investigating reward outcome using the monetary incentive delay task. During the anticipation stage, AUD patients displayed decreased activation in response to monetary cues in mesocortical-limbic circuits and sensory areas, including the ventral striatum (VS), insula, hippocampus, inferior occipital gyrus, supramarginal gyrus, lingual gyrus and fusiform gyrus. During the outcome stage, AUD patients exhibited reduced activation in the dorsal striatum, VS and insula, and increased activation in the orbital frontal cortex and medial temporal area. Our findings suggest that different activation patterns are associated with nondrug rewards during different reward processing stages, potentially reflecting a changed sensitivity to monetary reward in AUD.

Reward, motivation and brain imaging in human healthy participants - A narrative review

Over the past 20 years there has been an increasing number of brain imaging studies on the mechanisms underlying reward motivation in humans. This narrative review describes studies on the neural mechanisms associated with reward motivation and their relationships with cognitive function in healthy human participants. The brain's meso-limbic dopamine reward circuitry in humans is known to control reward-motivated behavior in humans. The medial and lateral Pre-Frontal Cortex (PFC) integrate motivation and cognitive control during decision-making and the dorsolateral PFC (dlPFC) integrates and transmits signals of reward to the mesolimbic and meso-cortical dopamine circuits and initiates motivated behavior. The thalamus and insula influence incentive processing in humans and the motor system plays a role in response to action control. There are reciprocal relationships between reward motivation, learning, memory, imagery, working memory, and attention. The most common method of assessing reward motivation is the monetary incentive delay task (DMRT) and there are several meta-analyses of this paradigm. Genetics modulates motivation reward, and dopamine provides the basis for the interaction between motivational and cognitive control. There is some evidence that male adolescents take more risky decisions than female adolescents and that the lateralization of reward-related DA release in the ventral striatum is confined to men. These studies have implications for our understanding of natural reward and psychiatric conditions like addiction, depression and ADHD. Furthermore, the association between reward and memory can help develop treatment techniques for drug addiction that interfere with consolidation of memory. Finally, there is a lack of research on reward motivation, genetics and sex differences and this can improve our understanding of the relationships between reward, motivation and the brain.

Reward Uncertainty and Expected Value Enhance Generalization of Episodic Memory

Previous research has revealed some mechanisms underlying the generalization of reward expectation of generalization stimuli, but little is known about the generalization of episodic memory for rewarding events, its consolidation, and how reward components such as expected value and reward uncertainty affect it. Participants underwent a Pavlovian reward-conditioning task to test whether reward conditioning would enhance episodic memory generalization and which reward components would directly affect it. Counterbalanced across participants, one semantic category was paired with a reward, while the other was never paired. Following a delay of either 5 min or 24 h, participants took a memory test consisting of old, highly similar, and new items. We found that participants were more likely to falsely recognize lure items as old in the reward-paired category after 5 min and 24 h delays. These results indicate that reward conditioning enhanced the generalization of episodic memory, but this effect was not necessarily dependent on consolidation. The composite score and raw data of generalization further showed that the uncertainty and expected value enhanced generalization. Together, these findings revealed an effect of reward conditioning on episodic memory generalization and supported the enhancement effects of expected value and uncertainty.

The role of discriminability in face perception: Interference processing of expression, gender, and gaze

Eye gaze plays a fundamental role in social interaction and facial recognition. However, interference processing between gaze and other facial variants (e.g., expression) and invariant information (e.g., gender) remains controversial and unclear, especially the role of facial information discriminability in interference. A Garner paradigm was used to conduct two experiments. This paradigm allows simultaneous investigation of the mutual influence of two kinds of facial information in one experiment. In Experiment 1, we manipulated facial expression discriminability and investigated its role in interference processing of gaze and facial expression. The results show that individuals were unable to ignore expression when classifying gaze with both high and low discriminability but could ignore gaze when classifying expression with high discriminability only. In Experiment 2, we manipulated gender discriminability and investigated its function in interference processing of gaze and gender. Participants were unable to ignore gender when classifying gaze with both high and low discriminability but could ignore gaze when classifying gender with low discriminability only. The results indicate that gaze categorization is affected by facial expression and gender regardless of facial information discriminability, whereas interference of gaze on facial expression and gender depends on the degree of discriminability. The present study provides evidence that the processing of gaze and other variant and invariant information is interdependent.

Effect of reward and punishment on no-risk decision-making in young men: an EEG study

Decision-making is a process that allows adapting behavior in response to feedback to achieve a goal. Previous studies have suggested that the cerebral cortex shows different activation patterns in response to feedback. However, the effects of reward and punishment on learning contexts and decision-making are not clear. Thus, this experiment compared the effects of reward and punishment on behavior and the electroencephalographic activity of cortical areas related to decision-making in a no-risk context. Twenty healthy males were asked to perform a decision-making task under two conditions in which the goal was to finish in the shortest time possible. In the reward condition, the more points the participant accumulated the sooner the task ended, while in the punishment condition, the more points accumulated the longer the task lasted. Lower reaction times were found in the reward condition, characterized by a higher absolute power of the slow bands in almost all the cortices recorded. Changes in the interhemispheric correlation were also obtained in the comparison of the two feedback conditions. Results suggest that changes in the type of feedback affect cortical functionality and behavioral execution during decision-making, with the reward being related to a quick emotional response strategy and punishment associated with slower and, likely, more reasoned responses.

Hypothalamic Functional Connectivity and Apathy in People with Alzheimer's Disease and Cognitively Normal Healthy Controls

BACKGROUND

Earlier studies have described the neural markers of apathy in Alzheimer's disease (AD) and mild cognitive impairment (MCI), but few focused on the motivation circuits. Here, we targeted hypothalamus, a hub of the motivation circuit.

OBJECTIVE

To examine hypothalamic resting state functional connectivity (rsFC) in relation to apathy.

METHODS

We performed whole-brain regression of hypothalamic rsFC against Apathy Evaluation Scale (AES) total score and behavioral, cognitive, and emotional subscores in 29 patients with AD/MCI and 28 healthy controls (HC), controlling for age, sex, education, cognitive status, and depression. We evaluated the results at a corrected threshold and employed path analyses to assess possible interaction between hypothalamic rsFCs, apathy and depression/memory. Finally, we re-examined the findings in a subsample of amyloid-β-verified AD.

RESULTS

AES total score correlated negatively with hypothalamic precuneus (PCu)/posterior cingulate cortex (PCC) and positively with left middle temporal gyrus (MTG) and supramarginal gyrus rsFCs. Behavioral subscore correlated negatively with hypothalamic PCu/PCC and positively with middle frontal gyrus rsFC. Cognitive subscore correlated positively with hypothalamic MTG rsFC. Emotional subscore correlated negatively with hypothalamic calcarine cortex rsFC. In path analyses, hypothalamic-PCu/PCC rsFC negatively modulated apathy and, in turn, depression. The model where hypothalamic MTG rsFC and memory independently modulated apathy also showed a good fit. The findings of diminished hypothalamic-PCu/PCC rsFC in relation to apathy and, in turn, depression were confirmed in amyloid-verified AD.

CONCLUSION

The findings together support a role of altered hypothalamic connectivity in relation to apathy and depression, and modulation of apathy by memory dysfunction.

Current Perspectives on Kisspeptins Role in Behaviour

The neuropeptide kisspeptin is now well-established as the master regulator of the mammalian reproductive axis. Beyond the hypothalamus, kisspeptin and its cognate receptor are also extensively distributed in extra-hypothalamic brain regions. An expanding pool of animal and human data demonstrates that kisspeptin sits within an extensive neuroanatomical and functional framework through which it can integrate a range of internal and external cues with appropriate neuroendocrine and behavioural responses. In keeping with this, recent studies reveal wide-reaching effects of kisspeptin on key behaviours such as olfactory-mediated partner preference, sexual motivation, copulatory behaviour, bonding, mood, and emotions. In this review, we provide a comprehensive update on the current animal and human literature highlighting the far-reaching behaviour and mood-altering roles of kisspeptin. A comprehensive understanding of this important area in kisspeptin biology is key to the escalating development of kisspeptin-based therapies for common reproductive and related psychological and psychosexual disorders.

In for a penny, in for a pound: examining motivated memory through the lens of retrieved context models

When people encounter items that they believe will help them gain reward, they later remember them better than others. A recent model of emotional memory, the emotional context maintenance and retrieval model (eCMR), predicts that these effects would be stronger when stimuli that predict high and low reward can compete with each other during both encoding and retrieval. We tested this prediction in two experiments. Participants were promised £1 for remembering some pictures, but only a few pence for remembering others. Their recall of the content of the pictures they saw was tested after 1 min and, in experiment 2, also after 24 h. Memory at the immediate test showed effects of list composition. Recall of stimuli that predicted high reward was greater than of stimuli that predicted lower reward, but only when high- and low-reward items were studied and recalled together, not when they were studied and recalled separately. More high-reward items in mixed lists were forgotten over a 24-h retention interval compared with items studied in other conditions, but reward did not modulate the forgetting rate, a null effect that should be replicated in a larger sample. These results confirm eCMR's predictions, although further research is required to compare that model against alternatives.

Mnemonic prediction errors promote detailed memories

When our experience violates our predictions, it is adaptive to update our knowledge to promote a more accurate representation of the world and facilitate future predictions. Theoretical models propose that these mnemonic prediction errors should be encoded into a distinct memory trace to prevent interference with previous, conflicting memories. We investigated this proposal by repeatedly exposing participants to pairs of sequentially presented objects (A → B), thus evoking expectations. Then, we violated participants' expectations by replacing the second object in the pairs with a novel object (A → C). The following item memory test required participants to discriminate between identical old items and similar lures, thus testing detailed and distinctive item memory representations. In two experiments, mnemonic prediction errors enhanced item memory: Participants correctly identified more old items as old when those items violated expectations during learning, compared with items that did not violate expectations. This memory enhancement for C items was only observed when participants later showed intact memory for the related A → B pairs, suggesting that strong predictions are required to facilitate memory for violations. Following up on this, a third experiment reduced prediction strength prior to violation and subsequently eliminated the memory advantage of violations. Interestingly, mnemonic prediction errors did not increase gist-based mistakes of identifying old items as similar lures or identifying similar lures as old. Enhanced item memory in the absence of gist-based mistakes suggests that violations enhanced memory for items' details, which could be mediated via distinct memory traces. Together, these results advance our knowledge of how mnemonic prediction errors promote memory formation.

Hypoactivation in the precuneus and posterior cingulate cortex during ambiguous decision making in individuals with HIV

People with human immunodeficiency virus (HIV) often have neurocognitive impairment. People with HIV make riskier decisions when the outcome probabilities are known, and have abnormal neural architecture underlying risky decision making. However, ambiguous decision making, when the outcome probabilities are unknown, is more common in daily life, but the neural architecture underlying ambiguous decision making in people with HIV is unknown. Eighteen people with HIV and 20 controls completed a decision making task while undergoing functional magnetic resonance imaging scanning. Participants chose between a certain reward and uncertain reward with a known (risky) or unknown (ambiguous) probability of winning. There were three levels of risk: high, medium, and low. Ambiguous > risky brain activity was compared between groups. Ambiguous > risky brain activity was correlated with emotional/psychiatric functioning in people with HIV. Both groups were similarly ambiguity-averse. People with HIV were more risk-averse than controls and chose the high-risk uncertain option less often. People with HIV had hypoactivity in the precuneus, posterior cingulate cortex (PCC), and fusiform gyrus during ambiguous > medium risk decision making. Ambiguous > medium risk brain activity was negatively correlated with emotional/psychiatric functioning in individuals with HIV. To make ambiguous decisions, people with HIV underrecruit key regions of the default mode network, which are thought to integrate internally and externally derived information to come to a decision. These regions and related cognitive processes may be candidates for interventions to improve decision-making outcomes in people with HIV.

Signed and unsigned reward prediction errors dynamically enhance learning and memory

Memory helps guide behavior, but which experiences from the past are prioritized? Classic models of learning posit that events associated with unpredictable outcomes as well as, paradoxically, predictable outcomes, deploy more attention and learning for those events. Here, we test reinforcement learning and subsequent memory for those events, and treat signed and unsigned reward prediction errors (RPEs), experienced at the reward-predictive cue or reward outcome, as drivers of these two seemingly contradictory signals. By fitting reinforcement learning models to behavior, we find that both RPEs contribute to learning by modulating a dynamically changing learning rate. We further characterize the effects of these RPE signals on memory and show that both signed and unsigned RPEs enhance memory, in line with midbrain dopamine and locus-coeruleus modulation of hippocampal plasticity, thereby reconciling separate findings in the literature.

Dopamine agonist treatment increases sensitivity to gamble outcomes in the hippocampus in de novo Parkinson's disease

BACKGROUND

Parkinson's disease is associated with severe nigro-striatal dopamine depletion, leading to motor dysfunction and altered reward processing. We previously showed that drug-naïve patients with Parkinson's disease had a consistent attenuation of reward signalling in the mesolimbic and mesocortical system. Here, we address the neurobiological effects of dopaminergic therapy on reward sensitivity in the mesolimbic circuitry, and how this may contribute to neuropsychiatric symptoms.

OBJECTIVES

We tested the hypothesis that (1) dopaminergic treatment would restore the attenuated, mesolimbic and mesocortical responses to reward; and (2) restoration of reward responsivity by dopaminergic treatment would predict motor performance and the emergence of impulse control symptoms.

METHODS

In 11 drug-naïve Parkinson patients, we prospectively assessed treatment-induced changes in reward processing before, and eight weeks after initiation of monotherapy with dopamine agonists. They were compared to 10 non-medicated healthy controls who were also measured longitudinally. We used whole-brain functional magnetic resonance imaging at 3 Tesla to assess the reward responsivity of the brain to monetary gains and losses, while participants performed a simple consequential gambling task.

RESULTS

In patients, dopaminergic treatment improved clinical motor symptoms without significantly changing task performance. Dopamine agonist therapy induced a stronger reward responsivity in the right hippocampus with higher doses being less effective. None of the patients developed impulse control disorders in the follow-up period of four years.

CONCLUSIONS

Short-term treatment with first-ever dopaminergic medication partially restores deficient reward-related processing in the hippocampus in de novo Parkinson's disease.

Functional connectivity between memory and reward centers across task and rest track memory sensitivity to reward

External motivation, such as a promise of future monetary reward for remembering an event, can affect which events are remembered. Reward-based memory modulation is thought to result from encoding and post-encoding interactions between dopaminergic midbrain, signaling reward, and hippocampus and parahippocampal cortex, supporting episodic memory. We asked whether hippocampal and parahippocampal interactions with other reward-related regions are related to reward modulation of memory and whether such relationships are stable over time. Individuals' memory sensitivity to reward was measured using a monetary incentive encoding task in which a cue indicated potential monetary reward (penny, dime, or dollar) for remembering an upcoming object pair. Functional connectivity between memory and reward regions was measured before, during, and following the task. Reward-related regions of interest were generated using a meta-analysis of existing studies on reward and included ventral striatum, medial and orbital prefrontal cortices and anterior cingulate cortex, in addition to midbrain. The results showed that connectivity between memory and reward regions tracked individual differences in reward modulation of memory, irrespective of when connectivity was measured. Connectivity patterns of anterior cingulate, orbitofrontal cortex, and ventral striatum covaried together and tracked behavior most strongly. These findings implicate a broader set of reward regions in reward modulation of memory than considered previously and provide new evidence that stable connectivity patterns between memory and reward centers relate to individual differences in how reward impacts memory.

Hippocampal contributions to value-based learning: Converging evidence from fMRI and amnesia

Recent evidence suggests that the human hippocampus-known primarily for its involvement in episodic memory-plays a role in a host of motivationally relevant behaviors, including some forms of value-based decision-making. However, less is known about the role of the hippocampus in value-based learning. Such learning is typically associated with a striatal system, yet a small number of studies, both in human and nonhuman species, suggest hippocampal engagement. It is not clear, however, whether this engagement is necessary for such learning. In the present study, we used both functional MRI (fMRI) and lesion-based neuropsychological methods to clarify hippocampal contributions to value-based learning. In Experiment 1, healthy participants were scanned while learning value-based contingencies (whether players in a "game" win money) in the context of a probabilistic learning task. Here, we observed recruitment of the hippocampus, in addition to the expected ventral striatal (nucleus accumbens) activation that typically accompanies such learning. In Experiment 2, we administered this task to amnesic patients with medial temporal lobe damage and to healthy controls. Amnesic patients, including those with damage circumscribed to the hippocampus, failed to acquire value-based contingencies, thus confirming that hippocampal engagement is necessary for task performance. Control experiments established that this impairment was not due to perceptual demands or memory load. Future research is needed to clarify the mechanisms by which the hippocampus contributes to value-based learning, but these findings point to a broader role for the hippocampus in goal-directed behaviors than previously appreciated.

Reward prediction errors create event boundaries in memory

We remember when things change. Particularly salient are experiences where there is a change in rewards, eliciting reward prediction errors (RPEs). How do RPEs influence our memory of those experiences? One idea is that this signal directly enhances the encoding of memory. Another, not mutually exclusive, idea is that the RPE signals a deeper change in the environment, leading to the mnemonic separation of subsequent experiences from what came before, thereby creating a new latent context and a more separate memory trace. We tested this in four experiments where participants learned to predict rewards associated with a series of trial-unique images. High-magnitude RPEs indicated a change in the underlying distribution of rewards. To test whether these large RPEs created a new latent context, we first assessed recognition priming for sequential pairs that included a high-RPE event or not (Exp. 1: n = 27 & Exp. 2: n = 83). We found evidence of recognition priming for the high-RPE event, indicating that the high-RPE event is bound to its predecessor in memory. Given that high-RPE events are themselves preferentially remembered (Rouhani, Norman, & Niv, 2018), we next tested whether there was an event boundary across a high-RPE event (i.e., excluding the high-RPE event itself; Exp. 3: n = 85). Here, sequential pairs across a high RPE no longer showed recognition priming whereas pairs within the same latent reward state did, providing initial evidence for an RPE-modulated event boundary. We then investigated whether RPE event boundaries disrupt temporal memory by asking participants to order and estimate the distance between two events that had either included a high-RPE event between them or not (Exp. 4). We found (n = 49) and replicated (n = 77) worse sequence memory for events across a high RPE. In line with our recognition priming results, we did not find sequence memory to be impaired between the high-RPE event and its predecessor, but instead found worse sequence memory for pairs across a high-RPE event. Moreover, greater distance between events at encoding led to better sequence memory for events across a low-RPE event, but not a high-RPE event, suggesting separate mechanisms for the temporal ordering of events within versus across a latent reward context. Altogether, these findings demonstrate that high-RPE events are both more strongly encoded, show intact links with their predecessor, and act as event boundaries that interrupt the sequential integration of events. We captured these effects in a variant of the Context Maintenance and Retrieval model (CMR; Polyn, Norman, & Kahana, 2009), modified to incorporate RPEs into the encoding process.

Kisspeptin enhances brain responses to olfactory and visual cues of attraction in men

Successful reproduction is a fundamental physiological process that relies on the integration of sensory cues of attraction with appropriate emotions and behaviors and the reproductive axis. However, the factors responsible for this integration remain largely unexplored. Using functional neuroimaging, hormonal, and psychometric analyses, we demonstrate that the reproductive hormone kisspeptin enhances brain activity in response to olfactory and visual cues of attraction in men. Furthermore, the brain regions enhanced by kisspeptin correspond to areas within the olfactory and limbic systems that govern sexual behavior and perception of beauty as well as overlap with its endogenous expression pattern. Of key functional and behavioral significance, we observed that kisspeptin was most effective in men with lower sexual quality-of-life scores. As such, our results reveal a previously undescribed attraction pathway in humans activated by kisspeptin and identify kisspeptin signaling as a new therapeutic target for related reproductive and psychosexual disorders.

Kisspeptin enhances brain processing in response to olfactory and visual cues of attraction and is most effective in men with lower sexual quality of life.

Differential impact of reward and punishment on functional connectivity after skill learning

Reward and punishment shape behavior, but the mechanisms underlying their effect on skill learning are not well understood. Here, we tested whether the functional connectivity of premotor cortex (PMC), a region known to be critical for learning of sequencing skills, is altered after training when reward or punishment is given during training. Resting-state fMRI was collected in two experiments before and after participants trained on either a serial reaction time task (SRTT; n = 36) or force-tracking task (FTT; n = 36) with reward, punishment, or control feedback. In each experiment, training-related change in PMC functional connectivity was compared across feedback groups. In both tasks, we found that reward and punishment differentially affected PMC functional connectivity. On the SRTT, participants trained with reward showed an increase in functional connectivity between PMC and cerebellum as well as PMC and striatum, while participants trained with punishment showed an increase in functional connectivity between PMC and medial temporal lobe connectivity. After training on the FTT, subjects trained with control and reward showed increases in PMC connectivity with parietal and temporal cortices after training, while subjects trained with punishment showed increased PMC connectivity with ventral striatum. While the results from the two experiments overlapped in some areas, including ventral pallidum, temporal lobe, and cerebellum, these regions showed diverging patterns of results across the two tasks for the different feedback conditions. These findings suggest that reward and punishment strongly influence spontaneous brain activity after training, and that the regions implicated depend on the task learned.

Abstract Representation of Prospective Reward in the Hippocampus

Motivation enhances memory by increasing hippocampal engagement during encoding. However, whether such increased hippocampal activation reflects encoding of the value of highly rewarding events per se is less understood. Here, using a monetary incentive encoding task with a novel manipulation, we tested in humans whether the hippocampus represents abstract reward value, independent of perceptual content. During functional MRI scanning, men and women studied object pairs, each preceded by a monetary reward cue indicating the amount of money they would receive if they successfully remembered the object pair at test. Reward cues varied on both the level of reward (penny, dime, and dollar) and visual form (picture or word) across trials to dissociate hippocampal responses to reward value from those reflecting the perceptual properties of the cues. Behaviorally, participants remembered pairs associated with the high reward (dollar) more often than pairs associated with lower rewards. Neural pattern-similarity analysis revealed that hippocampal and parahippocampal cortex activation patterns discriminated between cues of different value regardless of their visual form, and that hippocampal discrimination of value was most pronounced in participants who showed the greatest behavioral sensitivity to reward. Strikingly, hippocampal patterns were most distinct for reward cues that differed in value but had similar visual appearance, consistent with theoretical proposals of hippocampal-pattern differentiation of competing representations. Our data illustrate how contextual representations within the hippocampus go beyond space and time to include information about the motivational salience of events, with hippocampal reward coding tracking the motivational impact on later memory.SIGNIFICANCE STATEMENT Motivation, such as the promise of future rewards, enhances hippocampal engagement during encoding and promotes successful retention of events associated with valuable rewards. However, whether the hippocampus explicitly encodes reward value, dissociable from sensory information, is unclear. Here, we show that the hippocampus forms abstract representation of valuable rewards, encoding conceptual rather than perceptual information about the motivational context of individual events. Reward representation within the hippocampus is associated with preferential retention of high-value events in memory. Furthermore, we show that hippocampal-pattern differentiation serves to emphasize differences between visually similar events with distinct motivational salience. Collectively, these findings indicate that hippocampal contextual representations enable individuals to distinguish the motivational value of events, leading to prioritized encoding of significant memories.

Neural Correlates of Outcome Anticipation in Multiple Sclerosis

Outcome anticipation is not only a mental preparation for upcoming consequences, but also an essential component of learning and decision-making. Thus, anticipation of consequences is a key process in everyday functioning. The striatum and the ventromedial prefrontal cortex are among the key regions that have been shown to be involved in outcome anticipation. However, while structural abnormalities of these regions as well as altered decision-making have been noted in individuals with multiple sclerosis (MS), neural correlates of outcome anticipation have not been explored in this population. Thus, we examined the neural correlates of outcome anticipation in MS by analyzing brain activation in individuals with MS while they performed a modified version of a card-guessing task. Seventeen MS and 13 healthy controls performed the task while functional magnetic resonance imaging (fMRI) was obtained. To achieve maximal anticipatory response and prevent the possibility of differential performance on the task, participants were presented with monetary rewards only on 50% of the trials. While replicating previous evidence of structural abnormalities of the striatum in MS, our results further showed that individuals with MS exhibited greater activation in the putamen, right hippocampus, and posterior cingulate cortex during outcome anticipation compared to healthy controls. Furthermore, even though there was no strategy that participants could learn in order to predict outcomes, 76% of participants with MS indicated that they used strategies while performing the task. We thus propose that the increased neural activation observed in MS during outcome anticipation might be explained by a failure in recognizing the lack of regularity in the task structure that could result in using strategies to perform the task.

Manipulating memory efficacy affects the behavioral and neural profiles of deterministic learning and decision-making

When making a decision, we have to identify, collect, and evaluate relevant bits of information to ensure an optimal outcome. How we approach a given choice can be influenced by prior experience. Contextual factors and structural elements of these past decisions can cause a shift in how information is encoded and can in turn influence later decision-making. In this two-experiment study, we sought to manipulate declarative memory efficacy and decision-making in a concurrent discrimination learning task by altering the amount of information to be learned. Subjects learned correct responses to pairs of items across several repetitions of a 50- or 100-pair set and were tested for memory retention. In one experiment, this memory test interrupted learning after an initial encoding experience in order to test for early encoding differences and associate those differences with changes in decision-making. In a second experiment, we used fMRI to probe neural differences between the two list-length groups related to decision-making across learning and assessed subsequent memory retention. We found that a striatum-based system was associated with decision-making patterns when learning a longer list of items, while a medial cortical network was associated with patterns when learning a shorter list. Additionally, the hippocampus was exclusively active for the shorter list group. Altogether, these behavioral, computational, and imaging results provide evidence that multiple types of mnemonic representations contribute to experienced-based decision-making. Moreover, contextual and structural factors of the task and of prior decisions can influence what types of evidence are drawn upon during decision-making.

Context and Time Matter: Effects of Emotion and Motivation on Episodic Memory Overtime

Previous studies have shown that compared with neutral cues, stimuli with positive and negative/stressful contexts or reward and punishment cues are remembered better. However, it is unclear whether the enhanced effect differs in emotion or motivation dimensions and the passage of time. We addressed these issues by manipulating different contextual cues for neutral words at different time intervals. In experiment 1, subjects were asked to learn words with picture contexts in positive, negative/stressful, and neutral valences and were tested by old/new word recognition and contextual judgment 10 min, 1 day, and 1 week later. In experiment 2, the reward and punishment motivations were manipulated by monetary cues during learning. Word recognition and contextual judgment were assessed 10 min, 1 day, 1 week, and 1 month after the study. Compared with negative and punishment conditions, the words in positive and reward contexts were recognized better at shorter intervals, which was associated with recollection process. In contrast, the words in negative and punishment contexts were recognized better at longer intervals, which was mainly associated with familiarity process. These results clarified how different dimensions of emotional and motivational contexts influence memory at short and long intervals and highlighted the role of contextual features in memory formation and consolidation.

Motivational valence alters memory formation without altering exploration of a real-life spatial environment

Volitional exploration and learning are key to adaptive behavior, yet their characterization remains a complex problem for cognitive science. Exploration has been posited as a mechanism by which motivation promotes memory, but this relationship is not well-understood, in part because novel stimuli that motivate exploration also reliably elicit changes in neuromodulatory brain systems that directly alter memory formation, via effects on neural plasticity. To deconfound interrelationships between motivation, exploration, and memory formation we manipulated motivational state prior to entering a spatial context, measured exploratory responses to the context and novel stimuli within it, and then examined motivation and exploration as predictors of memory outcomes. To elicit spontaneous exploration, we used the physical space of an art exhibit with affectively rich content; we expected motivated exploration and memory to reflect multiple factors, including not only motivational valence, but also individual differences. Motivation was manipulated via an introductory statement framing exhibit themes in terms of Promotion- or Prevention-oriented goals. Participants explored the exhibit while being tracked by video. They returned 24 hours later for recall and spatial memory tests, followed by measures of motivation, personality, and relevant attitude variables. Promotion and Prevention condition participants did not differ in terms of group-level exploration time or memory metrics, suggesting similar motivation to explore under both framing contexts. However, exploratory behavior and memory outcomes were significantly more closely related under Promotion than Prevention, indicating that Prevention framing disrupted expected depth-of-encoding effects. Additionally, while trait measures predicted exploration similarly across framing conditions, traits interacted with motivational framing context and facial affect to predict memory outcomes. This novel characterization of motivated learning implies that dissociable behavioral and biological mechanisms, here varying as a function of valence, contribute to memory outcomes in complex, real-life environments.

Parsing the Role of the Hippocampus in Approach-Avoidance Conflict

The hippocampus plays a central role in the approach-avoidance conflict that is central to the genesis of anxiety. However, its exact functional contribution has yet to be identified. We designed a novel gambling task that generated approach-avoidance conflict while controlling for spatial processing. We fit subjects' behavior using a model that quantified the subjective values of choice options, and recorded neural signals using functional magnetic resonance imaging (fMRI). Distinct functional signals were observed in anterior hippocampus, with inferior hippocampus selectively recruited when subjects rejected a gamble, to a degree that covaried with individual differences in anxiety. The superior anterior hippocampus, in contrast, uniquely demonstrated value signals that were potentiated in the context of approach-avoidance conflict. These results implicate the anterior hippocampus in behavioral avoidance and choice monitoring, in a manner relevant to understanding its role in anxiety. Our findings highlight interactions between subregions of the hippocampus as an important focus for future study.

Locus Coeruleus Activity Strengthens Prioritized Memories Under Arousal

Recent models posit that bursts of locus ceruleus (LC) activity amplify neural gain such that limited attention and encoding resources focus even more on prioritized mental representations under arousal. Here, we tested this hypothesis in human males and females using fMRI, neuromelanin MRI, and pupil dilation, a biomarker of arousal and LC activity. During scanning, participants performed a monetary incentive encoding task in which threat of punishment motivated them to prioritize encoding of scene images over superimposed objects. Threat of punishment elicited arousal and selectively enhanced memory for goal-relevant scenes. Furthermore, trial-level pupil dilations predicted better scene memory under threat, but were not related to object memory outcomes. fMRI analyses revealed that greater threat-evoked pupil dilations were positively associated with greater scene encoding activity in LC and parahippocampal cortex, a region specialized to process scene information. Across participants, this pattern of LC engagement for goal-relevant encoding was correlated with neuromelanin signal intensity, providing the first evidence that LC structure relates to its activation pattern during cognitive processing. Threat also reduced dynamic functional connectivity between high-priority (parahippocampal place area) and lower-priority (lateral occipital cortex) category-selective visual cortex in ways that predicted increased memory selectivity. Together, these findings support the idea that, under arousal, LC activity selectively strengthens prioritized memory representations by modulating local and functional network-level patterns of information processing.SIGNIFICANCE STATEMENT Adaptive behavior relies on the ability to select and store important information amid distraction. Prioritizing encoding of task-relevant inputs is especially critical in threatening or arousing situations, when forming these memories is essential for avoiding danger in the future. However, little is known about the arousal mechanisms that support such memory selectivity. Using fMRI, neuromelanin MRI, and pupil measures, we demonstrate that locus ceruleus (LC) activity amplifies neural gain such that limited encoding resources focus even more on prioritized mental representations under arousal. For the first time, we also show that LC structure relates to its involvement in threat-related encoding processes. These results shed new light on the brain mechanisms by which we process important information when it is most needed.

A Closer Look at the Hippocampus and Memory

Current interpretations of hippocampal memory function are blind to the fact that viewing behaviors are pervasive and complicate the relationships among perception, behavior, memory, and brain activity. For example, hippocampal activity and associative memory demands increase with stimulus complexity. Stimulus complexity also strongly modulates viewing. Associative processing and viewing thus are often confounded, rendering interpretation of hippocampal activity ambiguous. Similar considerations challenge many accounts of hippocampal function. To explain relationships between memory and viewing, we propose that the hippocampus supports the online memory demands necessary to guide visual exploration. The hippocampus thus orchestrates memory-guided exploration that unfolds over time to build coherent memories. This new perspective on hippocampal function harmonizes with the fact that memory formation and exploratory viewing are tightly intertwined.

Reward retroactively enhances memory consolidation for related items

Reward motivation has been shown to modulate episodic memory processes in order to support future adaptive behavior. However, for a memory system to be truly adaptive, it should enhance memory for rewarded events as well as for neutral events that may seem inconsequential at the time of encoding but can gain importance later. Here, we investigated the influence of reward motivation on retroactive memory enhancement selectively for conceptually related information. We found behavioral evidence that reward retroactively enhances memory at a 24-h memory test, but not at an immediate memory test, suggesting a role for post-encoding mechanisms of consolidation.