Integrating hippocampus and striatum in decision-making

Two routes to value-based decisions in Parkinson's disease: differentiating incremental reinforcement learning from episodic memory

Patients with Parkinson's disease are impaired at incremental reward-based learning. It is typically assumed that this impairment reflects a loss of striatal dopamine. However, many open questions remain about the nature of reward-based learning deficits in Parkinson's. Recent studies have found that a combination of different cognitive and computational strategies contribute even to simple reward-based learning tasks, suggesting a possible role for episodic memory. These findings raise critical questions about how incremental learning and episodic memory interact to support learning from past experience and what their relative contributions are to impaired decision-making in Parkinson's disease. Here we addressed these questions by asking patients with Parkinson's disease (n=26) both on and off their dopamine replacement medication and age- and education-matched healthy controls (n=26) to complete a task designed to isolate the contributions of incremental learning and episodic memory to reward-based learning and decision-making. We found that Parkinson's patients performed as well as healthy controls when using episodic memory, but were impaired at incremental reward-based learning. Dopamine replacement medication remediated this deficit while enhancing subsequent episodic memory for the value of motivationally relevant stimuli. These results demonstrate that Parkinson's patients are impaired at learning about reward from trial-and-error when episodic memory is properly controlled for, and that learning based on the value of single experiences remains intact in patients with Parkinson's disease.

Temporally organized representations of reward and risk in the human brain

The value and uncertainty associated with choice alternatives constitute critical features relevant for decisions. However, the manner in which reward and risk representations are temporally organized in the brain remains elusive. Here we leverage the spatiotemporal precision of intracranial electroencephalography, along with a simple card game designed to elicit the unfolding computation of a set of reward and risk variables, to uncover this temporal organization. Reward outcome representations across wide-spread regions follow a sequential order along the anteroposterior axis of the brain. In contrast, expected value can be decoded from multiple regions at the same time, and error signals in both reward and risk domains reflect a mixture of sequential and parallel encoding. We further highlight the role of the anterior insula in generalizing between reward prediction error and risk prediction error codes. Together our results emphasize the importance of neural dynamics for understanding value-based decisions under uncertainty.

Grid cells: the missing link in understanding Parkinson's disease?

The mechanisms underlying Parkinson's disease (PD) are complex and not fully understood, and the box-and-arrow model among other current models present significant challenges. This paper explores the potential role of the allocentric brain and especially its grid cells in several PD motor symptoms, including bradykinesia, kinesia paradoxa, freezing of gait, the bottleneck phenomenon, and their dependency on cueing. It is argued that central hubs, like the locus coeruleus and the pedunculopontine nucleus, often narrowly interpreted in the context of PD, play an equally important role in governing the allocentric brain as the basal ganglia. Consequently, the motor and secondary motor (e.g., spatially related) symptoms of PD linked with dopamine depletion may be more closely tied to erroneous computation by grid cells than to the basal ganglia alone. Because grid cells and their associated central hubs introduce both spatial and temporal information to the brain influencing velocity perception they may cause bradykinesia or hyperkinesia as well. In summary, PD motor symptoms may primarily be an allocentric disturbance resulting from virtual faulty computation by grid cells revealed by dopamine depletion in PD.

Circuit mechanisms of navigation strategy learning in mice

Navigation tasks involve the gradual selection and deployment of increasingly effective searching procedures to reach targets. The brain mechanisms underlying such complex behavior are poorly understood, but their elucidation might provide insights into the systems linking exploration and decision making in complex learning. Here, we developed a trial-by-trial goal-related search strategy analysis as mice learned to navigate identical water mazes encompassing distinct goal-related rules and monitored the strategy deployment process throughout learning. We found that navigation learning involved the following three distinct phases: an early phase during which maze-specific search strategies are deployed in a minority of trials, a second phase of preferential increasing deployment of one search strategy, and a final phase of increasing commitment to this strategy only. The three maze learning phases were affected differently by inhibition of retrosplenial cortex (RSC), dorsomedial striatum (DMS), or dorsolateral striatum (DLS). Through brain region-specific inactivation experiments and gain-of-function experiments involving activation of learning-related cFos+ ensembles, we unraveled how goal-related strategy selection relates to deployment throughout these sequential processes. We found that RSC is critically important for search strategy selection, DMS mediates strategy deployment, and DLS ensures searching consistency throughout maze learning. Notably, activation of specific learning-related ensembles was sufficient to direct strategy selection (RSC) or strategy deployment (DMS) in a different maze. Our results establish a goal-related search strategy deployment approach to dissect unsupervised navigation learning processes and suggest that effective searching in navigation involves evidence-based goal-related strategy direction by RSC, reinforcement-modulated strategy deployment through DMS, and online guidance through DLS.

Accounting for multiscale processing in adaptive real-world decision-making via the hippocampus

For adaptive real-time behavior in real-world contexts, the brain needs to allow past information over multiple timescales to influence current processing for making choices that create the best outcome as a person goes about making choices in their everyday life. The neuroeconomics literature on value-based decision-making has formalized such choice through reinforcement learning models for two extreme strategies. These strategies are model-free (MF), which is an automatic, stimulus-response type of action, and model-based (MB), which bases choice on cognitive representations of the world and causal inference on environment-behavior structure. The emphasis of examining the neural substrates of value-based decision making has been on the striatum and prefrontal regions, especially with regards to the "here and now" decision-making. Yet, such a dichotomy does not embrace all the dynamic complexity involved. In addition, despite robust research on the role of the hippocampus in memory and spatial learning, its contribution to value-based decision making is just starting to be explored. This paper aims to better appreciate the role of the hippocampus in decision-making and advance the successor representation (SR) as a candidate mechanism for encoding state representations in the hippocampus, separate from reward representations. To this end, we review research that relates hippocampal sequences to SR models showing that the implementation of such sequences in reinforcement learning agents improves their performance. This also enables the agents to perform multiscale temporal processing in a biologically plausible manner. Altogether, we articulate a framework to advance current striatal and prefrontal-focused decision making to better account for multiscale mechanisms underlying various real-world time-related concepts such as the self that cumulates over a person's life course.

Different topological patterns in structural covariance networks between high and low delay discounters

Introduction

People prefer immediate over future rewards because they discount the latter's value (a phenomenon termed "delay discounting," used as an index of impulsivity). However, little is known about how the preferences are implemented in brain in terms of the coordinated pattern of large-scale structural brain networks.

Methods

To examine this question, we classified high discounting group (HDG) and low discounting group (LDG) in young adults by assessing their propensity for intertemporal choice. We compared global and regional topological properties in gray matter volume-based structural covariance networks between two groups using graph theoretical analysis.

Results

HDG had less clustering coefficient and characteristic path length over the wide sparsity range than LDG, indicating low network segregation and high integration. In addition, the degree of small-worldness was more significant in HDG. Locally, HDG showed less betweenness centrality (BC) in the parahippocampal gyrus and amygdala than LDG.

Discussion

These findings suggest the involvement of structural covariance network topology on impulsive choice, measured by delay discounting, and extend our understanding of how impulsive choice is associated with brain morphological features.

The Hippocampus Contributes to Temporal Discounting When Delays and Rewards Are Experienced in the Moment

Temporal discounting (TD) represents the mental devaluation of rewards that are available after a delay. Whether the hippocampus is critical for TD remains unclear, with marked discrepancies between animal and human studies: although animals with discrete hippocampal lesions display impaired TD, human participants with similar lesions show intact performance on classic intertemporal choice tasks. A candidate explanation for this discrepancy is that delays and rewards are experienced in the moment in animal studies but tend to be hypothetical in human studies. We tested this hypothesis by examining the performance of amnesic participants with hippocampal lesions (one female, six males) on a novel experiential intertemporal choice task that used interesting photographs occluded by thick lines as rewards (Patt et al., 2021). Using a logistic function to model indifference points data, we compared performance to that on a classic intertemporal choice task with hypothetical outcomes. Participants with hippocampal lesions displayed impaired patterns of choices in the experiential task but not in the hypothetical task. Specifically, hippocampal lesions were associated with decreased amplitude of the delay-reward trade-off, with persistent choice of the delayed option despite delay increase. These results help explain previous discrepancies across animal and human studies, indicating that the hippocampus plays a critical role in temporal discounting when the outcomes of decisions are experienced in the moment, but not necessarily when they are hypothetical.SIGNIFICANCE STATEMENT Impaired temporal discounting (TD) has been related to maladaptive behaviors, including substance dependence and nonadherence to medical treatment. There is consensus that TD recruits the brain valuation network but whether the hippocampal memory system is additionally recruited remains unclear. This study examined TD in hippocampal amnesia, providing a unique opportunity to explore the role of the hippocampus in cognition. Whereas most human studies have used hypothetical outcomes, this study used a novel experiential task with real-time delays and rewards. Results demonstrated hippocampal involvement in the experiential task, but not in a classic hypothetical task administered for comparison. These findings elucidate previous discrepancies between animal and human TD studies. This reconciliation is critical as animals serve as models of human neurocognition.

Temporally organized representations of reward and risk in the human brain

The value and uncertainty associated with choice alternatives constitute critical features along which decisions are made. While the neural substrates supporting reward and risk processing have been investigated, the temporal organization by which these computations are encoded remains elusive. Here we leverage the high spatiotemporal precision of intracranial electroencephalography (iEEG) to uncover how representations of decision-related computations unfold in time. We present evidence of locally distributed representations of reward and risk variables that are temporally organized across multiple regions of interest. Reward outcome representations across wide-spread regions follow a temporally cascading order along the anteroposterior axis of the brain. In contrast, expected value can be decoded from multiple regions at the same time, and error signals in both reward and risk domains reflect a mixture of sequential and parallel encoding. We highlight the role of the anterior insula in generalizing between reward prediction error (RePE) and risk prediction error (RiPE), within which the encoding of RePE in the distributed iEEG signal predicts RiPE. Together our results emphasize the utility of uncovering temporal dynamics in the human brain for understanding how computational processes critical for value-based decisions under uncertainty unfold.

Dorsomedial striatum, but not dorsolateral striatum, is necessary for rat category learning

Categorization is an adaptive cognitive function that allows us to generalize knowledge to novel situations. Converging evidence from neuropsychological, neuroimaging, and neurophysiological studies suggest that categorization is mediated by the basal ganglia; however, there is debate regarding the necessity of each subregion of the basal ganglia and their respective functions. The current experiment examined the roles of the dorsomedial striatum (DMS; homologous to the head of the caudate nucleus) and dorsolateral striatum (DLS; homologous to the body and tail of the caudate nucleus) in category learning by combining selective lesions with computational modeling. Using a touchscreen apparatus, rats were trained to categorize distributions of visual stimuli that varied along two continuous dimensions (i.e., spatial frequency and orientation). The tasks either required attention to one stimulus dimension (spatial frequency or orientation; 1D tasks) or both stimulus dimensions (spatial frequency and orientation; 2D tasks). Rats with NMDA lesions of the DMS were impaired on both the 1D tasks and 2D tasks, whereas rats with DLS lesions showed no impairments. The lesions did not affect performance on a discrimination task that had the same trial structure as the categorization tasks, suggesting that the category impairments effected processes relevant to categorization. Model simulations were conducted using a neural network to assess the effect of the DMS lesions on category learning. Together, the results suggest that the DMS is critical to map category representations to appropriate behavioral responses, whereas the DLS is not necessary for categorization.

Neuronal signature of spatial decision-making during navigation by freely moving rats by using calcium imaging

This manuscript addresses a longstanding problem about how place cell firing contributes to navigation in spatial environments, and data derived from on-line calcium imaging offer a new solution. These data support the view that prior to entering an arena where an animal will navigate to only one of several possible places on each day, a population of spatially responsive cells, including place cells, fire during the last 5 s. This population is shown to be predictive of the destination and trajectory that the animal is about to take.

A challenge in spatial memory is understanding how place cell firing contributes to decision-making in navigation. A spatial recency task was created in which freely moving rats first became familiar with a spatial context over several days and thereafter were required to encode and then selectively recall one of three specific locations within it that was chosen to be rewarded that day. Calcium imaging was used to record from more than 1,000 cells in area CA1 of the hippocampus of five rats during the exploration, sample, and choice phases of the daily task. The key finding was that neural activity in the startbox rose steadily in the short period prior to entry to the arena and that this selective population cell firing was predictive of the daily changing goal on correct trials but not on trials in which the animals made errors. Single-cell and population activity measures converged on the idea that prospective coding of neural activity can be involved in navigational decision-making.

Reward-based decision-making in mesial temporal lobe epilepsy patients with unilateral hippocampal sclerosis pre- and post-surgery

BACKGROUND

Correct functioning of the reward processing system is critical for optimizing decision-making as well as preventing the development of addictions and/or neuropsychiatric symptoms such as depression, apathy, and anhedonia. Consequently, patients with mesial temporal lobe epilepsy due to unilateral hippocampal sclerosis (mTLE-UHS) represent an excellent opportunity to study the brain networks involved in this system.

OBJECTIVE

The aim of the current study was to evaluate decision-making and the electrophysiological correlates of feedback processing in a sample of mTLE-UHS patients, compared to healthy controls. In addition, we assessed the impact of mesial temporal lobe surgical resection on these processes, as well as general, neuropsychological functioning.

METHOD

17 mTLE-UHS patients and 17 matched healthy controls completed: [1] a computerized version of the Game of Dice Task, [2] a Standard Iowa Gambling Task, and [3] a modified ERP version of a probabilistic gambling task coupled with multichannel electroencephalography. Neuropsychological scores were also obtained both pre- and post-surgery.

RESULTS

Behavioral analyses showed a pattern of increased risk for the mTLE-UHS group in decision-making under ambiguity compared to the control group. A decrease in the amplitude of the Feedback Related Negativity (FRN), a weaker effect of valence on delta power, and a general reduction of delta and theta power in the mTLE-UHS group, as compared to the control group, were also found. The beta-gamma activity associated with the delivery of positive reward was similar in both groups. Behavioral performance and electrophysiological measures did not worsen post-surgery.

CONCLUSIONS

Patients with mTLE-UHS showed impairments in decision-making under ambiguity, particularly when they had to make decisions based on the outcomes of their choices, but not in decision-making under risk. No group differences were observed in decision-making when feedbacks were random. These results might be explained by the abnormal feedback processing seen in the EEG activity of patients with mTLE-UHS, and by concomitant impairments in working memory, and memory. These impairments may be linked to the disruption of mesial temporal lobe networks. Finally, feedback processing and decision-making under ambiguity were already affected in mTLE-UHS patients pre-surgery and did not show evidence of clear worsening post-surgery.

A systematic and meta-analytic review on the neural correlates of viewing high- and low-calorie foods among normal-weight adults

In the context of current-day online shopping, people select foods based on pictures and using their visual systems. Although there are some reviews of previous neuroimaging studies on appetitive behaviors, the findings on neural activation in response to pictures of high- and low-calorie foods seem inconsistent. This study aims to systematically review, integrate, and meta-analyze neuroimaging evidence of viewing high- and low-calorie foods. There were 25 samples from 24 studies, totalizing 489 normal-weight participants (311 female, 160 male, and 18 of unknown sex). We conducted a systematic review and Activation Likelihood Estimation (ALE) meta-analysis on viewing high-calorie foods (versus non-foods), low-calorie foods (versus non-foods), and high- versus low-calorie foods. In systematic review, several brain regions were shown to be activated when viewing high- or low-calorie foods (versus non-foods) and viewing high- versus low-calorie foods, including the prefrontal cortex, orbitofrontal cortex, amygdala, insula, ventral striatum, hippocampus, superior parietal lobe, and fusiform gyrus. However, the ALE meta-analysis showed that the left orbitofrontal cortex, left amygdala, insula, superior parietal lobe, and fusiform gyrus were activated when viewing high-calorie foods (versus non-foods); the left fusiform gyrus was activated when viewing low-calorie foods (versus non-foods); and no cluster was activated when viewing high- versus low-calorie foods. Our research suggests an appetitive brain network that includes visual perception and attentional processing, sensory input integration, subjective reward value encoding, decision-making, and top-down cognitive control. Future studies should control for the effects of methodological and physiological variables when examining the neural correlates of viewing high- and low-calorie foods.

Hippocampal Contribution to Probabilistic Feedback Learning: Modeling Observation- and Reinforcement-based Processes

Simple probabilistic reinforcement learning is recognized as a striatum-based learning system, but in recent years, has also been associated with hippocampal involvement. This study examined whether such involvement may be attributed to observation-based learning (OL) processes, running in parallel to striatum-based reinforcement learning. A computational model of OL, mirroring classic models of reinforcement-based learning (RL), was constructed and applied to the neuroimaging data set of Palombo, Hayes, Reid, and Verfaellie (2019). Hippocampal contributions to value-based learning: Converging evidence from fMRI and amnesia. Cognitive, Affective & Behavioral Neuroscience, 19(3), 523-536. Results suggested that OL processes may indeed take place concomitantly to reinforcement learning and involve activation of the hippocampus and central orbitofrontal cortex. However, rather than independent mechanisms running in parallel, the brain correlates of the OL and RL prediction errors indicated collaboration between systems, with direct implication of the hippocampus in computations of the discrepancy between the expected and actual reinforcing values of actions. These findings are consistent with previous accounts of a role for the hippocampus in encoding the strength of observed stimulus-outcome associations, with updating of such associations through striatal reinforcement-based computations. In addition, enhanced negative RL prediction error signaling was found in the anterior insula with greater use of OL over RL processes. This result may suggest an additional mode of collaboration between the OL and RL systems, implicating the error monitoring network.

The effects of stress across the lifespan on the brain, cognition and mental health: A UK biobank study

Repeated overstimulation of the stress response system, caused by exposure to prolonged highly stressful experiences, is thought to affect brain structure, cognitive ability, and mental health. We tested the effects of highly stressful experiences during childhood and adulthood using data from the UK Biobank, a large-scale national health and biomedical study with over 500,000 participants. To do this, we defined four groups with high or low levels of childhood and/or adulthood stress. We then used T1-and diffusion-weighted MRI data to assess the macrostructure of grey matter and microstructure of white matter within limbic brain regions, commonly associated with the stress response. We also compared executive function and working memory between these groups. Our findings suggest that in females, higher levels of Childhood stress were associated with reduced connectivity within the posterior thalamic radiation and cingulum of the hippocampus. In males however, higher levels of Adulthood stress is associated with similar changes in brain microstructure in the posterior thalamic radiation and cingulum of the hippocampus. High stress in Childhood and Adulthood was associated with decreases in executive function and working memory in both males and females. Stress across the lifespan was also positively associated with the number of diagnosed mental health problems, with a stronger effect in females than in males. Finally, our findings also suggest that cognitive and mental health outcomes due to stress may be mediated by the sex specific stress related changes in brain microstructure. Together our findings demonstrate clear links between stress at distinct phases of the lifespan, changes in measures of brain microstructure, impairments in cognitive abilities and negative mental health outcomes.

Environmental overlap influences goal-oriented coding of spatial sequences differently along the long-axis of hippocampus

When navigating our world we often first plan or retrieve a route to our goal, avoiding alternative paths to other destinations. Inspired by computational and animal models, we have recently demonstrated evidence that the human hippocampus supports prospective spatial coding, mediated by interactions with the prefrontal cortex. But the relationship between such signals and the need to discriminate possible routes based on their goal remains unclear. In the current study, we combined human fMRI, multi-voxel pattern analysis, and an established paradigm for contrasting memories of nonoverlapping routes with those of routes that cross paths and must be disambiguated. By classifying goal-oriented representations at the initiation of a navigational route, we demonstrate that environmental overlap modulates goal-oriented representations in the hippocampus. This modulation manifest through representational shifts from posterior to anterior components of the right hippocampus. Moreover, declines in goal-oriented decoding due to overlapping memories were predicted by the strength of the alternative memory, suggesting co-expression and competition between alternatives in the hippocampus during prospective thought. Moreover, exploratory whole-brain analyses revealed that a region of frontopolar cortex, which we have previously tied to prospective route planning, represented goal-states in new overlapping routes. Together, our findings provide insight into the influences of contextual overlap on the long-axis of the hippocampus and a broader memory and planning network that we have long-associated with such navigation tasks.

Computational validity: using computation to translate behaviours across species

We propose a new conceptual framework (computational validity) for translation across species and populations based on the computational similarity between the information processing underlying parallel tasks. Translating between species depends not on the superficial similarity of the tasks presented, but rather on the computational similarity of the strategies and mechanisms that underlie those behaviours. Computational validity goes beyond construct validity by directly addressing questions of information processing. Computational validity interacts with circuit validity as computation depends on circuits, but similar computations could be accomplished by different circuits. Because different individuals may use different computations to accomplish a given task, computational validity suggests that behaviour should be understood through the subject's point of view; thus, behaviour should be characterized on an individual level rather than a task level. Tasks can constrain the computational algorithms available to a subject and the observed subtleties of that behaviour can provide information about the computations used by each individual. Computational validity has especially high relevance for the study of psychiatric disorders, given the new views of psychiatry as identifying and mediating information processing dysfunctions that may show high inter-individual variability, as well as for animal models investigating aspects of human psychiatric disorders. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Navigating for reward

An organism's survival can depend on its ability to recall and navigate to spatial locations associated with rewards, such as food or a home. Accumulating research has revealed that computations of reward and its prediction occur on multiple levels across a complex set of interacting brain regions, including those that support memory and navigation. However, how the brain coordinates the encoding, recall and use of reward information to guide navigation remains incompletely understood. In this Review, we propose that the brain's classical navigation centres - the hippocampus and the entorhinal cortex - are ideally suited to coordinate this larger network by representing both physical and mental space as a series of states. These states may be linked to reward via neuromodulatory inputs to the hippocampus-entorhinal cortex system. Hippocampal outputs can then broadcast sequences of states to the rest of the brain to store reward associations or to facilitate decision-making, potentially engaging additional value signals downstream. This proposal is supported by recent advances in both experimental and theoretical neuroscience. By discussing the neural systems traditionally tied to navigation and reward at their intersection, we aim to offer an integrated framework for understanding navigation to reward as a fundamental feature of many cognitive processes.

Temporal discounting when outcomes are experienced in the moment: Validation of a novel paradigm and comparison with a classic hypothetical intertemporal choice task

When faced with intertemporal choices, people typically devalue rewards available in the future compared to rewards more immediately available, a phenomenon known as temporal discounting. Decisions involving intertemporal choices arise daily, with critical impact on health and financial wellbeing. Although many such decisions are "experiential" in that they involve delays and rewards that are experienced in real-time and can inform subsequent choices, most studies have focused on intertemporal choices with hypothetical outcomes (or outcomes delivered after all decisions are made). The present study focused on experiential intertemporal choices. First, a novel intertemporal choice task was developed and validated, using delays experienced in real time and artistic photographs as consumable perceptual rewards. Second, performance on the experiential task was compared to performance on a classic intertemporal choice task with hypothetical outcomes. Involvement of distinct processes across tasks was probed by examining differential relations to state and trait anxiety. A two-parameter logistic function framework was proposed to fit indifference point data. This approach accounts for individual variability not only in the delay at which an individual switches from choosing the delayed to more immediate option, but also in the slope of that switch. Fit results indicated that the experiential task elicited temporal discounting, with effective trade-off between delay and perceptual reward. Comparison with the hypothetical intertemporal choice task suggested distinct mechanisms: first, temporal discounting across the two tasks was not correlated; and second, state and trait anxiety both were associated with choice behavior in the experiential task, albeit in distinct ways, whereas neither was significantly associated with choice behavior in the hypothetical task. The engagement of different processes in the experiential compared to hypothetical task may align with neural evidence for the recruitment of the hippocampus in animal but not in classic human intertemporal choice studies.

Potential roles of the rodent medial prefrontal cortex in conflict resolution between multiple decision-making systems

Mammalian decision-making is mediated by the interaction of multiple, neurally and computationally separable decision systems. Having multiple systems requires a mechanism to manage conflict and converge onto the selection of singular actions. A long history of evidence has pointed to the prefrontal cortex as a central component in processing the interactions between distinct decision systems and resolving conflicts among them. In this chapter we review four theories of how that interaction might occur and identify how the medial prefrontal cortex in the rodent may be involved in each theory. We then present experimental predictions implied by the neurobiological data in the context of each theory as a starting point for future investigation of medial prefrontal cortex and decision-making.

Mushroom Body Extrinsic Neurons in Walking Bumblebees Correlate With Behavioral States but Not With Spatial Parameters During Exploratory Behavior

Central place foraging insects like honeybees and bumblebees learn to navigate efficiently between nest and feeding site. Essential components of this behavior can be moved to the laboratory. A major component of navigational learning is the active exploration of the test arena. These conditions have been used here to search for neural correlates of exploratory walking in the central arena (ground), and thigmotactic walking in the periphery (slope). We chose mushroom body extrinsic neurons (MBENs) because of their learning-related plasticity and their multi-modal sensitivities that may code relevant parameters in a brain state-dependent way. Our aim was to test whether MBENs code space-related components or are more involved in state-dependent processes characterizing exploration and thigmotaxis. MBENs did not respond selectively to body directions or locations. Their spiking activity differently correlated with walking speed depending on the animals' locations: on the ground, reflecting exploration, or on the slope, reflecting thigmotaxis. This effect depended on walking speed in different ways for different animals. We then asked whether these effects depended on spatial parameters or on the two states, exploration and thigmotaxis. Significant epochs of stable changes in spiking did not correlate with restricted locations in the arena, body direction, or walking transitions between ground and slope. We thus conclude that the walking speed dependencies are caused by the two states, exploration and thigmotaxis, rather than by spatial parameters.

Interindividual differences in memory system local field potential activity predict behavioral strategy on a dual-solution T-maze

Individuals can use diverse behavioral strategies to navigate their environment including hippocampal-dependent place strategies reliant upon cognitive maps and striatal-dependent response strategies reliant upon egocentric body turns. The existence of multiple memory systems appears to facilitate successful navigation across a wide range of environmental and physiological conditions. The mechanisms by which these systems interact to ultimately generate a unitary behavioral response, however, remain unclear. We trained 20 male, Sprague-Dawley rats on a dual-solution T-maze while simultaneously recording local field potentials that were targeted to the dorsolateral striatum and dorsal hippocampus. Eight rats spontaneously exhibited a place strategy while the remaining 12 rats exhibited a response strategy. Interindividual differences in behavioral strategy were associated with distinct patterns of LFP activity between the dorsolateral striatum and dorsal hippocampus. Specifically, striatal-hippocampal theta activity was in-phase in response rats and out-of-phase in place rats and response rats exhibited elevated striatal-hippocampal coherence across a wide range of frequency bands. These contrasting striatal-hippocampal activity regimes were (a) present during both maze-learning and a 30 min premaze habituation period and (b) could be used to train support vector machines to reliably predict behavioral strategy. Distinct patterns of neuronal activity across multiple memory systems, therefore, appear to bias behavioral strategy selection and thereby contribute to interindividual differences in behavior.

Stress drives deliberative tendencies by influencing vicarious trial and error in decision making

Previous studies have reported the effects of stress on decision making. However, the wide range of findings make it difficult to identify the fundamental effects of stress on decision making and, therefore, how stress affects decision making remains unknown. To investigate the influence of stress on decision making, we employed "vicarious trial and error" (VTE), which refers to a rat's behavior of orienting the head toward options at a decision point. VTE is thought to reflect mental simulation for possible options preceding a decision. We examined effects of acute restraint stress on VTE in a T-maze choice task. VTE depended on learning and past reward outcomes. Acute restraint stress before rats ran the T-maze choice task induced VTE, especially in trials with low demand of VTE, and increased the number of head orientations and time spent during each VTE. On the other hand, stress did not affect task performance (probability of advantageous choice) and patterns of behavioral choice (win-stay lose-shift, exploration-exploitation). In addition, stress activated serotonergic and noradrenergic neurons in the dorsal raphe nucleus and locus coeruleus, which are modulators of impulsivity and attentional control in decision making. These results suggest that stress in decision making drives the VTE process, which may lead to deep consideration, over-thinking, and indecisiveness.

Particle radiation-induced dysregulation of protein homeostasis in primary human and mouse neuronal cells

Space particle radiations may cause significant damage to proteins and oxidative stress in the cells within the central nervous system and pose a potential health hazard to humans in long-term manned space explorations. Dysregulation of the ubiquitin-proteasome system as evidenced by abnormal accumulation of polyubiquitin (pUb) chain linkages has been implicated in several age-related neurodegenerative disorders by mechanisms that may involve the inter-neuronal spread of toxic misfolded proteins, the induction of chronic neuroinflammation, or the inappropriate inhibition or activation of key enzymes, which could lead to dysfunction in, for example, proteolysis, or the accumulation of post-translationally-modified substrates.In this study, we employed a quantitative proteomics method to evaluate the impact of particle-radiation induced alterations in three major pUb-linked chains at lysine residues Lys-48 (K-48), Lys-63 (K-63), and Lys-11 (K-11), and probed for global proteomic changes in mouse and human neural cells that were irradiated with low doses of 250 MeV proton, 260 MeV/u silicon or 1 GeV/u iron ions. We found significant accumulation in K-48 linkage after 1 Gy protons and K-63 linkage after 0.5 Gy iron ions in human neural cells. Cells derived from different regions of the mouse brain (cortex, striatum and mesencephalon) showed differential sensitivity to particle radiation exposure. Although none of the linkages were altered after proton exposure, both K-48 and K-63 linkages in mouse striatal neuronal cells were elevated after 0.5 Gy of silicon or iron ions. Changes were also seen in proteins commonly used as markers of neural progenitor and stem cells, in DNA binding/damage repair and cellular redox pathways. In contrast, no significant changes were observed at the same time point after proton irradiation. These results suggest that the quality of the particle radiation plays a key role in the level, linkage and cell type specificity of protein homeostasis in key populations of neuronal cells.

Preparation for upcoming attentional states in the hippocampus and medial prefrontal cortex

Goal-directed attention is usually studied by providing individuals with explicit instructions on what they should attend to. But in daily life, we often use past experiences to guide our attentional states. Given the importance of memory for predicting upcoming events, we hypothesized that memory-guided attention is supported by neural preparation for anticipated attentional states. We examined preparatory coding in the human hippocampus and mPFC, two regions that are important for memory-guided behaviors, in two tasks: one where attention was guided by memory and another in which attention was explicitly instructed. Hippocampus and mPFC exhibited higher activity for memory-guided vs. explicitly instructed attention. Furthermore, representations in both regions contained information about upcoming attentional states. In the hippocampus, this preparation was stronger for memory-guided attention, and occurred alongside stronger coupling with visual cortex during attentional guidance. These results highlight the mechanisms by which memories are used to prepare for upcoming attentional goals.

Stress Disrupts Human Hippocampal-Prefrontal Function during Prospective Spatial Navigation and Hinders Flexible Behavior

The ability to anticipate and flexibly plan for the future is critical for achieving goal-directed outcomes. Extant data suggest that neural and cognitive stress mechanisms may disrupt memory retrieval and restrict prospective planning, with deleterious impacts on behavior. Here, we examined whether and how acute psychological stress influences goal-directed navigational planning and efficient, flexible behavior. Our methods combined fMRI, neuroendocrinology, and machine learning with a virtual navigation planning task. Human participants were trained to navigate familiar paths in virtual environments and then (concurrent with fMRI) performed a planning and navigation task that could be most efficiently solved by taking novel shortcut paths. Strikingly, relative to non-stressed control participants, participants who performed the planning task under experimentally induced acute psychological stress demonstrated (1) disrupted neural activity critical for mnemonic retrieval and mental simulation and (2) reduced traversal of shortcuts and greater reliance on familiar paths. These neural and behavioral changes under psychological stress were tied to evidence for disrupted neural replay of memory for future locations in the spatial environment, providing mechanistic insight into why and how stress can alter planning and foster inefficient behavior.

Modeling awake hippocampal reactivations with model-based bidirectional search

Hippocampal offline reactivations during reward-based learning, usually categorized as replay events, have been found to be important for performance improvement over time and for memory consolidation. Recent computational work has linked these phenomena to the need to transform reward information into state-action values for decision making and to propagate it to all relevant states of the environment. Nevertheless, it is still unclear whether an integrated reinforcement learning mechanism could account for the variety of awake hippocampal reactivations, including variety in order (forward and reverse reactivated trajectories) and variety in the location where they occur (reward site or decision-point). Here, we present a model-based bidirectional search model which accounts for a variety of hippocampal reactivations. The model combines forward trajectory sampling from current position and backward sampling through prioritized sweeping from states associated with large reward prediction errors until the two trajectories connect. This is repeated until stabilization of state-action values (convergence), which could explain why hippocampal reactivations drastically diminish when the animal's performance stabilizes. Simulations in a multiple T-maze task show that forward reactivations are prominently found at decision-points while backward reactivations are exclusively generated at reward sites. Finally, the model can generate imaginary trajectories that are not allowed to the agent during task performance. We raise some experimental predictions and implications for future studies of the role of the hippocampo-prefronto-striatal network in learning.

The hippocampus encodes delay and value information during delay-discounting decision making

The hippocampus, a region critical for memory and spatial navigation, has been implicated in delay discounting, the decline in subjective reward value when a delay is imposed. However, how delay information is encoded in the hippocampus is poorly understood. Here, we recorded from CA1 of mice performing a delay-discounting decision-making task, where delay lengths, delay positions, and reward amounts were changed across sessions, and identified subpopulations of CA1 neurons that increased or decreased their firing rate during long delays. The activity of both delay-active and -suppressed cells reflected delay length, delay position, and reward amount; but manipulating reward amount differentially impacted the two populations, suggesting distinct roles in the valuation process. Further, genetic deletion of the N-methyl-D-aspartate (NMDA) receptor in hippocampal pyramidal cells impaired delay-discount behavior and diminished delay-dependent activity in CA1. Our results suggest that distinct subclasses of hippocampal neurons concertedly support delay-discounting decisions in a manner that is dependent on NMDA receptor function.

Striatal and hippocampal contributions to flexible navigation in rats and humans

The hippocampus has been firmly established as playing a crucial role in flexible navigation. Recent evidence suggests that dorsal striatum may also play an important role in such goal-directed behaviour in both rodents and humans. Across recent studies, activity in the caudate nucleus has been linked to forward planning and adaptation to changes in the environment. In particular, several human neuroimaging studies have found the caudate nucleus tracks information traditionally associated with that by the hippocampus. In this brief review, we examine this evidence and argue the dorsal striatum encodes the transition structure of the environment during flexible, goal-directed behaviour. We highlight that future research should explore the following: (1) Investigate neural responses during spatial navigation via a biophysically plausible framework explained by reinforcement learning models and (2) Observe the interaction between cortical areas and both the dorsal striatum and hippocampus during flexible navigation.

Association Between Hippocampus, Thalamus, and Caudate in Mild Cognitive Impairment APOEε4 Carriers: A Structural Covariance MRI Study

Objective: Although, the apolipoprotein E (APOE) genotype is widely recognized as one of the most important risk factors for Alzheimer's disease (AD) development, the neural mechanisms by which the ε4 allele promotes the AD occurring remain under debate. The aim of this study was to evaluate neurobiological effects of the APOE-genotype on the pattern of the structural covariance in mild cognitive impairment (MCI) subjects. Methods: We enrolled 95 MCI subjects and 49 healthy controls. According to APOE-genotype, MCI subjects were divided into three groups: APOEε4 non-carriers (MCIε4-/-, n = 55), APOEε4 heterozygous carriers (MCIε4+/-, n = 31), and APOEε4 homozygous carriers (MCIε4+/+, n = 9) while all controls were APOEε4 non-carriers. In order to explore their brain structural pattern, T1-weighted anatomical brain 1.5-T MRI scans were collected. A whole-brain voxel-based morphometry analysis was performed, and all significant regions (p < 0.05 family-wise error, whole brain) were selected as a region of interest for the structural covariance analysis. Moreover, in order to evaluate the progression of the disease, a clinical follow-up was performed for 2 years. Results: The F-test showed in voxel-based morphometry analysis a strong overall difference among the groups in the middle frontal and temporal gyri and in the bilateral hippocampi, thalami, and parahippocampal gyri, with a grading in the atrophy in these latter three structures according to the following order: MCIε4+/+ > MCIε4+/- > MCIε4-/- > controls. Structural covariance analysis revealed a strong structural association between the left thalamus and the left caudate and between the right hippocampus and the left caudate (p < 0.05 family-wise error, whole brain) in the MCIε4 carrier groups (MCIε4+/+ > MCIε4+/-), whereas no significant associations were observed in MCIε4-/- subjects. Of note, the 38% of MCIs enrolled in this study developed AD within 2 years of follow-up. Conclusion: This study improves the knowledge on neurobiological effect of APOE ε4 in early pathophysiological phenomena underlying the MCI-to-AD evolution, as our results demonstrate changes in the structural association between hippocampal formation and thalamo-striatal connections occurring in MCI ε4 carriers. Our results strongly support the role of subcortical structures in MCI ε4 carriers and open a clinical window on the role of these structures as early disease markers.

Loss of PTEN-induced kinase 1 (Pink1) reduces hippocampal tyrosine hydroxylase and impairs learning and memory

Phosphatase and tensin homolog (PTEN)-induced kinase 1 (Pink1) is involved in mitochondrial quality control, which is essential for maintaining energy production and minimizing oxidative damage from dysfunctional/depolarized mitochondria. Pink1 mutations are the second most common cause of autosomal recessive Parkinson's disease (PD). In addition to characteristic motor impairments, PD patients also commonly exhibit cognitive impairments. As the hippocampus plays a prominent role in cognition, we tested if loss of Pink1 in mice influences learning and memory. While wild-type mice were able to perform a contextual discrimination task, age-matched Pink1 knockout (Pink1^-/-) mice showed an impaired ability to differentiate between two similar contexts. Similarly, Pink1^-/- mice performed poorly in a delayed alternation task as compared to age-matched controls. Poor performance in these cognitive tasks was not the result of overt hippocampal pathology. However, a significant reduction in hippocampal tyrosine hydroxylase (TH) protein levels was detected in the Pink1^-/- mice. This decrease in hippocampal TH levels was also associated with reduced DOPA decarboxylase and dopamine D2 receptor levels, but not post-synaptic dopamine D1 receptor levels. These presynaptic changes appeared to be selective for dopaminergic fibers as hippocampal dopamine beta hydroxylase, choline acetyltransferase, and tryptophan hydroxylase levels were unchanged in Pink1^-/- mice. Administration of the dopamine D1 receptor agonist SKF38393 to Pink1^-/- mice was found to improve performance in the context discrimination task. Taken together, our results show that Pink1 loss may alter dopamine signaling in the hippocampus, which could be a contributing mechanism for the observed learning and memory impairments.

New perspectives on dimensionality and variability from large-scale cortical dynamics

The neocortex is a multi-scale network, with intricate local circuitry interwoven into a global mesh of long-range connections. Neural activity propagates within this network on a wide range of temporal and spatial scales. At the micro scale, neurophysiological recordings reveal coordinated dynamics in local neural populations, which support behaviorally relevant computations. At the macro scale, neuroimaging modalities measure global activity fluctuations organized into spatiotemporal patterns across the entire brain. Here we review recent advances linking the local and global scales of cortical dynamics and their relationship to behavior. We argue that diverse experimental observations on the dimensionality and variability of neural activity can be reconciled by considering how activity propagates in space and time on multiple spatial scales.

Neural Representation of Overlapping Path Segments and Reward Acquisitions in the Monkey Hippocampus

Disambiguation of overlapping events is thought to be the hallmark of episodic memory. Recent rodent studies have reported that when navigating overlapping path segments in the different routes place cell activity in the same overlapping path segments were remapped according to different goal locations in different routes. However, it is unknown how hippocampal neurons disambiguate reward delivery in overlapping path segments in different routes. In the present study, we recorded monkey hippocampal neurons during performance of three virtual navigation (VN) tasks in which a monkey alternately navigated two different routes that included overlapping path segments (common central hallway) and acquired rewards in the same locations in overlapping path segments by manipulating a joystick. The results indicated that out of 106 hippocampal neurons, 57 displayed place-related activity (place-related neurons), and 18 neurons showed route-dependent activity in the overlapping path segments, consistent with a hippocampal role in the disambiguation of overlapping path segments. Moreover, 75 neurons showed neural correlates to reward delivery (reward-related neurons), whereas 56 of these 75 reward-related neurons showed route-dependent reward-related activity in the overlapping path segments. The ensemble activity of reward-related neurons represented reward delivery, locations, and routes in the overlapping path segments. In addition, ensemble activity patterns of hippocampal neurons more distinctly represented overlapping path segments than non-overlapping path segments. The present results provide neurophysiological evidence of disambiguation in the monkey hippocampus, consistent with a hippocampal role in episodic memory, and support a recent computational model of "neural differentiation," in which overlapping items are better represented by repeated retrieval with competitive learning.

Neurocognitive free will

Free will is an apparent paradox because it requires a historical identity to escape its history in a self-guided fashion. Philosophers have itemized design features necessary for this escape, scaling from action to agency and vice versa. These can be organized into a coherent framework that neurocognitive capacities provide and that form a basis for neurocognitive free will. These capacities include (1) adaptive access to unpredictability, (2) tuning of this unpredictability in the service of hierarchical goal structures, (3) goal-directed deliberation via search over internal cognitive representations, and (4) a role for conscious construction of the self in the generation and choice of alternatives. This frames free will as a process of generative self-construction, by which an iterative search process samples from experience in an adaptively exploratory fashion, allowing the agent to explore itself in the construction of alternative futures. This provides an explanation of how effortful conscious control modulates adaptive access to unpredictability and resolves one of free will's key conceptual problems: how randomness is used in the service of the will. The implications provide a contemporary neurocognitive grounding to compatibilist and libertarian positions on free will, and demonstrate how neurocognitive understanding can contribute to this debate by presenting free will as an interaction between our freedom and our will.

Neuronal representation of environmental boundaries in egocentric coordinates

Movement through space is a fundamental behavior for all animals. Cognitive maps of environments are encoded in the hippocampal formation in an allocentric reference frame, but motor movements that comprise physical navigation are represented within an egocentric reference frame. Allocentric navigational plans must be converted to an egocentric reference frame prior to implementation as overt behavior. Here we describe an egocentric spatial representation of environmental boundaries in the dorsomedial striatum.

The hippocampus represents an allocentric map of space, however, motor movements used for navigation are defined in an egocentric framework. Here, the authors report that dorsomedial striatal neurons exhibit an egocentric representation of the boundaries in the environment.

Increased flanker task and forward digit span performance in caudate-nucleus-dependent response strategies

One of two memory systems can be used to navigate in a new environment. Hippocampus-dependent spatial strategy consists of creating a cognitive map of an environment and caudate nucleus-dependent response strategy consists of memorizing a rigid sequence of turns. Spontaneous use of the response strategy is associated with greater activity and grey matter within the caudate nucleus while the spatial strategy is associated with greater activity and grey matter in the hippocampus. The caudate nucleus is involved in executive functions such as working memory, cognitive control and certain aspects of attention such as attentional disengaging. This study therefore aimed to investigate whether response learners would display better performance on tests of executive and attention functioning compared to spatial learners. Fifty participants completed the 4/8 virtual maze to assess navigational strategy, the forward and backward visual digit span and the Attention Network Test - Revised to assess both attention disengagement and cognitive control. Results revealed that response learners showed significantly higher working memory capacity, more efficient attention disengagement and better cognitive control. Results suggest that response learners, who putatively display more grey matter and activity in the caudate nucleus, are associated with better working memory span, cognitive control and attentional disengagement.

Hippocampal Sequences During Exploration: Mechanisms and Functions

Although the hippocampus plays a critical role in spatial and episodic memories, the mechanisms underlying memory formation, stabilization, and recall for adaptive behavior remain relatively unknown. During exploration, within single cycles of the ongoing theta rhythm that dominates hippocampal local field potentials, place cells form precisely ordered sequences of activity. These neural sequences result from the integration of both external inputs conveying sensory-motor information, and intrinsic network dynamics possibly related to memory processes. Their endogenous replay during subsequent sleep is critical for memory consolidation. The present review discusses possible mechanisms and functions of hippocampal theta sequences during exploration. We present several lines of evidence suggesting that these neural sequences play a key role in information processing and support the formation of initial memory traces, and discuss potential functional distinctions between neural sequences emerging during theta vs. awake sharp-wave ripples.

Concreteness and emotional valence of episodic future thinking (EFT) independently affect the dynamics of intertemporal decisions

During intertemporal decisions, the value of future rewards decreases as a function of the delay of its receipt (temporal discounting, TD). Since high discount rates have been associated with a series of problematic behaviours and clinical conditions, current research has focused on possible modulators of TD. Specifically, a reduction of individual discount rates has been shown during episodic future thinking (EFT), wherein time intervals are anchored to personal future events. However, it is not entirely clear whether this effect is mediated by a change in the representation of future events (i.e., from abstract to concrete) or by a positive-emotion modulation. Here, we investigated this issue by manipulating the valence of the EFT (i.e., using negative, neutral and positive episodic tags), and by collecting explicit and implicit measures of behaviour. The results showed a significant reduction of TD in all the three emotional conditions compared to the baseline, with differences among them, thus suggesting the existence of a cumulative effect of the concreteness and affective components of the EFT. The analyses of implicit measures additionally revealed that this effect was mediated by a simultaneous increase/decrease of attraction toward the delayed/immediate alternative. Finally, these effects appeared to be modulated by participants' baseline discounting preferences. These findings provide important insights on clinical applications in reward-related disorders.

The perirhinal cortex supports spatial intertemporal choice stability

In order to optimize outcomes in the face of uncertainty, one must recall past experiences and extrapolate to the future by assigning values to different choice outcomes. This behavior requires an interplay between memory and reward valuation, necessitating communication across many brain regions. At the anatomical nexus of this interplay is the perirhinal cortex (PRC). The PRC is densely connected to the amygdala and orbital frontal cortex, regions that have been implicated in reward-based decision making, as well as the hippocampus. Thus, the PRC could serve as a hub for integrating memory, reward, and prediction. The PRC's role in value-based decision making, however, has not been empirically examined. Therefore, we tested the role of the PRC in a spatial delay discounting task, which allows rats to choose between a 1-s delay for a small food reward and a variable delay for a large food reward, with the delay to the large reward increasing after choice of each large reward and decreasing after each small reward. The rat can therefore adjust the delay by consecutively choosing the same reward or stabilize the delay by alternating between sides. The latter has been shown to occur once the 'temporal cost' of the large reward is established and is a decision-making process termed 'exploitation'. When the PRC was bilaterally inactivated with the GABA(A) agonist muscimol, rats spent fewer trials successfully exploiting to maintain a fixed delay compared to the vehicle control condition. Moreover, PRC inactivation resulted in an increased number of vicarious trial and error (VTE) events at the choice point, where rats had to decide between the two rewards. These behavioral patterns suggest that the PRC is critical for maintaining stability in linking a choice to a reward outcome in the face of a variable cost.

Pharmacological activation of the nuclear receptor REV-ERB reverses cognitive deficits and reduces amyloid-β burden in a mouse model of Alzheimer's disease

Alzheimer's disease currently lacks treatment options that effectively reverse the biological/anatomical pathology and cognitive deficits associated with the disease. Loss of function of the nuclear receptor REV-ERB is associated with reduced cognitive function in mouse models. The effect of enhanced REV-ERB activity on cognitive function has not been examined. In this study, we tested the hypothesis that enhanced REV-ERB function may enhance cognitive function in a model of Alzheimer's disease. We utilized the REV-ERB agonist SR9009 to pharmacologically activate the activity of REV-ERB in the SAMP8 mouse model of Alzheimer's disease. SR9009 reversed cognitive dysfunction of an aged SAMP8 mouse in several behavioral assays including novel object recognition, T-maze foot shock avoidance, and lever press operant conditioning task assessments. SR9009 treatment reduced amyloid-β 1-40 and 1-42 levels in the cortex, which is consistent with improved cognitive function. Furthermore, SR9009 treatment led to increased hippocampal PSD-95, cortical synaptophysin expression and the number of synapses suggesting improvement in synaptic function. We conclude that REV-ERB is a potential target for treatment of Alzheimer's disease.

Caudate nucleus-dependent navigation strategies are associated with increased risk-taking and set-shifting behavior

When people navigate, they use strategies dependent on one of two memory systems. The hippocampus-based spatial strategy consists of using multiple landmarks to create a cognitive map of the environment. In contrast, the caudate nucleus-based response strategy is based on the memorization of a series of turns. Importantly, response learners display more gray matter and functional activity in the caudate nucleus and less gray matter in the hippocampus. In parallel, the caudate nucleus is involved in decision-making by mediating attention toward rewards and in set-shifting by mediating preparatory actions. The present study, therefore, examined the link between navigational strategy use, that are associated with gray matter differences in the caudate nucleus and hippocampus, and decision-making and set-shifting performance. Fifty-three participants completed the 4 on 8 virtual maze, the Iowa Gambling Task (IGT), the Wisconsin Card Sorting Test-64 (WCST-64), and a task-switching test. The results revealed that people who use response strategies displayed increased risk-taking behavior in the IGT compared to the people using hippocampus-dependent spatial strategies. Response strategy was also associated with enhanced set-shifting performance in the WCST-64 and task-switching test. These results confirm that risk-taking and set-shifting behavior, that are differentially impacted by the caudate nucleus and hippocampus memory systems, can be predicted by navigational strategy.

The role of the hippocampus in weighting expectations during inference under uncertainty

Making inference under uncertainty requires an optimal weighting of prior expectations and observations. How this weighting is realized in the brain remains elusive. To investigate this, we recorded functional neuroimaging data while participants estimated a number based on noisy observations. Crucially, the prior expectation about the variability of observations (an expected variability) was manipulated. Consistent with normative models, when novel observations were characterized by higher expected or observed variability, participants' estimates relied more on expectations than novel observations and were characterized by higher stochasticity. Activity in hippocampus increased when novel evidence was characterized by higher expected or observed variability. Response in superior parietal cortex reflected a precision-weighted prediction error signal (i.e., the distance between observations and expectations) that was modulated by hippocampal activity. Our findings implicate the hippocampus during inference under uncertainty, suggesting a role in weighting prior representations over observations and in modulating responsivity of superior parietal cortex to prediction error.

Mesolimbic connectivity signatures of impulsivity and BMI in early adolescence

Across age groups, differences in connectivity of the mesolimbic and the prefrontal cortex co-vary with trait impulsivity and sensation-seeking. Impulsivity and sensation-seeking are also known to increase during early adolescence as maturation of subcortical structures outpaces that of the prefrontal cortex. While an imbalance between the striatum and prefrontal cortex is considered a normal developmental process, higher levels of adolescent impulsivity and sensation-seeking are associated with an increased risk for diverse problems, including obesity. To determine how the relationship between sensation-seeking, impulsivity and body mass index (BMI) is related to shared neural correlates we measured their relationships with the connectivity of nuclei in the striatum and dopaminergic midbrain in young adolescents. Data were collected from 116 children between the ages of 12 and 14, and included resting state functional magnetic resonance imaging, personality measures from the Substance Use Risk Profile Scale, and BMI Z-score for age. The shared variance for the connectivity of regions of interest in the substantia nigra, ventral tegmental area, ventral striatum and sub-thalamic nucleus, personality measures and BMI Z-score for age, were analyzed using partial least squares correlation. This analysis identified a single significant striato-limbic network that was connected with the substantia nigra, ventral tegmental area and sub-thalamic nuclei (p = 0.002). Connectivity within this network which included the hippocampi, amygdalae, parahippocampal gyri and the regions of interest, correlated positively with impulsivity and BMI Z-score for age and negatively with sensation-seeking. Together, these findings emphasize that, in addition to the well-established role that frontostriatal circuits play in the development of adolescent personality traits, connectivity of limbic regions with the striatum and midbrain also impact impulsivity, sensation-seeking and BMI Z-score in adolescents.

Hippocampal replays under the scrutiny of reinforcement learning models

Multiple in vivo studies have shown that place cells from the hippocampus replay previously experienced trajectories. These replays are commonly considered to mainly reflect memory consolidation processes. Some data, however, have highlighted a functional link between replays and reinforcement learning (RL). This theory, extensively used in machine learning, has introduced efficient algorithms and can explain various behavioral and physiological measures from different brain regions. RL algorithms could constitute a mechanistic description of replays and explain how replays can reduce the number of iterations required to explore the environment during learning. We review the main findings concerning the different hippocampal replay types and the possible associated RL models (either model-based, model-free, or hybrid model types). We conclude by tying these frameworks together. We illustrate the link between data and RL through a series of model simulations. This review, at the frontier between informatics and biology, paves the way for future work on replays.

Remembering rewarding futures: A simulation-selection model of the hippocampus

Despite tremendous progress, the neural circuit dynamics underlying hippocampal mnemonic processing remain poorly understood. We propose a new model for hippocampal function-the simulation-selection model-based on recent experimental findings and neuroecological considerations. Under this model, the mammalian hippocampus evolved to simulate and evaluate arbitrary navigation sequences. Specifically, we suggest that CA3 simulates unexperienced navigation sequences in addition to remembering experienced ones, and CA1 selects from among these CA3-generated sequences, reinforcing those that are likely to maximize reward during offline idling states. High-value sequences reinforced in CA1 may allow flexible navigation toward a potential rewarding location during subsequent navigation. We argue that the simulation-selection functions of the hippocampus have evolved in mammals mostly because of the unique navigational needs of land mammals. Our model may account for why the mammalian hippocampus has evolved not only to remember, but also to imagine episodes, and how this might be implemented in its neural circuits.

Rethinking dopamine as generalized prediction error

Midbrain dopamine neurons are commonly thought to report a reward prediction error (RPE), as hypothesized by reinforcement learning (RL) theory. While this theory has been highly successful, several lines of evidence suggest that dopamine activity also encodes sensory prediction errors unrelated to reward. Here, we develop a new theory of dopamine function that embraces a broader conceptualization of prediction errors. By signalling errors in both sensory and reward predictions, dopamine supports a form of RL that lies between model-based and model-free algorithms. This account remains consistent with current canon regarding the correspondence between dopamine transients and RPEs, while also accounting for new data suggesting a role for these signals in phenomena such as sensory preconditioning and identity unblocking, which ostensibly draw upon knowledge beyond reward predictions.

Pattern classification differentiates decision of intertemporal choices using multi-voxel pattern analysis

In daily life, individuals frequently make trade-offs between the small-but-immediate benefits and large-but-delayed profits. This type of decision is known as intertemporal choice. Previous studies have uncovered the neurobiological mechanism of the intertemporal choice, but it still remains unclear how the patterns of brain activity predict the decisions of intertemporal choices. To fill this gap, we used functional magnetic resonance imaging (fMRI), in conjunction with the machine learning technique of multi-voxel pattern analysis (MVPA), to ascertain the predictive capability of the neuronal pattern for classifying individuals' intertemporal decisions across two independent samples. To further probe how this neuronal pattern worked in predicting individual intertemporal decision, we drew on the Power Atlas to examine the accuracies of classification within each regional mask as well. Classification findings showed that the pattern of neuronal activity over the whole-brain can correctly classify the accuracies of individual decisions up to 84.3%. Encouragingly, further analysis shows that the neuronal information encoded in three brain functional networks can predict individuals' decisions with significant discriminative power in cross-samples, namely the valuation network (e.g., striatum), the cognitive control network (e.g., dorsolateral prefrontal cortex) and the episodic prospection network (e.g., amygdala, parahippocampus gyrus, insula). Collectively, these findings advance our comprehension of the neuronal mechanism of human intertemporal decisions, and substantially reshape our understanding for this cardinal behaviour from behavioural-brain scheme to brain-behavioural configuration.

Visual statistical learning deficits in memory-impaired individuals

Visual statistical learning (VSL) refers to the learning of environmental regularities. Classically considered an implicit process, one patient with isolated hippocampal damage is severely impaired at VSL tasks, suggesting involvement of explicit memory. Here, we asked whether memory impairment (MI) alone, absent of clear hippocampal pathology, predicted deficits across different VSL tasks. A classic VSL task revealed no learning in MI participants (Exp. 1), while imposing attentional demands (Exp. 2: flicker detection, Exp. 3: gender/location categorization) during familiarization revealed modest residual VSL. MI with nonspecific neural correlates predicted impaired VSL overall, but attentional processes may be harnessed for rehabilitation.