Prefrontal reinstatement of contextual task demand is predicted by separable hippocampal patterns

Schemas, reinforcement learning and the medial prefrontal cortex

Schemas are rich and complex knowledge structures about the typical unfolding of events in a context; for example, a schema of a dinner at a restaurant. In this Perspective, we suggest that reinforcement learning (RL), a computational theory of learning the structure of the world and relevant goal-oriented behaviour, underlies schema learning. We synthesize literature about schemas and RL to offer that three RL principles might govern the learning of schemas: learning via prediction errors, constructing hierarchical knowledge using hierarchical RL, and dimensionality reduction through learning a simplified and abstract representation of the world. We then suggest that the orbitomedial prefrontal cortex is involved in both schemas and RL due to its involvement in dimensionality reduction and in guiding memory reactivation through interactions with posterior brain regions. Last, we hypothesize that the amount of dimensionality reduction might underlie gradients of involvement along the ventral-dorsal and posterior-anterior axes of the orbitomedial prefrontal cortex. More specific and detailed representations might engage the ventral and posterior parts, whereas abstraction might shift representations towards the dorsal and anterior parts of the medial prefrontal cortex.

Cost-benefit tradeoff mediates the transition from rule-based to memory-based processing during practice

Practice not only improves task performance but also changes task execution from rule- to memory-based processing by incorporating experiences from practice. However, how and when this change occurs is unclear. We test the hypothesis that strategy transitions in task learning can result from decision-making guided by cost-benefit analysis. Participants learn 2 task sequences and are then queried about the task type at a cued sequence and position. Behavioral improvement with practice can be accounted for by a computational model implementing cost-benefit analysis and the model-predicted strategy transition points align with the observed behavioral slowing. Model comparisons using behavioral data show that strategy transitions are better explained by a cost-benefit analysis across alternative strategies rather than solely on memory strength. Model-guided fMRI findings suggest that the brain encodes a decision variable reflecting the cost-benefit analysis and that different strategy representations are double-dissociated. Further analyses reveal that strategy transitions are associated with activation patterns in the dorsolateral prefrontal cortex and increased pattern separation in the ventromedial prefrontal cortex. Together, these findings support cost-benefit analysis as a mechanism of practice-induced strategy shift.

Contextualizing predictive minds

The structure of human memory seems to be optimized for efficient prediction, planning, and behavior. We propose that these capacities rely on a tripartite structure of memory that includes concepts, events, and contexts-three layers that constitute the mental world model. We suggest that the mechanism that critically increases adaptivity and flexibility is the tendency to contextualize. This tendency promotes local, context-encoding abstractions, which focus event- and concept-based planning and inference processes on the task and situation at hand. As a result, cognitive contextualization offers a solution to the frame problem-the need to select relevant features of the environment from the rich stream of sensorimotor signals. We draw evidence for our proposal from developmental psychology and neuroscience. Adopting a computational stance, we present evidence from cognitive modeling research which suggests that context sensitivity is a feature that is critical for maximizing the efficiency of cognitive processes. Finally, we turn to recent deep-learning architectures which independently demonstrate how context-sensitive memory can emerge in a self-organized learning system constrained by cognitively-inspired inductive biases.

A neural network model of differentiation and integration of competing memories

What determines when neural representations of memories move together (integrate) or apart (differentiate)? Classic supervised learning models posit that, when two stimuli predict similar outcomes, their representations should integrate. However, these models have recently been challenged by studies showing that pairing two stimuli with a shared associate can sometimes cause differentiation, depending on the parameters of the study and the brain region being examined. Here, we provide a purely unsupervised neural network model that can explain these and other related findings. The model can exhibit integration or differentiation depending on the amount of activity allowed to spread to competitors - inactive memories are not modified, connections to moderately active competitors are weakened (leading to differentiation), and connections to highly active competitors are strengthened (leading to integration). The model also makes several novel predictions - most importantly, that when differentiation occurs as a result of this unsupervised learning mechanism, it will be rapid and asymmetric, and it will give rise to anticorrelated representations in the region of the brain that is the source of the differentiation. Overall, these modeling results provide a computational explanation for a diverse set of seemingly contradictory empirical findings in the memory literature, as well as new insights into the dynamics at play during learning.

Reconstructing Spatiotemporal Trajectories of Visual Object Memories in the Human Brain

How the human brain reconstructs, step-by-step, the core elements of past experiences is still unclear. Here, we map the spatiotemporal trajectories along which visual object memories are reconstructed during associative recall. Specifically, we inquire whether retrieval reinstates feature representations in a copy-like but reversed direction with respect to the initial perceptual experience, or alternatively, this reconstruction involves format transformations and regions beyond initial perception. Participants from two cohorts studied new associations between verbs and randomly paired object images, and subsequently recalled the objects when presented with the corresponding verb cue. We first analyze multivariate fMRI patterns to map where in the brain high- and low-level object features can be decoded during perception and retrieval, showing that retrieval is dominated by conceptual features, represented in comparatively late visual and parietal areas. A separately acquired EEG dataset is then used to track the temporal evolution of the reactivated patterns using similarity-based EEG-fMRI fusion. This fusion suggests that memory reconstruction proceeds from anterior frontotemporal to posterior occipital and parietal regions, in line with a conceptual-to-perceptual gradient but only partly following the same trajectories as during perception. Specifically, a linear regression statistically confirms that the sequential activation of ventral visual stream regions is reversed between image perception and retrieval. The fusion analysis also suggests an information relay to frontoparietal areas late during retrieval. Together, the results shed light onto the temporal dynamics of memory recall and the transformations that the information undergoes between the initial experience and its later reconstruction from memory.

Differentiation and Integration of Competing Memories: A Neural Network Model

What determines when neural representations of memories move together (integrate) or apart (differentiate)? Classic supervised learning models posit that, when two stimuli predict similar outcomes, their representations should integrate. However, these models have recently been challenged by studies showing that pairing two stimuli with a shared associate can sometimes cause differentiation, depending on the parameters of the study and the brain region being examined. Here, we provide a purely unsupervised neural network model that can explain these and other related findings. The model can exhibit integration or differentiation depending on the amount of activity allowed to spread to competitors - inactive memories are not modified, connections to moderately active competitors are weakened (leading to differentiation), and connections to highly active competitors are strengthened (leading to integration). The model also makes several novel predictions - most importantly, that when differentiation occurs as a result of this unsupervised learning mechanism, it will be rapid and asymmetric, and it will give rise to anticorrelated representations in the region of the brain that is the source of the differentiation. Overall, these modeling results provide a computational explanation for a diverse set of seemingly contradictory empirical findings in the memory literature, as well as new insights into the dynamics at play during learning.

Presynaptic GABAA receptors control integration of nicotinic input onto dopaminergic axons in the striatum

Axons of dopaminergic neurons express gamma-aminobutyric acid type-A receptors (GABAARs) and nicotinic acetylcholine receptors (nAChRs) which are both independently positioned to shape striatal dopamine release. Using electrophysiology and calcium imaging, we investigated how interactions between GABAARs and nAChRs influence dopaminergic axon excitability. Direct axonal recordings showed that benzodiazepine application suppresses subthreshold axonal input from cholinergic interneurons (CINs). In imaging experiments, we used the first temporal derivative of presynaptic calcium signals to distinguish between direct- and nAChR-evoked activity in dopaminergic axons. We found that GABAAR antagonism with gabazine selectively enhanced nAChR-evoked axonal signals. Acetylcholine release was unchanged in gabazine suggesting that GABAARs located on dopaminergic axons, but not CINs, mediated this enhancement. Unexpectedly, we found that a widely used GABAAR antagonist, picrotoxin, inhibits axonal nAChRs and should be used cautiously for striatal circuit analysis. Overall, we demonstrate that GABAARs on dopaminergic axons regulate integration of nicotinic input to shape presynaptic excitability.

Dorsolateral prefrontal activity supports a cognitive space organization of cognitive control

Cognitive control resolves conflicts between task-relevant and -irrelevant information to enable goal-directed behavior. As conflicts can arise from different sources (e.g., sensory input, internal representations), how a limited set of cognitive control processes can effectively address diverse conflicts remains a major challenge. Based on the cognitive space theory, different conflicts can be parameterized and represented as distinct points in a (low-dimensional) cognitive space, which can then be resolved by a limited set of cognitive control processes working along the dimensions. It leads to a hypothesis that conflicts similar in their sources are also represented similarly in the cognitive space. We designed a task with five types of conflicts that could be conceptually parameterized. Both human performance and fMRI activity patterns in the right dorsolateral prefrontal cortex support that different types of conflicts are organized based on their similarity, thus suggesting cognitive space as a principle for representing conflicts.

Memory for abstract control states does not decay with increasing retrieval delays

Recent studies have suggested that abstract control states (i.e., internal attentional states independent from concrete stimuli and responses) can be stored in episodic memory and retrieved subsequently. However, the duration of such a control state memory remains unclear. Previous research has found a quick and complete decay for stimulus-response bindings after 2000-5000 ms. Here, we tested a possible decay of control state bindings with retrieval delays of 2000, 3000, or 5000 ms. Five preregistered experiments used a confound-minimized prime-target task to measure the congruency sequence effect (CSE) separately for trials in which a nominally irrelevant context feature changed or repeated across trials. Analyses of the individual experiments did not result in conclusive evidence. A mega-analysis integrating the data of all experiments (Ntotal = 326) replicated evidence for binding and retrieval of control states, in that larger CSEs were found for context repetition trials. Importantly, Bayesian analysis indicated that this effect was not modulated by the length of retrieval delay. While this finding suggests that bindings of abstract control states can be relatively robust, we also discuss possible limitations of the present research.

Hippocampal mechanisms resolve competition in memory and perception

Behaving adaptively requires selection of relevant memories and sensations and suppression of competing ones. We hypothesized that these mechanisms are linked, such that hippocampal computations that resolve competition in memory also shape the precision of sensory representations to guide selective attention. We leveraged f MRI-based pattern similarity, receptive field modeling, and eye tracking to test this hypothesis in humans performing a memory-dependent visual search task. In the hippocampus, differentiation of competing memories predicted the precision of memory-guided eye movements. In visual cortex, preparatory coding of remembered target locations predicted search successes, whereas preparatory coding of competing locations predicted search failures due to interference. These effects were linked: stronger hippocampal memory differentiation was associated with lower competitor activation in visual cortex, yielding more precise preparatory representations. These results demonstrate a role for memory differentiation in shaping the precision of sensory representations, highlighting links between mechanisms that overcome competition in memory and perception.

Learning of irrelevant stimulus-response associations modulates cognitive control

It has been shown that manipulating the proportion of congruent to incongruent trials in conflict tasks (e.g., Stroop, Simon, and flanker tasks) can vary the size of conflict effects, however, by two different mechanisms. One theory is the control learning account (the brain learns the probability of conflict and uses it to proactively adjust the control demand for future trials). The other is the irrelevant stimulus-response learning account (the brain learns the probability of irrelevant stimulus-response associations and uses it to prepare responses). Previous fMRI studies have detected the brain regions that contribute to the control-learning-modulated conflict effects, but it is less known what neural substrates underlie the conflict effects modulated by irrelevant S-R learning. We here investigated this question with a model-based fMRI study, in which the proportion of congruent to incongruent trials changed dynamically in the Simon task and the models learned the probability of irrelevant S-R associations quantitatively. Behavioral analyses showed that the unsigned prediction errors (PEs) of responses generated by the learning models correlated with reaction times irrespective of congruent and incongruent trials, indicating that large unsigned PEs associated with slow responses. The fMRI results showed that the regions of fronto-parietal and cingulo-opercular network involved in cognitive control were significantly modulated by the unsigned PEs, also irrespective of congruent and incongruent trials, indicating that large unsigned PEs associated with transiently increased activity in these regions. These results together suggest that learning of irrelevant S-R associations modulates reactive control, which demonstrates a new way to modulate cognitive control compared to the control learning account.

Representational integration and differentiation in the human hippocampus following goal-directed navigation

As we learn, dynamic memory processes build structured knowledge across our experiences. Such knowledge enables the formation of internal models of the world that we use to plan, make decisions, and act. Recent theorizing posits that mnemonic mechanisms of differentiation and integration - which at one level may seem to be at odds - both contribute to the emergence of structured knowledge. We tested this possibility using fMRI as human participants learned to navigate within local and global virtual environments over the course of 3 days. Pattern similarity analyses on entorhinal cortical and hippocampal patterns revealed evidence that differentiation and integration work concurrently to build local and global environmental representations, and that variability in integration relates to differences in navigation efficiency. These results offer new insights into the neural machinery and the underlying mechanisms that translate experiences into structured knowledge that allows us to navigate to achieve goals.

Neural distinctiveness and reinstatement of hippocampal representations support unitization for associations

The medial temporal lobe (MTL) is critical to associative memory success, yet not all types of associations may be processed in a similar manner within MTL subregions. In particular, previous work suggests intra- and inter-item associations not only exhibit differences in overall rates of recollection, but also recruit different MTL subregions. Whereas intra-item associations, akin to unitization, take advantage of associations between within-item features, inter-item associations form links across discrete items. The current work examines the neural differences between these two types of associations using fMRI and multivoxel analyses. Specifically, the current study examines differences across face-occupation as a function of whether the pairing was viewed as a person performing the given job (intra-item binding) or a person saying they knew someone who had a particular job (inter-item binding). The results show that at encoding, successfully recollected neural patterns related to intra- and inter-item associations are distinct from one another in the hippocampus, parahippocampal and perirhinal cortex. Additionally, the two trial types are reinstated distinctly such that inter-item trials have higher neural reinstatement from encoding to retrieval compared to intra-item trials in the hippocampus. We conclude that intra- and inter- associative pairs may utilize similar neural regions that represent patterns of activation differentially at encoding. However, to reinstate information to the same degree (i.e., subsequently successfully recollected) inter-item associations, that are all encoded in the same manner, may be reinstated more similarly compared to intra-item associations that are encoded by imagining pairs differently and occupation specific. This may indicate that intra-item associations promote more efficient reinstatement.

Episodic future thinking predicts differences in delay discounting: The mediating role of hippocampal structure

A growing body of evidence suggests that engagement in episodic future thinking (EFT) could reduce delay discounting rates. However, little is known about whether individual differences in the ability of EFT are associated with differences in delay discounting in young adults. In the present study, this association was tested in healthy college students (n = 106, 19.98 ± 1.56 years), and the neural basis underlying this association was also examined using voxel-based morphometry (VBM) method. Behavioral analysis indicated that individual differences in EFT ability can significantly negatively predict discounting rates. VBM analysis first revealed that the EFT score positively correlated with gray matter volume (GMV) of a cluster in hippocampus, while negatively correlated with GMV of a cluster in rostral anterior cingulate cortex. We also found the GMV of a cluster in the mPFC was positively correlated with delay discounting. ROI analysis further revealed that individual differences in delay discounting could be reliably predicted by the GMV in the hippocampus and mPFC. The final mediation analysis showed that the GMV of the hippocampus plays a significant mediating role in the association between EFT and delay discounting, and the indirect effect of the hippocampal GMV accounts for 33.2% of the total effect. Our results suggest that individuals’ EFT ability may be an important determinant of differences in delay discounting, and highlight the hippocampal structure as a neural biomarker for explaining the association between EFT ability and delay discounting.

Readiness to remember: predicting variability in episodic memory

Learning and remembering are fundamental to our lives, so what causes us to forget? Answers often highlight preparatory processes that precede learning, as well as mnemonic processes during the act of encoding or retrieval. Importantly, evidence now indicates that preparatory processes that precede retrieval attempts also have powerful influences on memory success or failure. Here, we review recent work from neuroimaging, electroencephalography, pupillometry, and behavioral science to propose an integrative framework of retrieval-period dynamics that explains variance in remembering in the moment and across individuals as a function of interactions among preparatory attention, goal coding, and mnemonic processes. Extending this approach, we consider how a 'readiness to remember' (R2R) framework explains variance in high-level functions of memory and mnemonic disruptions in aging.

Encoding contexts are incidentally reinstated during competitive retrieval and track the temporal dynamics of memory interference

The ability to remember an episode from our past is often hindered by competition from similar events. For example, if we want to remember the article a colleague recommended during the last lab meeting, we may need to resolve interference from other article recommendations from the same colleague. This study investigates if the contextual features specifying the encoding episodes are incidentally reinstated during competitive memory retrieval. Competition between memories was created through the AB/AC interference paradigm. Individual word-pairs were presented embedded in a slowly drifting real-word-like context. Multivariate pattern analysis (MVPA) of high temporal-resolution electroencephalographic (EEG) data was used to investigate context reactivation during memory retrieval. Behaviorally, we observed proactive (but not retroactive) interference; that is, performance for AC competitive retrieval was worse compared with a control DE noncompetitive retrieval, whereas AB retrieval did not suffer from competition. Neurally, proactive interference was accompanied by an early reinstatement of the competitor context and interference resolution was associated with the ensuing reinstatement of the target context. Together, these findings provide novel evidence showing that the encoding contexts of competing discrete events are incidentally reinstated during competitive retrieval and that such reinstatement tracks retrieval competition and subsequent interference resolution.

Rapid neural reorganization during retrieval practice predicts subsequent long-term retention and false memory

Active retrieval can alter the strength and content of a memory, yielding either enhanced or distorted subsequent recall. However, how consolidation influences these retrieval-induced seemingly contradictory outcomes remains unknown. Here we show that rapid neural reorganization over an eight-run retrieval practice predicted subsequent recall. Retrieval practice boosted memory retention following a 24-hour (long-term) but not 30-minute delay, and increased false memory at both delays. Long-term retention gains were predicted by multi-voxel representation distinctiveness in the posterior parietal cortex (PPC) that increased progressively over retrieval practice. False memory was predicted by unstable representation distinctiveness in the medial temporal lobe (MTL). Retrieval practice enhanced the efficiency of memory-related brain networks, through building up PPC and MTL connections with the ventrolateral and dorsolateral prefrontal cortex that predicted long-term retention gains and false memory, respectively. Our findings indicate that retrieval-induced rapid neural reorganization together with consecutive consolidation fosters long-term retention and false memories via distinct pathways.

Further Evidence for the Binding and Retrieval of Control-States From the Flanker Task

In response-interference tasks, congruency effects are reduced in trials that follow an incongruent trial. This congruence sequence effect (CSE) has been taken to reflect top-down cognitive control processes that monitor for and intervene in case of conflict. In contrast, episodic-memory accounts explain CSEs with bottom-up retrieval of stimulus-response links. Reconciling these opposing views, an emerging perspective holds that memory stores instances of control - abstract control-states - creating a shortcut for effortful control processes. Support comes from a study that assessed CSEs in a prime-target task. Here, repeating an irrelevant context feature boosted CSEs, possibly by retrieving previously stored control-states. We present a conceptual replication using the Eriksen flanker task because previous research found that CSEs in the flanker task reflect different control mechanisms than CSEs in the prime-target task. We measured CSEs while controlling for stimulus-response memory effects and manipulated contextual information (vertical spatial location) independently from the stimulus information, which introduced the conflict (horizontal spatial location). Results replicate previous findings - CSEs increased for context-repetition compared to context-changes. This study shows that retrieval of control-states is not limited to a specific task or context feature and therefore generalizes the notion that abstract control parameters are stored into trial-specific event files.

Abrupt hippocampal remapping signals resolution of memory interference

Remapping refers to a decorrelation of hippocampal representations of similar spatial environments. While it has been speculated that remapping may contribute to the resolution of episodic memory interference in humans, direct evidence is surprisingly limited. We tested this idea using high-resolution, pattern-based fMRI analyses. Here we show that activity patterns in human CA3/dentate gyrus exhibit an abrupt, temporally-specific decorrelation of highly similar memory representations that is precisely coupled with behavioral expressions of successful learning. The magnitude of this learning-related decorrelation was predicted by the amount of pattern overlap during initial stages of learning, with greater initial overlap leading to stronger decorrelation. Finally, we show that remapped activity patterns carry relatively more information about learned episodic associations compared to competing associations, further validating the learning-related significance of remapping. Collectively, these findings establish a critical link between hippocampal remapping and episodic memory interference and provide insight into why remapping occurs.

Age-Related Structural and Functional Changes of the Hippocampus and the Relationship with Inhibitory Control

Aging is associated with structural and functional changes in the hippocampus, and hippocampal dysfunction represents a risk marker of Alzheimer’s disease. Previously, we demonstrated age-related changes in reactive and proactive control in the stop signal task, each quantified by the stop signal reaction time (SSRT) and sequential effect computed as the correlation between the estimated stop signal probability and go trial reaction time. Age was positively correlated with the SSRT, but not with the sequential effect. Here, we explored hippocampal gray matter volume (GMV) and activation to response inhibition and to p(Stop) in healthy adults 18 to 72 years of age. The results showed age-related reduction of right anterior hippocampal activation during stop success vs. go trials, and the hippocampal activities correlated negatively with the SSRT. In contrast, the right posterior hippocampus showed higher age-related responses to p(Stop), but the activities did not correlate with the sequential effect. Further, we observed diminished GMVs of the anterior and posterior hippocampus. However, the GMVs were not related to behavioral performance or regional activities. Together, these findings suggest that hippocampal GMVs and regional activities represent distinct neural markers of cognitive aging, and distinguish the roles of the anterior and posterior hippocampus in age-related changes in cognitive control.