Lingering Cognitive States Shape Fundamental Mnemonic Abilities

The persistence of memory: prior memory responses modulate behavior and brain state engagement

Memory brain states may influence how we experience an event. Memory encoding and retrieval constitute neurally dissociable brain states that individuals can selectively engage based on top-down goals. To the extent that memory states linger in time - as suggested by prior behavioral work - memory states may influence not only the current experience, but also subsequent stimuli and judgments. Thus lingering memory states may have broad influences on cognition, yet this account has not been directly tested utilizing neural measures of memory states. Here we address this gap by testing the hypothesis that memory brain states are modulated by memory judgments, and that these brain states persist for several hundred milliseconds. We recorded scalp electroencephalography (EEG) while participants completed a recognition memory task. We used an independently validated multivariate mnemonic state classifier to assess memory state engagement. We replicate prior behavioral findings; however, our neural findings run counter to the predictions made on the basis of the behavioral data. Surprisingly, we find that prior responses modulate current memory state engagement on the basis of response congruency. That is, we find strong engagement of the retrieval state on incongruent trials - when a target is preceded by a correct rejection of a lure and when a lure is preceded by successful recognition of a target. These findings indicate that cortical brain states are influenced by prior judgments and suggest that a non-mnemonic, internal attention state may be recruited to in the face of changing demands in a dynamic environment.

Top-down task goals induce the retrieval state

Engaging the retrieval state (Tulving, 1983) impacts processing and behavior (Long & Kuhl, 2019, 2021; Smith, Moore, & Long, 2022), but the extent to which top-down factors - explicit instructions and goals - vs. bottom-up factors - stimulus properties such as repetition and similarity - jointly or independently induce the retrieval state is unclear. Identifying the impact of bottom-up and top-down factors on retrieval state engagement is critical for understanding how control of task-relevant vs. task-irrelevant brain states influence cognition. We conducted between-subjects recognition memory tasks on male and female human participants in which we varied test phase goals. We recorded scalp electroencephalography and used an independently validated mnemonic state classifier (Long, 2023) to measure retrieval state engagement as a function of top-down task goals (recognize old vs. detect new items) and bottom-up stimulus repetition (hits vs. correct rejections). We find that whereas the retrieval state is engaged for hits regardless of top-down goals, the retrieval state is only engaged during correct rejections when the top-down goal is to recognize old items. Furthermore, retrieval state engagement is greater for low compared to high confidence hits when the task goal is to recognize old items. Together, these results suggest that top-down demands to recognize old items induce the retrieval state independent from bottom-up factors, potentially reflecting the recruitment of internal attention to enable access of a stored representation.

Significance Statement

Both top-down goals and automatic bottom-up influences may lead us into a retrieval brain state - a whole-brain pattern of activity that supports our ability to remember the past. Here we tested the extent to which top-down vs. bottom-up factors independently influence the retrieval state by manipulating participants' goals and stimulus repetition during a memory test. We find that in response to the top-down goal to recognize old items, the retrieval state is engaged for both old and new probes, suggesting that top-down and bottom-up factors independently engage the retrieval state. Our interpretation is that top-down demands recruit internal attention in service of the attempt to access a stored representation.

Spatiotemporal Dynamics of Memory Encoding and Memory Retrieval States

Memory encoding and memory retrieval are neurally distinct brain states that can be differentiated on the basis of cortical network activity. However, it is unclear whether sustained engagement of one network or fluctuations between multiple networks give rise to these memory states. The spatiotemporal dynamics of memory states may have important implications for memory behavior and cognition; however, measuring temporally resolved signals of cortical networks poses a challenge. Here, we recorded scalp electroencephalography from participants performing a mnemonic state task in which they were biased toward memory encoding or retrieval. We performed a microstate analysis to measure the temporal dynamics of cortical networks throughout this mnemonic state task. We find that Microstate E, a putative analog of the default mode network, shows temporally sustained dissociations between memory encoding and retrieval, with greater engagement during retrieve compared with encode trials. We further show that decreased engagement of Microstate E is a general property of encoding, rather than a reflection of retrieval suppression. Thus, memory success, as well as cognition more broadly, may be influenced by the ability to engage or disengage Microstate E in a goal-dependent manner.

Effects of familiar music exposure on deliberate retrieval of remote episodic and semantic memories in healthy aging adults

Familiar music facilitates memory retrieval in adults with dementia. However, mechanisms behind this effect, and its generality, are unclear because of a lack of parallel work in healthy aging. Exposure to familiar music enhances spontaneous recall of memories directly cued by the music, but it is unknown whether such effects extend to deliberate recall more generally - e.g., to memories not directly linked to the music being played. It is also unclear whether familiar music boosts recall of specific episodes versus more generalised semantic memories, or whether effects are driven by domain-general mechanisms (e.g., improved mood). In a registered report study, we examined effects of familiar music on deliberate recall in healthy adults ages 65-80 years (N = 75) by presenting familiar music from earlier in life, unfamiliar music, and non-musical audio clips across three sessions. After each clip, we assessed free recall of remote memories for pre-selected events. Contrary to our hypotheses, we found no effects of music exposure on recall of prompted events, though familiar music evoked spontaneous memories most often. These results suggest that effects of familiar music on recall may be limited to memories specifically evoked in response to the music (Preprint and registered report protocol at https://osf.io/kjnwd/).

Hippocampal convergence during anticipatory midbrain activation promotes subsequent memory formation

The hippocampus has been a focus of memory research since H.M's surgery abolished his ability to form new memories, yet its mechanistic role in memory remains debated. Here, we identify a candidate memory mechanism: an anticipatory hippocampal "convergence state", observed while awaiting valuable information, and which predicts subsequent learning. During fMRI, participants viewed trivia questions eliciting high or low curiosity, followed seconds later by its answer. We reasoned that encoding success requires a confluence of conditions, so that hippocampal states more conducive to memory formation should converge in state space. To operationalize convergence of neural states, we quantified the typicality of multivoxel patterns in the medial temporal lobes during anticipation and encoding of trivia answers. We found that the typicality of anticipatory hippocampal patterns increased during high curiosity. Crucially, anticipatory hippocampal pattern typicality increased with dopaminergic midbrain activation and uniquely accounted for the association between midbrain activation and subsequent recall. We propose that hippocampal convergence states may complete a cascade from motivation and midbrain activation to memory enhancement, and may be a general predictor of memory formation.

Readiness to remember: predicting variability in episodic memory

Learning and remembering are fundamental to our lives, so what causes us to forget? Answers often highlight preparatory processes that precede learning, as well as mnemonic processes during the act of encoding or retrieval. Importantly, evidence now indicates that preparatory processes that precede retrieval attempts also have powerful influences on memory success or failure. Here, we review recent work from neuroimaging, electroencephalography, pupillometry, and behavioral science to propose an integrative framework of retrieval-period dynamics that explains variance in remembering in the moment and across individuals as a function of interactions among preparatory attention, goal coding, and mnemonic processes. Extending this approach, we consider how a 'readiness to remember' (R2R) framework explains variance in high-level functions of memory and mnemonic disruptions in aging.

How associative thinking influences scene perception

Perception of our external environment is not isolated from the influence of our internal thoughts, and past evidence points to a possible common associative mechanism underlying both the perception of scenes and our internal thought. Here, we investigated the nature of the interaction between an associative mindset and scene perception, hypothesizing a functional advantage to an associative thought pattern in the perception of scenes. Experiments 1 and 2 showed that associative thinking facilitates scene perception, which evolved over the course of the experiments. In contrast to scene perception, Experiment 3 showed that associative thinking hinders the perception of mundane objects, in which associative information is minimized. Nevertheless, object perception was facilitated when associative thinking was reduced. This double dissociation suggests that an associative mind is more receptive of externally perceived associative information, and that a match between the orientation of internal and external processing may be key for perception.

A neural network model of when to retrieve and encode episodic memories

Recent human behavioral and neuroimaging results suggest that people are selective in when they encode and retrieve episodic memories. To explain these findings, we trained a memory-augmented neural network to use its episodic memory to support prediction of upcoming states in an environment where past situations sometimes reoccur. We found that the network learned to retrieve selectively as a function of several factors, including its uncertainty about the upcoming state. Additionally, we found that selectively encoding episodic memories at the end of an event (but not mid-event) led to better subsequent prediction performance. In all of these cases, the benefits of selective retrieval and encoding can be explained in terms of reducing the risk of retrieving irrelevant memories. Overall, these modeling results provide a resource-rational account of why episodic retrieval and encoding should be selective and lead to several testable predictions.

Cortical Representations of Visual Stimuli Shift Locations with Changes in Memory States

Episodic memory retrieval is thought to rely on reactivation of the same content-sensitive neural activity patterns initially expressed during memory encoding.^1-6 Yet there are emerging examples of content representations expressed in different brain regions during encoding versus retrieval.^7-14 Although these differences have been observed by comparing encoding and retrieval tasks that differ in terms of perceptual experience and cognitive demands, there are many real-world contexts-e.g., meeting a new colleague who reminds you of an old acquaintance-where the memory system might be intrinsically biased either toward encoding (the new colleague) or retrieval (the old acquaintance).¹⁵¹⁶ Here, we test whether intrinsic memory states, independent of task demands, determine the cortical location of content representations. In a human fMRI study, subjects (n = 33) viewed object images and were instructed to either encode the current object or retrieve a similar object from memory. Using pattern classifiers, we show that biases toward encoding versus retrieval were reflected in large-scale attentional networks.^17-19 Critically, memory states decoded from these networks-even when entirely independent from task instructions-predicted shifts of object representations from visual cortex (encoding) to ventral parietal cortex (retrieval). Finally, visual versus ventral parietal cortices exhibited differential connectivity with the hippocampus during memory encoding versus retrieval, consistent with the idea that the hippocampus mediates cortical shifts in content representations. Collectively, these findings demonstrate that intrinsic biases toward memory encoding versus retrieval determine the specific cortical locations that express content information.

When the Memory System Gets Ahead of Itself

Humans are adept at learning and exploiting statistical regularities to predict future events from current experience. A recent paper by Sherman and Turk-Browne demonstrates that statistical regularities bias the hippocampus toward representing future states over current experience and reduce the degree to which current experience is encoded into memory.

Preparation for upcoming attentional states in the hippocampus and medial prefrontal cortex

Goal-directed attention is usually studied by providing individuals with explicit instructions on what they should attend to. But in daily life, we often use past experiences to guide our attentional states. Given the importance of memory for predicting upcoming events, we hypothesized that memory-guided attention is supported by neural preparation for anticipated attentional states. We examined preparatory coding in the human hippocampus and mPFC, two regions that are important for memory-guided behaviors, in two tasks: one where attention was guided by memory and another in which attention was explicitly instructed. Hippocampus and mPFC exhibited higher activity for memory-guided vs. explicitly instructed attention. Furthermore, representations in both regions contained information about upcoming attentional states. In the hippocampus, this preparation was stronger for memory-guided attention, and occurred alongside stronger coupling with visual cortex during attentional guidance. These results highlight the mechanisms by which memories are used to prepare for upcoming attentional goals.

Modulating the Use of Multiple Memory Systems in Value-based Decisions with Contextual Novelty

With multiple learning and memory systems at its disposal, the human brain can represent the past in many ways, from extracting regularities across similar experiences (incremental learning) to storing rich, idiosyncratic details of individual events (episodic memory). The unique information carried by these neurologically distinct forms of memory can bias our behavior in different directions, raising crucial questions about how these memory systems interact to guide choice and the factors that cause one to dominate. Here, we devised a new approach to estimate how decisions are independently influenced by episodic memories and incremental learning. Furthermore, we identified a biologically motivated factor that biases the use of different memory types-the detection of novelty versus familiarity. Consistent with computational models of cholinergic memory modulation, we find that choices are more influenced by episodic memories following the recognition of an unrelated familiar image but more influenced by incrementally learned values after the detection of a novel image. Together this work provides a new behavioral tool enabling the disambiguation of key memory behaviors thought to be supported by distinct neural systems while also identifying a theoretically important and broadly applicable manipulation to bias the arbitration between these two sources of memories.

Decoding the tradeoff between encoding and retrieval to predict memory for overlapping events

When new events overlap with past events, there is a natural tradeoff between encoding the new event and retrieving the past event. Given the ubiquity of overlap among memories, this tradeoff between memory encoding and retrieval is of central importance to computational models of episodic memory (O'Reilly & McClelland 1994; Hasselmo 2005). However, prior studies have not directly linked neural markers of encoding/retrieval tradeoffs to behavioral measures of how overlapping events are remembered. Here, by decoding patterns of scalp electroencephalography (EEG) from male and female human subjects, we show that tradeoffs between encoding and retrieval states are reflected in distributed patterns of neural activity and, critically, these neural tradeoffs predict how overlapping events will later be remembered. Namely, new events that overlapped with past events were more likely to be subsequently remembered if neural patterns were biased toward a memory encoding state-or, conversely, away from a retrieval state. Additionally, we show that neural markers of encoding vs. retrieval states are surprisingly independent from previously-described EEG predictors of subsequent memory. Instead, we demonstrate that previously-described EEG predictors of subsequent memory are better explained by task engagement than by memory encoding, per se. Collectively, our findings provide important insight into how the memory system balances memory encoding and retrieval states and, more generally, into the neural mechanisms that support successful memory formation.

Hippocampal representations as a function of time, subregion, and brain state

How does the hippocampus represent interrelated experiences in memory? We review prominent yet seemingly contradictory theoretical perspectives, which propose that the hippocampus distorts experiential representations to either emphasize their distinctiveness or highlight common elements. These fundamentally different kinds of memory representations may be instantiated in the brain via conjunctive separated codes and adaptively differentiated codes on the one hand, or integrated relational codes on the other. After reviewing empirical support for these different coding schemes within the hippocampus, we outline two organizing principles which may explain the conflicting findings in the literature. First focusing on where the memories are formed and stored, we argue that distinct hippocampal regions represent experiences at multiple levels of abstraction and may transmit them to distinct cortical networks. Then focusing on when memories are formed, we identify several factors that can open and maintain specialized time windows, during which the very same hippocampal network is biased toward one coding scheme over the others. Specifically, we discuss evidence for (1) excitability-mediated integration windows, maintained by persistently elevated CREB levels following encoding of a specific memory, (2) fleeting cholinergically-mediated windows favoring memory separation, and (3) sustained dopaminergically-mediated windows favoring memory integration. By presenting a broad overview of different hippocampal coding schemes across species, we hope to inspire future empirical and modeling research to consider how factors surrounding memory formation shape the representations in which they are stored.