The nature of recollection in behavior and the brain

Tuned geometries of hippocampal representations meet the computational demands of social memory

Social memory consists of two processes: the detection of familiar compared with novel conspecifics and the detailed recollection of past social episodes. We investigated the neural bases for these processes using calcium imaging of dorsal CA2 hippocampal pyramidal neurons, known to be important for social memory, during social/spatial encounters with novel conspecifics and familiar littermates. Whereas novel individuals were represented in a low-dimensional geometry that allows for generalization of social identity across different spatial locations and of location across different identities, littermates were represented in a higher-dimensional geometry that supports high-capacity memory storage. Moreover, familiarity was represented in an abstract format, independent of individual identity. The degree to which familiarity increased the dimensionality of CA2 representations for individual mice predicted their performance in a social novelty recognition memory test. Thus, by tuning the geometry of structured neural activity, CA2 is able to meet the demands of distinct social memory processes.

Neural reactivation and judgements of vividness reveal separable contributions to mnemonic representation

Mnemonic representations vary in fidelity, sharpness, and strength-qualities that can be examined using both introspective judgements of mental states and objective measures of brain activity. Subjective and objective measures are both valid ways of "reading out" the content of someone's internal mnemonic states, each with different strengths and weaknesses. St-Laurent and colleagues (2015) compared the neural correlates of memory vividness ratings with patterns of neural reactivation evoked during memory recall and found considerable overlap between the two, suggesting a common neural basis underlying these different markers of representational quality. Here we extended this work with meta-analytic methods by pooling together four neuroimaging datasets in order to contrast the neural substrates of neural reactivation and those of vividness judgements. While reactivation and vividness judgements correlated positively with one another and were associated with common univariate activity in the dorsal attention network and anterior hippocampus, some notable differences were also observed. Vividness judgments were tied to stronger activation in the striatum and dorsal attention network, together with activity suppression in default mode network nodes. We also observed a trend for reactivation to be more closely associated with early visual cortex activity. A mediation analysis found support for the hypothesis that neural reactivation is necessary for memory vividness, with activity in the anterior hippocampus associated with greater reactivation. Our results suggest that neural reactivation and vividness judgements reflect common mnemonic processes but differ in the extent to which they engage effortful, attentional processes. Additionally, the similarity between reactivation and vividness appears to arise, partly, through hippocampal engagement during memory retrieval.

A computational model of familiarity detection for natural pictures, abstract images, and random patterns: Combination of deep learning and anti-Hebbian training

We present a neural network model for familiarity recognition of different types of images in the perirhinal cortex (the FaRe model). The model is designed as a two-stage system. At the first stage, the parameters of an image are extracted by a pretrained deep learning convolutional neural network. At the second stage, a two-layer feed forward neural network with anti-Hebbian learning is used to make the decision about the familiarity of the image. FaRe model simulations demonstrate high capacity of familiarity recognition memory for natural pictures and low capacity for both abstract images and random patterns. These findings are in agreement with psychological experiments.

Imaging recollection, familiarity, and novelty in the frontoparietal control and default mode networks and the anterior-posterior medial temporal lobe: An integrated view and meta-analysis

A network-level model of recollection-based recognition (R), familiarity-based recognition (F), and novelty recognition (N) was constructed, and its validity was evaluated through meta-analyses to produce an integrated view of neuroimaging data. The model predicted the following: (a) the overall magnitude of the frontoparietal control network (FPCN) activity (which supports retrieval and decision effort) is in the order of F > R > N; (b) that of the posterior medial temporal network (MTL) activity (which plays a direct role in retrieval) is in the order of R > N > F; (c) that of the anterior MTL activity (which supports novelty-encoding) is in the order of N > R > F; (d) that of the default mode network (DMN) activity (which supports the subjective experience of remembering) is in the order of R > N > F. The meta-analyses results were consistent with these predictions. Subsystem analysis indicated a functional dissociation between the cingulo-opercular vs. frontoparietal components of the FPCN and between the core vs. medial temporal components of the DMN.

Thalamic Functional Connectivity during Spatial Long-Term Memory and the Role of Sex

The thalamus has been implicated in many cognitive processes, including long-term memory. More specifically, the anterior (AT) and mediodorsal (MD) thalamic nuclei have been associated with long-term memory. Despite extensive mapping of the anatomical connections between these nuclei and other brain regions, little is known regarding their functional connectivity during long-term memory. The current study sought to determine which brain regions are functionally connected to AT and MD during spatial long-term memory and whether sex differences exist in the patterns of connectivity. During encoding, abstract shapes were presented to the left and right of fixation. During retrieval, shapes were presented at fixation, and participants made an “old-left” or “old-right” judgment. Activations functionally connected to AT and MD existed in regions with known anatomical connections to each nucleus as well as in a broader network of long-term memory regions. Sex differences were identified in a subset of these regions. A targeted region-of-interest analysis identified anti-correlated activity between MD and the hippocampus that was specific to females, which is consistent with findings in rodents. The current results suggest that AT and MD play key roles during spatial long-term memory and suggest that these functions may be sex specific.

The impact of focused attention on subsequent emotional recollection: A functional MRI investigation

In his seminal works, Endel Tulving argued that functionally distinct memory systems give rise to subjective experiences of remembering and knowing (i.e., recollection- vs. familiarity-based memory, respectively). Evidence shows that emotion specifically enhances recollection, and this effect is subserved by a synergistic mechanism involving the amygdala (AMY) and hippocampus (HC). In extreme circumstances, however, uncontrolled recollection of highly distressing memories may lead to symptoms of affective disorders. Therefore, it is important to understand the factors that can diminish such detrimental effects. Here, we investigated the effects of Focused Attention (FA) on emotional recollection. FA is an emotion regulation strategy that has been proven quite effective in reducing the impact of emotional responses associated with the recollection of distressing autobiographical memories, but its impact during emotional memory encoding is not known. Functional MRI and eye-tracking data were recorded while participants viewed a series of composite negative and neutral images with distinguishable foreground (FG) and background (BG) areas. Participants were instructed to focus either on the FG or BG content of the images and to rate their emotional responses. About 4 days later, participants' memory was assessed using the R/K procedure, to indicate whether they Recollected specific contextual details about the encoded images or the images were just familiar to them - i.e., participants only Knew that they saw the pictures without being able to remember specific contextual details. First, results revealed that FA was successful in decreasing memory for emotional pictures viewed in BG Focus condition, and this effect was driven by recollection-based retrieval. Second, the BG Focus condition was associated with decreased activity in the AMY, HC, and anterior parahippocampal gyrus for subsequently recollected emotional items. Moreover, correlation analyses also showed that reduced activity in these regions predicted greater reduction in emotional recollection following FA. These results demonstrate the effectiveness of FA in mitigating emotional experiences and emotional recollection associated with unpleasant emotional events.

The core episodic simulation network dissociates as a function of subjective experience and objective content

Episodic simulation - the mental construction of a possible future event - has been consistently associated with enhanced activity in a set of neural regions referred to as the core network. In the current functional neuroimaging study, we assessed whether members of the core network are differentially associated with the subjective experience of future events (i.e., vividness) versus the objective content comprising those events (i.e., the amount of episodic details). During scanning, participants imagined future events in response to object cues. On each trial, participants rated the subjective vividness associated with each future event. Participants completed a post-scan interview where they viewed each object cue from the scanner and verbally reported whatever they had thought about. For imagined events, we quantified the number of episodic or internal details in accordance with the Autobiographical Interview (i.e., who, what, when, and where details of each central event). To test whether core network regions are differentially associated with subjective experience or objective episodic content, imagined future events were sorted as a function of their rated vividness or the amount of episodic detail. Univariate analyses revealed that some regions of the core network were uniquely sensitive to the vividness of imagined future events, including the hippocampus (i.e., high > low vividness), whereas other regions, such as the lateral parietal cortex, were sensitive to the amount of episodic detail in the event (i.e., high > low episodic details). The present results indicate that members of the core network support distinct episodic simulation-related processes.

Compensation of Trial-to-Trial Latency Jitter Reveals the Parietal Retrieval Success Effect to be Both Variable and Thresholded in Older Adults

Although the neural mechanism supporting episodic recollection has been well characterized in younger adults, exactly how recollection is supported in older adults remains unclear. The electrophysiological correlate of recollection—the parietal retrieval success effect—for example, has been shown to be sensitive to both the amount of information recollected and the accuracy of remembered information in younger adults. To date, there is mixed evidence that parietal effect also scales with the amount of information remembered in older adults whilst there is little evidence that the same mechanism is sensitive to the accuracy of recollected information. Here, we address one potential concern when investigating Event Related Potentials (ERPs) among older adults—namely, the greater potential for single-trial latency variability to smear and reduces the amplitudes of averaged ERPs. We apply a well-established algorithm for correcting single-trial latency variability, Residual Iteration Decomposition Analysis (RIDE), to investigate whether the parietal retrieval success effect among older adults is sensitive to retrieval accuracy. Our results reveal that similar to younger adults, older adult parietal retrieval success effects scale with the accuracy of recollected information—i.e., is greater in magnitude when recollected information is of high accuracy, reduced in magnitude when accuracy is low, and entirely absent when guessing. The results help clarify the functional significance of the neural mechanism supporting recollection in older adults whilst also highlighting the potential issues with interpreting average ERPs in older adult populations.

Exercise Similarly Facilitates Men and Women's Selective Attention Task Response Times but Differentially Affects Memory Task Performance

Previous research has found that acute, moderate-intensity physical exercise enhances selective attention and memory and that men and women show differential performance on tasks measuring these skills. Although exercise and participant sex have been examined separately, it remains unknown whether acute, moderate-intensity exercise differentially affects men and women’s selective attention and memory encoding and retrieval. Participants in the present study completed two 10-min sessions of either moderate-intensity exercise comprised of jumping rope alternating with walking in place or an active control protocol comprised of watching wellness videos alternating with walking in place. Each participant completed a selective attention task and a task assessing recognition and object location memory immediately after exercising. Exercise was related to overall faster performance during the selective attention task, with no differences in men and women’s performance. Women showed better recognition memory compared to men. Exercise specifically improved object location memory among men, but only among participants who completed the memory task second. These findings suggest that acute, moderate-intensity exercise differentially affects men and women’s memory, which may be related to complex interactions between exercise, sex hormones, and the neurotrophin BDNF.

Graded expression of source memory revealed by analysis of gaze direction

During source memory studies, knowledge of some detail about the context of a previously experienced item or event is tested. Here, participants attended to different objects presented at different quadrants on a screen. In a later test phase, a single object was presented in all four quadrants, and participants verbally reported whether the object was new or previously seen (item recognition), and if it was previously seen, they indicated the original screen location (source memory). We combined this test with eye-tracking to determine whether attention to an object during encoding would correlate with later recognition of the object and memory of its source location, and whether eye movements at test can reveal attention to the correct source location in the absence of correct explicit verbal responses. The amount of time spent looking at an object during encoding was not related to later object recognition or source recollection. However, we found that eye movements at test reveal retention of source information about an object in the absence of accurate retrieval of source information as assessed by verbal response. When participants correctly recognized an object but incorrectly indicated the source information, significantly more time was spent looking at the correct source location than to incorrect, non-selected locations. Moreover, when participants correctly recognized an object but said they could not remember the source information, significantly more time was spent looking at the correct source location. These results are consistent with the hypothesis that eye movements are sensitive to attention or other graded mental processes which can underlie the retrieval of source memories that can then be expressed verbally in a thresholded manner.

False memory for context and true memory for context similarly activate the parahippocampal cortex

The role of the parahippocampal cortex is currently a topic of debate. One view posits that the parahippocampal cortex specifically processes spatial layouts and sensory details (i.e., the visual-spatial processing view). In contrast, the other view posits that the parahippocampal cortex more generally processes spatial and non-spatial contexts (i.e., the general contextual processing view). A large number of studies have found that true memories activate the parahippocampal cortex to a greater degree than false memories, which would appear to support the visual-spatial processing view as true memories are typically associated with greater visual-spatial detail than false memories. However, in previous studies, contextual details were also greater for true memories than false memories. Thus, such differential activity in the parahippocampal cortex may have reflected differences in contextual processing, which would challenge the visual-spatial processing view. In the present functional magnetic resonance imaging (fMRI) study, we employed a source memory paradigm to investigate the functional role of the parahippocampal cortex during true memory and false memory for contextual information to distinguish between the visual-spatial processing view and the general contextual processing view. During encoding, abstract shapes were presented to the left or right of fixation. During retrieval, old shapes were presented at fixation and participants indicated whether each shape was previously on the "left" or "right" followed by an "unsure", "sure", or "very sure" confidence rating. The conjunction of confident true memories for context and confident false memories for context produced activity in the parahippocampal cortex, which indicates that this region is associated with contextual processing. Furthermore, the direct contrast of true memory and false memory produced activity in the visual cortex but did not produce activity in the parahippocampal cortex. The present evidence suggests that the parahippocampal cortex is associated with general contextual processing rather than only being associated with visual-spatial processing.

Decoding the content of recollection within the core recollection network and beyond

Recollection - retrieval of qualitative information about a past event - is associated with enhanced neural activity in a consistent set of neural regions (the 'core recollection network') seemingly regardless of the nature of the recollected content. Here, we employed multi-voxel pattern analysis (MVPA) to assess whether retrieval-related functional magnetic resonance imaging (fMRI) activity in core recollection regions - including the hippocampus, angular gyrus, medial prefrontal cortex, retrosplenial/posterior cingulate cortex, and middle temporal gyrus - contain information about studied content and thus demonstrate retrieval-related 'reinstatement' effects. During study, participants viewed objects and concrete words that were subjected to different encoding tasks. Test items included studied words, the names of studied objects, or unstudied words. Participants judged whether the items were recollected, familiar, or new by making 'remember', 'know', and 'new' responses, respectively. The study history of remembered test items could be reliably decoded using MVPA in most regions, as well as from the dorsolateral prefrontal cortex, a region where univariate recollection effects could not be detected. The findings add to evidence that members of the core recollection network, as well as at least one neural region where mean signal is insensitive to recollection success, carry information about recollected content. Importantly, the study history of recognized items endorsed with a 'know' response could be decoded with equal accuracy. The results thus demonstrate a striking dissociation between mean signal and multi-voxel indices of recollection. Moreover, they converge with prior findings in suggesting that, as it is operationalized by classification-based MVPA, reinstatement is not uniquely a signature of recollection.

Spatial Memory Activity Distributions Indicate the Hippocampus Operates in a Continuous Manner

There is a long-standing debate as to whether recollection is a continuous/graded process or a threshold/all-or-none process. In the current spatial memory functional magnetic resonance imaging (fMRI) study, we examined the hippocampal activity distributions—the magnitude of activity as a function of memory strength—to determine the nature of processing in this region. During encoding, participants viewed abstract shapes in the left or right visual field. During retrieval, old shapes were presented at fixation and participants classified each shape as previously in the “left” or “right” visual field followed by an “unsure”–“sure”–“very sure” confidence rating. The contrast of left-hits and left-misses produced two activations in the hippocampus. The hippocampal activity distributions for left shapes and right shapes were completely overlapping. Critically, the magnitude of activity associated with right-miss-very sure responses was significantly greater than zero. These results support the continuous model of recollection, which predicts overlapping activity distributions, and contradict the threshold model of recollection, which predicts a threshold above which only one distribution exists. Receiver operating characteristic analysis did not distinguish between models. The present results demonstrate that the hippocampus operates in a continuous manner during recollection and highlight the utility of analyzing activity distributions to determine the nature of neural processing.

Neurocognitive Aging and the Hippocampus across Species

There is extensive evidence that aging is associated with impairments in episodic memory. Many of these changes have been ascribed to neurobiological alterations to the hippocampal network and its input pathways. A cross-species consensus is beginning to emerge suggesting that subtle synaptic and functional changes within this network may underlie the majority of age-related memory impairments. In this review we survey convergent data from animal and human studies that have contributed significantly to our understanding of the brain-behavior relationships in this network, particularly in the aging brain. We utilize a cognitive as well as a neurobiological perspective and synthesize data across approaches and species to reach a more detailed understanding of age-related alterations in hippocampal memory function.

Episodic memory in aspects of large-scale brain networks

Understanding human episodic memory in aspects of large-scale brain networks has become one of the central themes in neuroscience over the last decade. Traditionally, episodic memory was regarded as mostly relying on medial temporal lobe (MTL) structures. However, recent studies have suggested involvement of more widely distributed cortical network and the importance of its interactive roles in the memory process. Both direct and indirect neuro-modulations of the memory network have been tried in experimental treatments of memory disorders. In this review, we focus on the functional organization of the MTL and other neocortical areas in episodic memory. Task-related neuroimaging studies together with lesion studies suggested that specific sub-regions of the MTL are responsible for specific components of memory. However, recent studies have emphasized that connectivity within MTL structures and even their network dynamics with other cortical areas are essential in the memory process. Resting-state functional network studies also have revealed that memory function is subserved by not only the MTL system but also a distributed network, particularly the default-mode network (DMN). Furthermore, researchers have begun to investigate memory networks throughout the entire brain not restricted to the specific resting-state network (RSN). Altered patterns of functional connectivity (FC) among distributed brain regions were observed in patients with memory impairments. Recently, studies have shown that brain stimulation may impact memory through modulating functional networks, carrying future implications of a novel interventional therapy for memory impairment.

The neural mechanism underlying recollection is sensitive to the quality of episodic memory: Event related potentials reveal a some-or-none threshold

Although much is known about the underlying neural systems that support recollection, exactly how recollection operates remains unclear. One possibility is that recollection reflects the operation of a continuous retrieval process, whereby test cues always elicit some information from memory. Alternatively, recollection may reflect the operation of a thresholded process that allows for retrieval failure, whereby test cues sometimes elicit no information from memory at all. Here we demonstrate that recollection is thresholded by measuring a commonly reported electrophysiological correlate of episodic retrieval--known as the Left Parietal old/new effect. We use a novel source task designed to directly measure the accuracy of retrieval success, finding that the neural correlate of retrieval was sensitive to the precision of responses when recollection succeeded, but was absent when recollection failed. The results clarify the nature of the neural mechanism underlying episodic memory, providing novel evidence in support of some-or-none threshold models of recollection.

Associative recognition processes are modulated by the semantic unitizability of memoranda

Although memory of episodic associations is generally considered to be recollective in nature, it has been suggested that when stimuli are experienced as a unit, familiarity processes might contribute to their subsequent associative recognition. To investigate the effect of semantic relatedness during episodic encoding on the processes of retrieval of associative information, we had participants interactively encode pairs of object pictures, vertically arranged so as to suggest a functional or configural relationship between them. Half the pairs were independently judged to be of related objects (e.g., a lamp over a table) and half of unrelated objects (e.g., a key-ring over an apple). At test, participants discriminated between intact, recombined, and new pairs while event related potentials (ERPs) were recorded. In an early ERP marker of retrieval success generally associated with familiarity processes, differences related to associative memory only emerged for related pairs, while differences associated with item memory emerged for both related and unrelated pairs. In contrast, in a later ERP effect associated with recollection, differences related to associative memory emerged for both related and unrelated pairs. These findings may indicate that retrieval of episodic associations formed between two semantically related visual stimuli can be supported by familiarity-related processes.

False memory for context activates the parahippocampal cortex

Previous studies have reported greater activity in the parahippocampal cortex during true memory than false memory, which has been interpreted as reflecting greater sensory processing during true memory. However, in these studies, sensory detail and contextual information were confounded. In the present fMRI study, we employed a novel paradigm to dissociate these factors. During encoding, abstract shapes were presented in one of two contexts (i.e., moving or stationary). During retrieval, participants classified shapes as previously "moving" or "stationary." Critically, contextual processing was relatively greater during false memory ("moving" responses to stationary items), while sensory processing was relatively greater during true memory ("moving" responses to moving items). Within the medial temporal lobe, false memory versus true memory produced greater activity in the parahippocampal cortex, whereas true memory versus false memory produced greater activity in the hippocampus. The present results indicate that the parahippocampal cortex mediates contextual processing rather than sensory processing.

First come, last primed: FN400 reflects post-encoding editing of the memory trace

Explicit associative memory relies on different neural substrates depending on similarity of the items associated. Extant literature dissociates old/new effects elicited by homogeneous and heterogeneous pairs. However, current results scarcely address potential priming effects induced by the presentation sequence. In the present ERP study, participants learned associations between two pictures, which either belonged to the same semantic category (e.g., animal-animal), or to different categories (e.g., animal-scene). Pictures forming a pair were shown sequentially, allowing for investigation of the different neural processes related to presentation of the first and the second item. After the study phase, participants performed a recognition judgment. After recognition, participants were asked to recall the associated picture. During retrieval, between 260 and 350 ms post-stimulus there was a significant frontal effect of category (i.e., same-category versus different-category), but only for items shown first within a pair. In the 350-600 ms time window the parietal old/new effect was unaffected by semantic category, but was modulated by presentation order. Exploratory analyses revealed even earlier effects in the time windows 40-90 ms and 150-200 ms. This evidence supports the priming account of the FN400 and highlights the importance of sequence effects in electrophysiological activity during episodic retrieval.

Associative learning beyond the medial temporal lobe: many actors on the memory stage

Decades of research have established a model that includes the medial temporal lobe, and particularly the hippocampus, as a critical node for episodic memory. Neuroimaging and clinical studies have shown the involvement of additional cortical and subcortical regions. Among these areas, the thalamus, the retrosplenial cortex, and the prefrontal cortices have been consistently related to episodic memory performance. This article provides evidences that these areas are in different forms and degrees critical for human memory function rather than playing only an ancillary role. First we briefly summarize the functional architecture of the medial temporal lobe with respect to recognition memory and recall. We then focus on the clinical and neuroimaging evidence available on thalamo-prefrontal and thalamo-retrosplenial networks. The role of these networks in episodic memory has been considered secondary, partly because disruption of these areas does not always lead to severe impairments; to account for this evidence, we discuss methodological issues related to the investigation of these regions. We propose that these networks contribute differently to recognition memory and recall, and also that the memory stage of their contribution shows specificity to encoding or retrieval in recall tasks. We note that the same mechanisms may be in force when humans perform non-episodic tasks, e.g., semantic retrieval and mental time travel. Functional disturbance of these networks is related to cognitive impairments not only in neurological disorders, but also in psychiatric medical conditions, such as schizophrenia. Finally we discuss possible mechanisms for the contribution of these areas to memory, including regulation of oscillatory rhythms and long-term potentiation. We conclude that integrity of the thalamo-frontal and the thalamo-retrosplenial networks is necessary for the manifold features of episodic memory.