Material-specific neural correlates of memory retrieval

Event-related brain potential correlates of the other-race effect: A review

People are better at remembering own-race relative to other-race faces. Here, we review event-related brain potential (ERP) correlates of this so-called other-'race' effect (ORE) by discussing three critical aspects that characterize the neural signature of this phenomenon. First, difficulties with other-race faces initially emerge during perceptual processing, which is indexed by an increased N170. Second, as evidenced by 'difference due to subsequent memory' effects, more effortful processing of other-race faces is needed for successful encoding into long-term memory. Third, ERP old/new effects reveal that a stronger engagement of processing resources is also required for successful retrieval of other-race faces from memory. The ERP evidence available to date thus suggests widespread ethnicity-related modulations during both perceptual and mnemonic processing stages. We further discuss how findings from the ORE compared with potentially related memory biases (e.g. other-gender or other-age effects) and how ERP findings inform the ongoing debate regarding the mechanisms underlying the ORE. Finally, we outline open questions and potential future directions with an emphasis on using multiple, ecologically more valid 'ambient' images for each face to assess the ORE in paradigms that capture identity rather than image recognition.

Effects of emotional study context on immediate and delayed recognition memory: Evidence from event-related potentials

Whilst research has largely focused on the recognition of emotional items, emotion may be a more subtle part of our surroundings and conveyed by context rather than by items. Using ERPs, we investigated which effects an arousing context during encoding may have for item-context binding and subsequent familiarity-based and recollection-based item-memory. It has been suggested that arousal could facilitate item-context bindings and by this enhance the contribution of recollection to subsequent memory judgements. Alternatively, arousal could shift attention onto central features of a scene and by this foster unitisation during encoding. This could boost the contribution of familiarity to remembering. Participants learnt neutral objects paired with ecologically highly valid emotional faces whose names later served as neutral cues during an immediate and delayed test phase. Participants identified objects faster when they had originally been studied together with emotional context faces. Items with both neutral and emotional context elicited an early frontal ERP old/new difference (200-400 ms). Neither the neurophysiological correlate for familiarity nor recollection were specific to emotionality. For the ERP correlate of recollection, we found an interaction between stimulus type and day, suggesting that this measure decreased to a larger extend on Day 2 compared with Day 1. However, we did not find direct evidence for delayed forgetting of items encoded in emotional contexts at Day 2. Emotion at encoding might make retrieval of items with emotional context more readily accessible, but we found no significant evidence that emotional context either facilitated familiarity-based or recollection-based item-memory after a delay of 24 h.

Unitization of internal and external features contributes to associative recognition for faces: Evidence from modulations of the FN400

Associative recognition requires discriminating between old items and conjunction lures constructed by recombining elements from two different study items. This task can be solved not only by recollection but also by familiarity if the to-be-remembered stimuli are perceived as a unitized representation. In two event-related potential (ERP) studies, we provide evidence for the integration of internal and external facial features by showing that the early frontal old-new effect (considered a correlate of familiarity) is modulated by the specific combination of facial features. Participants studied faces consisting of internal features (eyes, eyebrows, nose, and mouth) paired with external features (hair, head shape, and ears). During the testing phase, intact, recombined, and new faces were presented. Recombined faces consisted of internal and external features taken from two different studied faces. The results showed that at the frontal sites, during the time window from 300 to 500 ms, ERPs to intact faces were more positive than those to new and recombined faces; the latter two did not differ from one another. The late parietal effect was observed only after a more extended study phase in Experiment 2. We take the modulation of the early frontal old-new effect as evidence for the contribution of familiarity to associative recognition for combinations of internal and external facial features.

An item's status in semantic memory determines how it is recognized: Dissociable patterns of brain activity observed for famous and unfamiliar faces

Are all faces recognized in the same way, or does previous experience with a face change how it is retrieved? Previous research using human scalp-recorded Event-Related Potentials (ERPs) demonstrates that recognition memory can produce dissociable brain signals under a variety of circumstances. While many studies have reported dissociations between the putative ‘dual processes’ of familiarity and recollection, a growing number of reports demonstrate that recollection itself may be fractionated into component processes. Many recognition memory studies using lexical materials as stimuli have reported a left parietal ERP old/new effect for recollection; however, when unfamiliar faces are recollected, an anterior effect can be observed. This paper addresses two separate hypotheses concerning the functional significance of the anterior old/new effect: perceptual retrieval and semantic status. The perceptual retrieval view is that the anterior effect reflects reinstatement of perceptual information bound up in an episodic representation, while the semantic status view is that information not represented in semantic memory pre-experimentally elicits the anterior effect instead of the left parietal effect. We tested these two competing accounts by investigating recognition memory for unfamiliar faces and famous faces in two separate experiments, in which same or different pictures of studied faces were presented as test items to permit brain activity associated with retrieving face and perceptual information to be examined independently. The difference in neural activity between same and different picture hits was operationalized as a pattern of activation associated with perceptual retrieval; while the contrast between different picture hits and correct rejection of new faces was assumed to reflect face retrieval. In Experiment 1, using unfamiliar faces, the anterior old/new effect (500–700 ms) was observed for face retrieval but not for perceptual retrieval, challenging the perceptual retrieval hypothesis. In Experiment 2, using famous faces, face retrieval was associated with a left parietal effect (500–700 ms), supporting the semantic representation hypothesis. A between-subjects analysis comparing scalp topography across the two experiments found that the anterior effect observed for unfamiliar faces is dissociable from the left parietal effect found for famous faces. This pattern of results supports the hypothesis that an item's status in semantic memory determines how it is recognized.

•

Famous faces are recognized more easily than unfamiliar faces.

•

Left parietal ERP old/new effect is observed for famous faces.

•

Anterior ERP old/new effect is observed for unfamiliar faces.

•

Semantic status determines how faces are recognized.

Investigating the Functional Utility of the Left Parietal ERP Old/New Effect: Brain Activity Predicts within But Not between Participant Variance in Episodic Recollection

A success story within neuroimaging has been the discovery of distinct neural correlates of episodic retrieval, providing insight into the processes that support memory for past life events. Here we focus on one commonly reported neural correlate, the left parietal old/new effect, a positive going modulation seen in event-related potential (ERP) data that is widely considered to index episodic recollection. Substantial evidence links changes in the size of the left parietal effect to changes in remembering, but the precise functional utility of the effect remains unclear. Here, using forced choice recognition of verbal stimuli, we present a novel population level test of the hypothesis that the magnitude of the left parietal effect correlates with memory performance. We recorded ERPs during old/new recognition, source accuracy and Remember/Know/Guess tasks in two large samples of healthy young adults, and successfully replicated existing within participant modulations of the magnitude of the left parietal effect with recollection. Critically, however, both datasets also show that across participants the magnitude of the left parietal effect does not correlate with behavioral measures of memory – including both subjective and objective estimates of recollection. We conclude that in these tasks, and across this healthy young adult population, the generators of the left parietal ERP effect do not index performance as expected. Taken together, these novel findings provide important constraints on the functional interpretation of the left parietal effect, suggesting that between group differences in the magnitude of old/new effects cannot always safely be used to infer differences in recollection.

Separating content-specific retrieval from post-retrieval processing

According to cortical reinstatement accounts, neural processes engaged at the time of encoding are re-engaged at the time of memory retrieval. The temporal precision of event-related potentials (ERPs) has been exploited to assess this possibility, and in this study ERPs were acquired while people made memory judgments to visually presented words encoded in two different ways. There were reliable differences between the scalp distributions of the signatures of successful retrieval of different contents from 300 to 1100 ms after stimulus presentation. Moreover, the scalp distributions of these content-sensitive effects changed during this period. These findings are, to our knowledge, the first demonstration in one study that ERPs reflect content-specific processing in two separable ways: first, via reinstatement, and second, via downstream processes that operate on recovered information in the service of memory judgments.

An ERP study of recognition memory for concrete and abstract pictures in school-aged children

Recognition memory for concrete, nameable pictures is typically faster and more accurate than for abstract pictures. A dual-coding account for these findings suggests that concrete pictures are processed into verbal and image codes, whereas abstract pictures are encoded in image codes only. Recognition memory relies on two successive and distinct processes, namely familiarity and recollection. Whether these two processes are similarly or differently affected by stimulus concreteness remains unknown. This study examined the effect of picture concreteness on visual recognition memory processes using event-related potentials (ERPs). In a sample of children involved in a longitudinal study, participants (N=96; mean age=11.3years) were assessed on a continuous visual recognition memory task in which half the pictures were easily nameable, everyday concrete objects, and the other half were three-dimensional abstract, sculpture-like objects. Behavioral performance and ERP correlates of familiarity and recollection (respectively, the FN400 and P600 repetition effects) were measured. Behavioral results indicated faster and more accurate identification of concrete pictures as "new" or "old" (i.e., previously displayed) compared to abstract pictures. ERPs were characterized by a larger repetition effect, on the P600 amplitude, for concrete than for abstract images, suggesting a graded recollection process dependent on the type of material to be recollected. Topographic differences were observed within the FN400 latency interval, especially over anterior-inferior electrodes, with the repetition effect more pronounced and localized over the left hemisphere for concrete stimuli, potentially reflecting different neural processes underlying early processing of verbal/semantic and visual material in memory.

Spatiotemporal neural pattern similarity supports episodic memory

Formal computational models of human memory posit a central role of feature representations in episodic memory encoding and retrieval [1-4]. Correspondingly, fMRI studies have found that, in addition to activity level [5, 6], the neural activation pattern similarity across repetitions (i.e., self-similarity) was greater for subsequently remembered than forgotten items [7-9]. This self-similarity has been suggested to reflect pattern reinstatement due to study-phase retrieval [7, 10, 11]. However, the low temporal resolution of fMRI measures could determine neither the temporal precision of study-phase reinstatement nor the processing stage at which the reinstatement supported subsequent memory [12]. Meanwhile, although self-similarity has been shown to correlate with the activity level in the left lateral prefrontal cortex (LPFC) [10, 13], a causal link between left LPFC function and pattern similarity remains to be established. Combining transcranial direct current stimulation (tDCS) and EEG, we found that greater spatiotemporal pattern similarity (STPS) across repetitions of the same item (i.e., self-STPS) during encoding predicted better subsequent memory. The self-STPS located in the right frontal electrodes occurred approximately 500 ms after stimulus onset, reflected item-specific encoding, and contributed to memory above and beyond the effects of ERP amplitude and global pattern similarity (i.e., similarity to all other items in memory space). Anodal stimulation over the left LPFC specifically enhanced memory performance and item-specific STPS in the right frontal electrodes. These results support a causal role of LPFC in enhancing STPS and memory and contribute to a mechanistic understanding of memory formation.

Internal versus external features in triggering the brain waveforms for conjunction and feature faces in recognition

Previous research has found that conjunction faces (whose internal features, e.g. eyes, nose, and mouth, and external features, e.g. hairstyle and ears, are from separate studied faces) and feature faces (partial features of these are studied) can produce higher false alarms than both old and new faces (i.e. those that are exactly the same as the studied faces and those that have not been previously presented) in recognition. The event-related potentials (ERPs) that relate to conjunction and feature faces at recognition, however, have not been described as yet; in addition, the contributions of different facial features toward ERPs have not been differentiated. To address these issues, the present study compared the ERPs elicited by old faces, conjunction faces (the internal and the external features were from two studied faces), old internal feature faces (whose internal features were studied), and old external feature faces (whose external features were studied) with those of new faces separately. The results showed that old faces not only elicited an early familiarity-related FN400, but a more anterior distributed late old/new effect that reflected recollection. Conjunction faces evoked similar late brain waveforms as old internal feature faces, but not to old external feature faces. These results suggest that, at recognition, old faces hold higher familiarity than compound faces in the profiles of ERPs and internal facial features are more crucial than external ones in triggering the brain waveforms that are characterized as reflecting the result of familiarity.

Temporal structure in associative retrieval

Electrophysiological data disclose rich dynamics in patterns of neural activity evoked by sensory objects. Retrieving objects from memory reinstates components of this activity. In humans, the temporal structure of this retrieved activity remains largely unexplored, and here we address this gap using the spatiotemporal precision of magnetoencephalography (MEG). In a sensory preconditioning paradigm, 'indirect' objects were paired with 'direct' objects to form associative links, and the latter were then paired with rewards. Using multivariate analysis methods we examined the short-time evolution of neural representations of indirect objects retrieved during reward-learning about direct objects. We found two components of the evoked representation of the indirect stimulus, 200 ms apart. The strength of retrieval of one, but not the other, representational component correlated with generalization of reward learning from direct to indirect stimuli. We suggest the temporal structure within retrieved neural representations may be key to their function.

Autobiographically Significant Concepts: More Episodic than Semantic in Nature? An Electrophysiological Investigation of Overlapping Types of Memory

A common assertion is that semantic memory emerges from episodic memory, shedding the distinctive contexts associated with episodes over time and/or repeated instances. Some semantic concepts, however, may retain their episodic origins or acquire episodic information during life experiences. The current study examined this hypothesis by investigating the ERP correlates of autobiographically significant (AS) concepts, that is, semantic concepts that are associated with vivid episodic memories. We inferred the contribution of semantic and episodic memory to AS concepts using the amplitudes of the N400 and late positive component, respectively. We compared famous names that easily brought to mind episodic memories (high AS names) against equally famous names that did not bring such recollections to mind (low AS names) on a semantic task (fame judgment) and an episodic task (recognition memory). Compared with low AS names, high AS names were associated with increased amplitude of the late positive component in both tasks. Moreover, in the recognition task, this effect of AS was highly correlated with recognition confidence. In contrast, the N400 component did not differentiate the high versus low AS names but, instead, was related to the amount of general knowledge participants had regarding each name. These results suggest that semantic concepts high in AS, such as famous names, have an episodic component and are associated with similar brain processes to those that are engaged by episodic memory. Studying AS concepts may provide unique insights into how episodic and semantic memory interact.

Age-related effects on perceptual and semantic encoding in memory

This study examined the age-related subsequent memory effect (SME) in perceptual and semantic encoding using event-related potentials (ERPs). Seventeen younger adults and 17 older adults studied a series of Chinese characters either perceptually (by inspecting orthographic components) or semantically (by determining whether the depicted object makes sounds). The two tasks had similar levels of difficulty. The participants made studied or unstudied judgments during the recognition phase. Younger adults performed better in both conditions, with significant SMEs detected in the time windows of P2, N3, P550, and late positive component (LPC). In the older group, SMEs were observed in the P2 and N3 latencies in both conditions but were only detected in the P550 in the semantic condition. Between-group analyses showed larger frontal and central SMEs in the younger sample in the LPC latency regardless of encoding type. Aging effect appears to be stronger on influencing perceptual than semantic encoding processes. The effects seem to be associated with a decline in updating and maintaining representations during perceptual encoding. The age-related decline in the encoding function may be due in part to changes in frontal lobe function.

Differentiation of perceptual and semantic subsequent memory effects using an orthographic paradigm

This study aimed to differentiate perceptual and semantic encoding processes using subsequent memory effects (SMEs) elicited by the recognition of orthographs of single Chinese characters. Participants studied a series of Chinese characters perceptually (by inspecting orthographic components) or semantically (by determining the object making sounds), and then made studied or unstudied judgments during the recognition phase. Recognition performance in terms of d-prime measure in the semantic condition was higher, though not significant, than that of the perceptual condition. The between perceptual-semantic condition differences in SMEs at P550 and late positive component latencies (700-1000ms) were not significant in the frontal area. An additional analysis identified larger SME in the semantic condition during 600-1000ms in the frontal pole regions. These results indicate that coordination and incorporation of orthographic information into mental representation is essential to both task conditions. The differentiation was also revealed in earlier SMEs (perceptual>semantic) at N3 (240-360ms) latency, which is a novel finding. The left-distributed N3 was interpreted as more efficient processing of meaning with semantically learned characters. Frontal pole SMEs indicated strategic processing by executive functions, which would further enhance memory.

Measuring recollection and familiarity: Improving the remember/know procedure

The remember/know (RK) procedure is the most widely used method to investigate recollection and familiarity. It uses trial-by-trial reports to determine how much recollection and familiarity contribute to different kinds of recognition. Few other methods provide information about individual memory judgements and no alternative allows such direct indications of recollection and familiarity influences. Here we review how the RK procedure has been and should be used to help resolve theoretical disagreements about the processing and neural bases of components of recognition memory. Emphasis is placed on procedural weaknesses and a possible confound of recollection and familiarity with recognition memory strength. Recommendations are made about how to minimise these problems including using modified versions of the procedure. The proposals here are important for improving behavioural and lesion research, and vital for brain imaging work.

Material-specific neural correlates of recollection: objects, words, and faces

It is unclear how neural correlates of episodic memory retrieval differ depending on the type of material that is retrieved. Here, we used a source memory task to compare electrical brain activity for the recollection of three types of stimulus material. At study, healthy adults judged how well visually presented objects, words, and faces fitted with paired auditorily presented names of locations. At test, only visual stimuli were presented. The task was to decide whether an item had been presented earlier and, if so, what location had been paired with the item. Stimulus types were intermixed across trials in Experiment 1 and presented in separate study-test lists in Experiment 2. A graded pattern of memory performance was observed across objects, words, and faces in both experiments. Between 300 and 500 msec, event-related potentials for recollected objects and faces showed a more frontal scalp distribution compared to words in both experiments. Later in the recording epoch, all three stimulus materials elicited recollection effects over left posterior scalp sites. However, these effects extended more anteriorly for objects and faces when stimulus categories were blocked. These findings demonstrate that the neural correlates of recollection are material specific, the crucial difference being between pictorial and verbal material. Faces do not appear to have a special status. The sensitivity of recollection effects to the kind of experimental design suggests that, in addition to type of stimulus material, higher-level control processes affect the cognitive and neural mechanisms underlying episodic retrieval.

The effects of aging on material-independent and material-dependent neural correlates of source memory retrieval

Age-related declines in source memory have been observed for various stimuli and associated details. These impairments may be related to alterations in brain regions contributing to source memory via material-independent processes and/or regions specialized for processing specific materials. Using event-related functional magnetic resonance imaging, we investigate the effects of aging on source memory and associated neural activity for words and objects. Source accuracy was equally impaired in older adults for both materials. Imaging data revealed both groups recruited similar networks of regions to support source memory accuracy irrespective of material, including parietal and prefrontal cortices (PFC) and the hippocampus. Age-related decreases in material-independent activity linked to postretrieval monitoring were observed in right lateral PFC. Additionally, age-related increases in source accuracy effects were shown in perirhinal cortex, which were positively correlated with performance in older adults, potentially reflecting functional compensation. In addition to group differences in material-independent regions, age-related crossover interactions for material-dependent source memory effects were observed in regions selectively engaged by objects. These results suggest that older adults' source memory impairments reflect alterations in regions making material-independent contributions to source memory retrieval, primarily the lateral PFC, but may be further impacted by changes in regions sensitive to particular materials.

Stimulus content and the neural correlates of source memory

It has been suggested that several regions of the brain, including subregions of the medial temporal lobe (MTL) and the posterior parietal cortex, contribute to source memory success in a material-general manner, with most models highlighting the importance of memory process rather than material type. For the MTL in particular, however, increasing evidence suggests that MTL subregions may be specialized for processing different materials, raising the possibility that source memory-related activity may be material-sensitive. Previous fMRI studies have not directly compared source memory activity for different categories of stimuli, and it remains unclear whether source memory effects, in the MTL or elsewhere, are influenced by material. To investigate this issue, young participants were scanned during study while they made semantic judgments about words, pictures of objects and scenes, and during test when they retrieved the context (source) in which these items were studied. Several regions, including the hippocampi, medial and lateral parietal cortex, exhibited source memory effects common to words, objects and scenes, at both study and test. Material-dependent source memory effects were also identified in the left posterior inferior frontal and left perirhinal cortex for words and objects, respectively, at study but not test. These results offer direct support for the hypothesis that the MTL and posterior parietal cortex make material-general contributions to recollection. These results also point to a dissociation between encoding and retrieval with regard to the influence of material on the neural correlates of source memory accuracy, supporting the idea that a relatively small proportion of the activity elicited by a stimulus during encoding is incorporated into an episodic memory representation of the stimulus.