Observing memory encoding while it unfolds: Functional interpretation and current debates regarding ERP subsequent memory effects

Our ability to remember the past depends on neural processes set in train in the moment an event is experienced. These processes can be studied by segregating brain activity according to whether an event is later remembered or forgotten. The present review integrates a large number of studies examining this differential brain activity, labeled subsequent memory effect (SME), with the ERP technique, into a functional organization and discusses routes for further research. Based on the reviewed literature, we suggest that memory encoding is implemented by multiple processes, typically reflected in three functionally different subcomponents of the ERP SME elicited by study stimuli, which presumably interact with preparatory SME activity preceding the to be encoded event. We argue that ERPs are a valuable method in the SME paradigm because they have a sufficiently high temporal resolution to disclose the subcomponents of encoding-related brain activity. Implications of the proposed functional organization for future studies using the SME procedure in basic and applied settings will be discussed.

Reduced concentration performance and heartbeat-evoked potential in individuals with a history of a SARS-CoV-2 infection

The goal of characterizing long-term psychological and neural consequences of a SARS-CoV-2 infection has recently gained importance. Here, we examined the effect of a prior SARS-CoV-2 infection on neural markers of exteroceptive (P300) and interoceptive (heartbeat-evoked potential; HEP) signal processing, as well as on neuropsychological tests of attention, inhibition and episodic memory, in 23 adults with a self-reported history of SARS-CoV-2 infection versus 23 healthy controls. We found that the group with a prior infection showed decreased HEP (but not P300) amplitudes, as well as reduced attention/concentration performance. These results suggest that SARS-CoV-2 may damage neural structures of cardiac interoception, thereby potentially contributing to cognitive and psychological long-term deficits. Modulations of interoceptive processing after a SARS-CoV-2 infection are thus a promising target for future research.

Reliability and stability of oddball P300 amplitude in older adults: The role of stimulus sequence effects

The P300 may be an individual difference marker of neuro-cognitive function, which due to age-related cognitive decline may be particularly useful in older adults. Recently, we reported effects of the local stimulus sequence in an oddball task (i.e., the number of non-targets that preceded a target) on P300 amplitude in young and older adults. The same older adults completed a second session of the task 4-8 months after the first session. Here, we examined the effect of the stimulus sequence on the between- and within-session reliability and stability of P300 amplitude and RT, and their intertrial variability, in this sample of older adults. The sequence effects - an inverted U-shape effect of the number of standards preceding a target on parietal P300, and a linear effect on frontal P300 - were stable within and across sessions on the group level. Within individuals, P300 amplitude at frontal and parietal electrodes was generally very reliable and stable, mostly independently of sequence effects, encouraging its use as an individual difference marker of neuro-cognitive function in older adults. However, measures of the strength of the sequence effects themselves showed unacceptable reliability, speaking against their use as individual difference markers, at least in older adults.

Pre-associative item encoding influences associative memory: Behavioral and ERP evidence

It is unknown whether the manner with which an item is encoded in isolation, immediately before it is encoded into an inter-inter association, influences associative memory. We therefore presented the items of to-be-encoded associative pairings sequentially and manipulated how each first item of a pair was encoded (before associative encoding could begin). Furthermore, we recorded ERPs during memory encoding to investigate the neurocognitive processes that might relate pre-associative item encoding to subsequent associative memory performance. Behaviorally, we found that pre-associative item elaboration (vs. no elaboration) led to a memory tradeoff-enhanced item memory relative to impaired associative memory. This tradeoff likely reflected that item elaboration reduced cognitive resources for ensuing associative encoding, indexed by a reduced P300 and frontal slow wave at the time of associative encoding. However, frontal slow wave subsequent memory effects measured during pre-associative item encoding revealed that, for a given item, greater semantic elaboration was related to better item and associative memory while greater visual elaboration was related to better item and worse associative memory. Thus, there are likely two opposing ways in which pre-associative item encoding can influence associative memory: (1) by depleting encoding resources to impair associative memory and (2) by scaffolding inter-item associations to enhance associative memory. When item encoding occurs immediately before associative encoding, it appears that the temporary depletion of encoding resources is more important in determining later memory performance. Future research should compare the independent effects of resource depletion and encoding strategy during pre-associative item encoding.

An examination of task factors that influence the associative memory deficit in aging

Aging is accompanied by a decline in associative memory, whereas item memory remains relatively stable compared to young adults. This age-related associative deficit is well replicated, but its mechanisms and influencing factors during learning are still largely unclear. In the present study, we examined mediators of the age-related associative deficit, including encoding intentionality, strategy instructions, the timing of the memory test (immediate vs. 24 h delayed) and the material being learned (words vs. pictures) in a within-subject design. Older and younger adults performed seven encoding tasks on word pairs and picture pairs on two consecutive days, followed by item and associative recognition tests. The associative deficit was evident after all encoding tasks. We found no evidence for a difference in the magnitude of the associative deficit between incidental vs. intentional learning conditions. However, there was some evidence for a larger associative memory deficit with pictures versus words when the encoding task was held equal. Sentence generation and interactive imagery instructions in which participants generated their own mediators reduced the magnitude of the associative deficit. However, increased encoding guidance through the provision of mediators did not lead to an alleviation of the deficit, potentially because the specified mediators were implausible or difficult for the older adults to reconcile with prior knowledge. Finally, we found some evidence for a reduced age-related associative deficit with a test delay of 24 h. These results contribute to a better understanding of the factors affecting the relative difficulty of older adults with encoding and retrieving novel associations.

Realistic context doesn't amplify the survival processing effect: Lessons learned from Covid-19 scenarios

Imagining being stranded in the grasslands of an unknown territory without basic survival materials and subsequently rating the relevance of words for this situation leads to exceptionally good memory for these words. This survival processing effect has received much attention, primarily because it has been argued to disclose the evolutionary foundations of human memory. So far, only fictitious scenarios were used to demonstrate this effect. To provide a fairer test of emotional response against richness-of-encoding explanations of the effect, we aimed at increasing everyday relevance and realism of the survival scenarios. For this purpose, we created two new Covid-19 scenarios, one focusing on emotional response (Covid-19-emotion) and the other on survival strategy (Covid-19-strategy). Both new scenarios were compared to the classical grassland and moving scenarios typically used to investigate the survival processing effect. In Experiment 1, we observed better memory for the grassland and Covid-19-strategy scenarios compared to the other two, but no significant difference between the former. A descriptively similar result pattern emerged in Experiment 2 for the number of ideas generated on how to use objects in the four scenarios. Theoretical implications are discussed.

Stress effects on the oddball P300 and N2 in males and females

We examined the effect of psychosocial stress on electrophysiological markers of novelty and deviance processing, the N2 and P300, as well as sex differences therein. Participants underwent the Trier Social Stress Test (TSST) or a control procedure, followed by an oddball paradigm. A physiological stress response was induced in both sexes in the TSST condition. Furthermore, a reduced target P300 amplitude was elicited in the TSST condition in fronto-central electrodes, an effect that did not differ in magnitude between both sexes. By contrast, stressor effects on N2 amplitude differed by sex: In females, N2 amplitude was decreased in the TSST condition, while in males, no stress effects were observed. Exploratory correlational analyses suggested that a stress-induced P300 amplitude reduction, at least in males, could be due to a modulation of adrenergic activity. These results are consistent with sex differences in the manner in which stress affects lower-level, stimulus-driven vs. higher-level, more controlled processing stages.

Older adults show a higher heartbeat-evoked potential than young adults and a negative association with everyday metacognition

The ability to monitor internal bodily and cognitive processes is essential for everyday functioning and independence in older adults, because it allows for adjustments when lapses in performance are imminent. In the present study, age-related morphological changes to the heartbeat evoked potential (HEP), an electrophysiological cortical representation of cardiac signals, and its association with self-reported everyday cognition were examined. A community sample of older adults showed an increased HEP amplitude, which could reflect a stronger representation of early stages of cardiac interoception, and a more anterior scalp distribution of the HEP, suggesting a more widespread configuration of the underlying neural generators, compared to a group of young adults. Furthermore, in older adults, HEP amplitude was negatively correlated with self-estimated everyday cognitive functioning. Older adults with pronounced cortical representations of peripheral signals may thus be more likely to take note of lapses in their own bodily and cognitive function, leading to lower estimates of their cognitive abilities. These results provide novel insights into age-related changes in interoceptive processing and their association with metacognitive judgments, with potentially far-reaching implications for cognitive aging and age-related cognitive decline.

Adaptive Memory: Independent Effects of Survival Processing and Reward Motivation on Memory

Humans preferentially remember information processed for their survival relevance, a memorial benefit known as the survival processing effect. Memory is also biased towards information associated with the prospect of reward. Given the adaptiveness of these effects, they may depend on similar mechanisms. We tested whether motivation drives both effects, with reward incentives that are known to boost extrinsic motivation and survival processing perhaps stimulating intrinsic motivation. Accordingly, we manipulated survival processing and reward incentive independently during an incidental-encoding task in which participants chose between pairs of words concerning their relevance for a scenario, and examined the effects on encoding event-related potentials (ERP) activity and later performance on a surprise recall test. We hypothesized that if survival processing fosters intrinsic motivation, it should reduce the beneficial effects of extrinsic motivation (reward incentive). In contrast to this prediction, we found that reward incentive and survival processing independently improved memory and that the P300, a measure of lower-level cognitive resource allocation, was increased by reward incentive independent of survival processing. Further, survival processing and reward incentive independently increased the frontal slow wave (FSW), a measure of higher-level elaboration. These findings suggest that while survival processing and reward incentive may both increase encoding elaboration, the memory-enhancing effect of survival processing does not depend on increased intrinsic motivation. Additionally, we replicated a recent finding whereby the survival processing effect generalizes to a choice-based encoding task and further showed that the beneficial effect of choice on memory likely does not interact with either survival processing or reward.

The timing-dependent effects of stress-induced cortisol release on evaluative conditioning

The neuro-physiological response to stress has far-reaching implications for learning and memory processes. Here, we examined whether and how the stress-induced release of cortisol, following the socially-evaluated cold pressor test, influenced the acquisition of preferences in an evaluative conditioning (EC) procedure. We found that when the stressor preceded the evaluation phase, cortisol responders showed decreased evaluative conditioning effects. By contrast, impairing effects of a stressor-induced cortisol release before encoding were not found. Moreover, explicit memory was not found to be affected by the stressor or its timing. Implications of the timing-dependent effects of stress-induced cortisol release on EC and the relation between stress and associative memory are discussed.

The neurocognitive basis of metamemory: Using the N400 to study the contribution of fluency to judgments of learning

Metamemory is crucial for monitoring, evaluating, and optimizing memory performance. The basis of metamemory, however, is a matter of considerable debate. In the present study, we examined the contribution of processing fluency-the ease of processing information during learning-to metamemory judgments. We recorded event-related potentials (ERPs) while participants studied related and unrelated word pairs across two study-test cycles in a judgment of learning (JOL) task. In the first study-test cycle, related pairs were associated with better cued recall, higher JOLs, and a reduced N400 amplitude than unrelated pairs. Crucially, between- and within subject correlational analyses indicated that reduced N400 amplitudes, indexing more fluent processing, were associated with higher JOLs. Furthermore, single-trial N400 mediated a small but significant portion of the relatedness effect on JOLs. In the second study-test cycle, relatedness still increased recall and JOLs. However, related and unrelated pairs did not differ in N400 amplitude. Rather, unrelated pairs elicited a parietal positivity in a later time window that partially mediated the relatedness effect on JOLs. Together, these results suggest that processing fluency, indexed by the N400, contributes to the relatedness effect on JOLs when novel word pairs are learned, but not when previously studied pairs are relearned. Our results also imply that aspects of fluency not captured by the N400 and/or explicit beliefs about memory contribute to JOLs. This study demonstrates the utility of ERPs in gaining new insights into the neurocognitive mechanisms of metamemory.

On the role of retrieval processes in the survival processing effect: Evidence from ROC and ERP analyses

Memory is enhanced for words encoded in the context of an imagined survival scenario, an effect modulated by word imageability or concreteness. However, the mechanisms underlying this "survival processing effect" are still controversial. To address this issue, we used receiver operating characteristic (ROC) curves and event-related potentials (ERPs) to examine the recognition retrieval processes associated with words previously encoded in either a survival or a control scenario. More specifically, we investigated how recollection- and familiarity-based retrieval are influenced by survival processing of high- and low-imageability words. Participants incidentally encoded words and then completed a surprise recognition test while their EEG was recorded. The encoding of concrete, high-imageability words in a survival context lead to improved recognition memory compared to the control context, and this improvement was associated with an increase in both the ROC and ERP measures of recollection-based memory retrieval. Survival processing was also associated with an increase in the ERP familiarity signal for these words, but the ROC analysis indicated that recognition judgments relied upon recollection rather than familiarity. These findings provide evidence that survival processing increases elaboration during encoding, leading to greater recollection at retrieval and, in turn, enhanced memory.

Unitization of word pairs in young and older adults: Encoding mechanisms and retrieval outcomes

We investigated whether healthy older adults are able to use an episodic encoding strategy known as unitization, which allows for subsequent associative retrieval based on familiarity, to overcome their associative memory deficit. Young and healthy older participants were presented with word pairs either together with a definition that allowed to combine the word pairs to a new concept (high unitization condition), or together with a sentence frame (low unitization condition). In Experiment 1, an age-related reduction in performance on a standard associative recognition test was observed in both conditions. This deficit was unexpectedly not reduced, but tended to be larger in the high than the low unitization condition. According to receiver-operating characteristics, this difference was due to a reduction of recollection, but not familiarity, in the high unitization condition. Instead of a standard recognition test, Experiment 2 used a 2 alternative forced choice (2AFC) test designed to maximize the contribution of familiarity to associative recognition. Although the disadvantage of older adults in the high versus the low unitization condition was abolished, there was still no performance advantage for the high unitization condition. Event-related potentials (ERPs) recorded during the encoding phase of Experiment 1 suggest that, while young adults engage in predictive processing during unitization, older adults do not engage in such predictive processing, which may prevent them from using unitization to their advantage in the subsequent associative memory test. We discuss the task characteristics that have an impact on the effect of unitization conditions on associative memory in older adults. (PsycINFO Database Record

ERP Subsequent Memory Effects Differ between Inter-Item and Unitization Encoding Tasks

The "subsequent memory paradigm" is an analysis tool to identify brain activity elicited during episodic encoding that is associated with successful subsequent retrieval. Two commonly observed event-related potential "subsequent memory effects" (SMEs) are the parietal SME in the P300 time window and the frontal slow wave SME, but to date a clear characterization of the circumstances under which each SME is observed is missing. To test the hypothesis that the parietal SME occurs when aspects of an experience are unitized into a single item representation, while inter-item associative encoding is reflected in the frontal slow wave effect, participants were assigned to one of two conditions that emphasized one of the encoding types under otherwise matched study phases of a recognition memory experiment. Word pairs were presented either in the context of a definition that allowed to combine the word pairs into a new concept (unitization or item encoding) or together with a sentence frame (inter-item encoding). Performance on the recognition test did not differ between the groups. The parietal SME was only found in the definition group, supporting the idea that this SME occurs when the components of an association are integrated in a unitized item representation. An early prefrontal negativity also exhibited an SME only in this group, suggesting that the formation of novel units occurs through interactions of multiple brain areas. The frontal slow wave SME was pronounced in both groups and may thus reflect processes generally involved in encoding of associations. Our results provide evidence for a partial dissociation of the eliciting conditions of the two types of SMEs and therefore provide a tool for future studies to characterize the different types of episodic encoding.

The Effect of Unitizing Word Pairs on Recollection Versus Familiarity-Based Retrieval- Further Evidence From ERPs

We investigated the contribution of familiarity and recollection to associative retrieval of word pairs depending on the extent to which the pairs have been unitized through task instructions in the encoding phase. Participants in the unitization condition encoded word pairs in the context of a definition that tied them together such that they were treated as a coherent new item, while in the control condition word pairs were inserted into a sentence frame in which each word remained an individual unit. Contrasting event-related potentials (ERERPs) elicited in a subsequent recognition test by old (intact) and recombined (a new combination of two words from different study pairs) word pairs, an early frontal effect, the putative ER P correlate of familiarity-based retrieval, was apparent in the unitization condition. The left parietal old/new effect, reflecting recollection-based retrieval, was elicited only in the control condition. This suggests that in the unitization condition only, familiarity was sufficiently diagnostic to distinguish old from recombined pairs, while in the control condition, recollection contributed to associative recognition. Our findings add to a body of literature suggesting that unitization of associations increases the relative contribution of familiarity to subsequent associative retrieval.

Neural response to action and reward prediction errors: Comparing the error-related negativity to behavioral errors and the feedback-related negativity to reward prediction violations

The error-related negativity (ERN) is thought to index an anterior cingulate (ACC) behavioral monitoring system. The feedback ERN (FRN) is elicited to error feedback when the correct response is not known, but also when a choice outcome is suboptimal and to passive reward prediction violation, suggesting that the monitoring system may not be restricted to actions. This study used principal components analysis to show that the ERN consists of a single central component whereas the reward prediction violation FRN is comprised of central and prefrontal components. A prefrontal component is also present in action monitoring but occurs later, at the error positivity latency. This suggests that ACC monitors both actions and events for reward prediction error. Prefrontal cortex may update reward expectation based on the prediction violation with the latency difference due to differential processing time for motor and perceptual information.

ERP and pupil responses to deviance in an oddball paradigm

We investigated the relationship between, and functional significance of, P300, novelty P3, and the pupil dilation response (PDR). Subjects categorized stimuli including (a) words of a frequent category, (b) words of an infrequent category (14%), and (c) pictures of the frequent category ("novels"; 14%). The P300 and novelty P3 were uncorrelated with the PDR and differed in their response to experimental manipulation. Therefore, although the three physiological responses often co-occur, they appear to each manifest a distinct function: The PDR may be more closely linked to aspects of behavioral responding than the event-related potentials. Within participants, P300 and PDR latencies accounted for unique portions of the reaction time variance, and amplitudes of all three responses were larger for stimuli recalled on a subsequent test, compared to not recalled. We discuss the possibility that all three responses reflect norepinephric input from the locus coeruleus.

The component structure of event-related potentials in the p300 speller paradigm

We investigated the componential structure of event-related potentials elicited while participants use the P300 BCI. Six healthy participants "typed" all characters in a 6 × 6 matrix twice in a random sequence. A principal component analysis indicated that in addition to the P300, target flashes elicited an earlier frontal positivity, possibly a Novelty P3. The amplitudes of both P300 and the Novelty P3 varied with the matrix row in which the target character was located. However, the P300 elicited by row flashes was largest for targets in the lower part of the matrix, whereas the Novelty P3 elicited by column flashes was largest in the top part. Classification accuracy using stepwise linear discriminant analysis mirrored the pattern in the Novelty P3 (an accuracy difference of 0.1 between rows 1 and 6). When separate classifiers were generated to rely solely on the P300 or solely on the Novelty P3, the latter function led to higher accuracy (a mean accuracy difference of about 0.2 between classifiers). A possible explanation is that some nontarget flashes elicit a P300, leading to lower selection accuracy of the respective classifier. In an additional set of data from six different participants we replicated the ERP structure of the initial analyses and characterized the spatial distributions more closely by using a dense electrode array. Overall, our findings provide new insights in the componential structure of ERPs elicited in the P300 speller paradigm and have important implications for optimizing the speller's selection accuracy.

The component structure of ERP subsequent memory effects in the Von Restorff paradigm and the word frequency effect in recall

We examined the degree to which ERP components elicited by items that are isolated from their context, either by their font size ("size isolates") or by their frequency of usage, are correlated with subsequent immediate recall. Study lists contained (a) 15 words including a size isolate, (b) 14 high frequency (HF) words with one low frequency word ("LF isolate"), or (c) 14 LF words with one HF word. We used spatiotemporal PCA to quantify ERP components. We replicated previously reported P300 subsequent memory effects for size isolates and found additional correlations with recall in the novelty P3, a right lateralized positivity, and a left lateralized slow wave that was distinct from the slow wave correlated with recall for nonisolates. LF isolates also showed evidence of a P300 subsequent memory effect and also elicited the left lateralized subsequent memory effect, supporting a role of distinctiveness in word frequency effects in recall.