Response to Paller et al.: the role of familiarity in making inferences about unknown quantities

An ERP measure of non-conscious memory reveals dissociable implicit processes in human recognition using an open-source automated analytic pipeline

Non-conscious processing of human memory has traditionally been difficult to objectively measure and thus understand. A prior study on a group of hippocampal amnesia (N = 3) patients and healthy controls (N = 6) used a novel procedure for capturing neural correlates of implicit memory using event-related potentials (ERPs): old and new items were equated for varying levels of memory awareness, with ERP differences observed from 400 to 800 ms in bilateral parietal regions that were hippocampal-dependent. The current investigation sought to address the limitations of that study by increasing the sample of healthy subjects (N = 54), applying new controls for construct validity, and developing an improved, open-source tool for automated analysis of the procedure used for equating levels of memory awareness. Results faithfully reproduced prior ERP findings of parietal effects that a series of systematic control analyses validated were not contributed to nor contaminated by explicit memory. Implicit memory effects extended from 600 to 1000 ms, localized to right parietal sites. These ERP effects were found to be behaviorally relevant and specific in predicting implicit memory response times, and were topographically dissociable from other traditional ERP measures of implicit memory (miss vs. correct rejections) that instead occurred in left parietal regions. Results suggest first that equating for reported awareness of memory strength is a valid, powerful new method for revealing neural correlates of non-conscious human memory, and second, behavioral correlations suggest that these implicit effects reflect a pure form of priming, whereas misses represent fluency leading to the subjective experience of familiarity.

Effects of emotion and semantic relatedness on recognition memory: Behavioral and electrophysiological evidence

Some aspects of our memory are enhanced by emotion, whereas others can be unaffected or even hindered. Previous studies reported impaired associative memory of emotional content, an effect termed associative "emotional interference". The current study used EEG and an associative recognition paradigm to investigate the cognitive and neural mechanisms associated with this effect. In two experiments, participants studied negative and neutral stimulus-pairs that were either semantically related or unrelated. In Experiment 1 emotions were relevant to the encoding task (valence judgment) whereas in Experiment 2 emotions were irrelevant (familiarity judgment). In a subsequent associative recognition test, EEG was recorded while participants discriminated between intact, rearranged, and new pairs. An associative emotional interference effect was observed in both experiments, but was attenuated for semantically related pairs in Experiment 1, where valence was relevant to the task. Moreover, a modulation of an early associative memory ERP component (300-550 ms) occurred for negative pairs when valence was task-relevant (Experiment 1), but for semantically related pairs when valence was irrelevant (Experiment 2). A later ERP component (550-800 ms) showed a more general pattern, and was observed in all experimental conditions. These results suggest that both valence and semantic relations can act as an organizing principle that promotes associative binding. Their ability to contribute to successful retrieval depends on specific task demands.

Recallable but not recognizable: The influence of semantic priming in recall paradigms

When people can successfully recall a studied word, they should be able to recognize it as having been studied. In cued-recall paradigms, however, participants sometimes correctly recall words in the presence of strong semantic cues but then fail to recognize those words as actually having been studied. Although the conditions necessary to produce this unusual effect are known, the underlying neural correlates have not been investigated. Across five experiments, involving both behavioral and electrophysiological methods (EEG), we investigated the cognitive and neural processes that underlie recognition failures. Experiments 1 and 2 showed behaviorally that assuming that recalled items can be recognized in cued-recall paradigms is a flawed assumption, because recognition failures occur in the presence of cues, regardless of whether those failures are measured. With event-related potentials (ERPs), Experiments 3 and 4 revealed that successfully recalled words that are recognized are driven by recollection at recall and then by a combination of recollection and familiarity at ensuing recognition. In contrast, recognition failures did not show that memory signature and may instead be driven by semantic priming at recall and followed at recognition stages by negative-going ERP effects consistent with implicit processes, such as repetition fluency. These results demonstrate that recall - long-characterized as predominantly reflecting recollection-based processing in episodic memory - may at times also be served by a confluence of implicit cognitive processes.

Distinct FN400/N400 memory effects for perceptually fluent and disfluent words

Recognition memory studies have shown that increased perceptual fluency results in more "old" responses and, presumably, increases familiarity. However, the exact neural mechanisms of these effects remain unresolved. We conducted two ERP experiments in which participants encoded words and performed a recognition test where fluency was manipulated by changing clarity of test words (half of them were clear or blurry). In the more demanding Experiment 1, we found a reversed effect of fluency on recognition (more hits for blurry words), which was accompanied by larger N400 and LPC old/new effects for blurry words. For high confidence responses, the topography of N400 shifted towards frontal electrodes (the FN400 for blurry words). In the less demanding Experiment 2, no behavioral differences between clear and blurry words were observed. However, there was a discrepancy in the ERP results, with the frontal FN400 for blurry words and the parietal N400 for clear words, suggesting that distinct neural pathways can support familiarity-based recognition for clear and blurry items. In both experiments, early perceptual fluency ERP effects were also observed. The results indicate that both semantic processing and familiarity can be enhanced by perceptual fluency and contribute to recognition judgments, depending on the interpretations of fluency.

Context influences the FN400 recognition event-related potential

Recognition of studied items often elicit more positive event-related potentials (ERPs) than unstudied items at mid-frontal electrodes about 300-500 ms (i.e., the FN400). The debate over the psychological processes associated with the FN400 has led to two competing hypotheses. One hypothesis is that the FN400 reflects familiarity, whereas another hypothesis is that it reflects conceptual implicit memory (i.e., conceptual fluency). The present experiment tested these hypotheses by presenting meaningless images that lack familiarity and conceptual fluency, off-brand products that lack pre-experimental familiarity, and name-brand products that have both pre-experimental familiarity and conceptual fluency. ERPs were recorded during judgments of lifetime and recent recognition. During both forms of recognition, ERPs in the FN400 window were greater for meaningless images than name- or off-brand products. Because this evidence is difficult to reconcile with either the familiarity or conceptual fluency hypotheses, the results are interpreted within a broader theoretical framework that includes top-down psychological (i.e., context) influences on the FN400.

Neural correlates of the Dunning-Kruger effect

The Dunning-Kruger effect (DKE) is a metacognitive phenomenon of illusory superiority in which individuals who perform poorly on a task believe they performed better than others, yet individuals who performed very well believe they under-performed compared to others. This phenomenon has yet to be directly explored in episodic memory, nor explored for physiological correlates or reaction times. We designed a novel method to elicit the DKE via a test of item recognition while electroencephalography (EEG) was recorded. Throughout the task, participants were asked to estimate the percentile in which they performed compared to others. Results revealed participants in the bottom 25th percentile over-estimated their percentile, while participants in the top 75th percentile under-estimated their percentile, exhibiting the classic DKE. Reaction time measures revealed a condition-by-group interaction whereby over-estimators responded faster than under-estimators when estimating being in the top percentile and responded slower when estimating being in the bottom percentile. Between-group EEG differences were evident between over-estimators and under-estimators during Dunning-Kruger responses, which revealed FN400-like effects of familiarity supporting differences for over-estimators, whereas "old-new" memory event-related potential effects revealed a late parietal component associated with recollection-based processing for under-estimators that was not evident for over-estimators. Findings suggest over- and under-estimators use differing cognitive processes when assessing their performance, such that under-estimators may rely on recollection during memory while over-estimators may draw upon excess familiarity when over-estimating their performance. Episodic memory thus appears to play a contributory role in metacognitive judgements of illusory superiority.

Briefly Flashed Scenes Can Be Stored in Long-Term Memory

The capacity of human memory is impressive. Previous reports have shown that when asked to memorize images, participants can recognize several thousands of visual objects in great details even with a single viewing of a few seconds per image. In this experiment, we tested recognition performance for natural scenes that participants saw for 20 ms only once (untrained group) or 22 times over many days (trained group) in an unrelated task. 400 images (200 previously viewed and 200 novel images) were flashed one at a time and participants were asked to lift their finger from a pad whenever they thought they had already seen the image (go/no-go paradigm). Compared to previous reports of excellent recognition performance with only single presentations of a few seconds, untrained participants were able to recognize only 64% of the 200 images they had seen few minutes before. On the other hand, trained participants, who had processed the flashed images (20 ms) several times, could correctly recognize 89% of them. EEG recordings confirmed these behavioral results. As early as 230 ms after stimulus onset, a significant event-related-potential (ERP) difference between familiar and new images was observed for the trained but not for the untrained group. These results show that briefly flashed unmasked scenes can be incidentally stored in long-term memory when repeated.

Format change and semantic relatedness effects on the ERP correlates of recognition: old pairs, new pairs, different stories

In this event-related potential (ERP) study, we investigated the effects of format change and semantic relatedness in a recognition task using pairs composed of a word and a line drawing. The semantic relatedness of the pairs (related: rabbit-carrot; unrelated: duck-artichoke) influenced their associative properties and corresponding distinctiveness, while format change refers to the switching of an item from the verbal form to the line drawing form between study and recognition (e.g., the word "egg" is associated with a drawing of a hen at study, and a line drawing of an egg is associated with the word "hen" at test). Study-test format change thus prevents visual matching while maintaining conceptual matching. While the N300 potential was only modulated by the semantic relatedness of the pair, both factors modulated recognition performance and corresponding ERP old/new effects with larger mid-frontal N400 old/new effect (300-500 ms) and larger parietal old/new effect (500-800 ms) in the same compared to the different-format condition, as well as for related compared to unrelated pairs. Furthermore, the semantic relatedness of correctly recognized old pairs modulated the anterior N400 while it modulated the posterior N400 for correctly rejected pairs. These results suggest that semantic relatedness and familiarity related to the amount of change between study and test present distinct ERP signatures in the N400 window. They suggest also that the distinctiveness and the ease of the retrieval of the pair could be determining for the parietal old/new effect.

A critical role of the human hippocampus in an electrophysiological measure of implicit memory

The hippocampus has traditionally been thought to be critical for conscious explicit memory but not necessary for unconscious implicit memory processing. In a recent study of a group of mild amnesia patients with evidence of MTL damage limited to the hippocampus, subjects were tested on a direct test of item recognition confidence while electroencephalogram (EEG) was acquired, and revealed intact measures of explicit memory from 400 to 600 ms (mid-frontal old-new effect, FN400). The current investigation re-analyzed this data to study event-related potentials (ERPs) of implicit memory, using a recently developed procedure that eliminated declarative memory differences. Prior ERP findings from this technique were first replicated in two independent matched control groups, which exhibited reliable implicit memory effects in posterior scalp regions from 400 to 600 ms, which were topographically dissociated from the explicit memory effects of familiarity. However, patients were found to be dramatically impaired in implicit memory effects relative to control subjects, as quantified by a reliable condition × group interaction. Several control analyses were conducted to consider alternative factors that could account for the results, including outliers, sample size, age, or contamination by explicit memory, and each of these factors was systematically ruled out. Results suggest that the hippocampus plays a fundamental role in aspects of memory processing that are beyond conscious awareness. The current findings therefore indicate that both memory systems of implicit and explicit memory may rely upon the same neural structures - but function in different physiological ways.

Inefficient Encoding as an Explanation for Age-Related Deficits in Recollection-Based Processing

A cardinal feature of aging is a decline in episodic memory (EM). Nevertheless, there is evidence that some older adults may be able to “compensate” for failures in recollection-based processing by recruiting brain regions and cognitive processes not normally recruited by the young. We review the evidence suggesting that age-related declines in EM performance and recollection-related brain activity (left-parietal EM effect; LPEM) are due to altered processing at encoding. We describe results from our laboratory on differences in encoding- and retrieval-related activity between young and older adults. We then show that, relative to the young, in older adults brain activity at encoding is reduced over a brain region believed to be crucial for successful semantic elaboration in a 400–1,400-ms interval (left inferior prefrontal cortex, LIPFC; Johnson, Nessler, & Friedman, 2013 ; Nessler, Friedman, Johnson, & Bersick, 2007 ; Nessler, Johnson, Bersick, & Friedman, 2006 ). This reduced brain activity is associated with diminished subsequent recognition-memory performance and the LPEM at retrieval. We provide evidence for this premise by demonstrating that disrupting encoding-related processes during this 400–1,400-ms interval in young adults affords causal support for the hypothesis that the reduction over LIPFC during encoding produces the hallmarks of an age-related EM deficit: normal semantic retrieval at encoding, reduced subsequent episodic recognition accuracy, free recall, and the LPEM. Finally, we show that the reduced LPEM in young adults is associated with “additional” brain activity over similar brain areas as those activated when older adults show deficient retrieval. Hence, rather than supporting the compensation hypothesis, these data are more consistent with the scaffolding hypothesis, in which the recruitment of additional cognitive processes is an adaptive response across the life span in the face of momentary increases in task demand due to poorly-encoded episodic memories.

Neurophysiological evidence for a recollection impairment in amnesia patients that leaves familiarity intact

In several previous behavioral studies, we have identified a group of amnestic patients that, behaviorally, appear to exhibit severe deficits in recollection with relative preservation of familiarity-based recognition. However, these studies have relied exclusively on behavioral measures, rather than direct measures of physiology. Event-related potentials (ERPs) have been used to identify putative neural correlates of familiarity- and recollection-based recognition memory, but little work has been done to determine the extent to which these ERP correlates are spared in patients with relatively specific memory disorders. ERP studies of recognition in healthy subjects have indicated that recollection and familiarity are related to a parietal old-new effect characterized as a late positive component (LPC) and an earlier mid-frontal old-new effect referred to as an 'FN400', respectively. Here, we sought to determine the extent to which the putative ERP correlates of recollection and familiarity are intact or impaired in these patients. We recorded ERPs in three amnestic patients and six age matched controls while they made item recognition and source recognition judgments. The current patients were able to discriminate between old and new items fairly well, but showed nearly chance-level performance at source recognition. Moreover, whereas control subjects exhibited ERP correlates of memory that have been linked to recollection and familiarity, the patients only exhibited the mid-frontal FN400 ERP effect related to familiarity-based recognition. The results show that recollection can be severely impaired in amnesia even when familiarity-related processing is relatively spared, and they also provide further evidence that ERPs can be used to distinguish between neural correlates of familiarity and recollection.