A predictive framework for evaluating models of semantic organization in free recall

Recognition memory: The probe, the returned signal, and the decision

In an attempt to better understand recognition memory we look at how three approaches (dual processing, signal detection, and global matching) have addressed the probe, the returned signal and the decision in four recognition paradigms. These are single-item recognition (including the remember/know paradigm), recognition in relational context, associative recognition, and source monitoring. The contrast, with regards to the double-miss rate (the probability of recognizing neither item in intact and rearranged pairs) and the effect of the oldness of the other member of the test pair, between identifying the old words in test pairs (the relational context paradigm) and first identifying the intact test pairs and then identifying the old words (adding associative recognition to the relational context paradigm) suggests that the retrieval of associative information in the relational context paradigm is unintentional, unlike the retrieval of associative information in associative recognition. It also seems possible that the information that is spontaneously retrieved in single-item recognition, possibly including the remember/know paradigm, is also unintentional, unlike the retrieval of information in source monitoring. Probable differences between intentional and unintentional retrieval, together with the pattern of effects with regards to the double-miss rate and the effect of the other member of the test pair, are used to evaluate the three approaches. Our conclusion is that all three approaches have something valid to say about recognition, but none is equally applicable across all four paradigms.

PEPPR: A post-encoding pre-production reinstatement model of dual-list free recall

Recent events are easy to recall, but they also interfere with the recall of more distant, non-recent events. In many computational models, non-recent memories are recalled by using the context associated with those events as a cue. Some models, however, do little to explain how people initially activate non-recent contexts in the service of accurate recall. We addressed this limitation by evaluating two candidate mechanisms within the Context-Maintenance and Retrieval model. The first is a Backward-Walk mechanism that iteratively applies a generate/recognize process to covertly retrieve progressively less recent items. The second is a Post-Encoding Pre-Production Reinstatement (PEPPR) mechanism that formally implements a metacognitive control process that reinstates non-recent contexts prior to retrieval. Models including these mechanisms make divergent predictions about the dynamics of response production and monitoring when recalling non-recent items. Before producing non-recent items, Backward-Walk cues covert retrievals of several recent items, whereas PEPPR cues few, if any, covert retrievals of that sort. We tested these predictions using archival data from a dual-list externalized free recall paradigm that required subjects to report all items that came to mind while recalling from the non-recent list. Simulations showed that only the model including PEPPR accurately predicted covert recall patterns. That same model fit the behavioral data well. These findings suggest that self-initiated context reinstatement plays an important role in recall of non-recent memories and provides a formal model that uses a parsimonious non-hierarchical context representation of how such reinstatement might occur.

Integrating word-form representations with global similarity computation in recognition memory

In recognition memory, retrieval is thought to occur by computing the global similarity of the probe to each of the studied items. However, to date, very few global similarity models have employed perceptual representations of words despite the fact that false recognition errors for perceptually similar words have consistently been observed. In this work, we integrate representations of letter strings from the reading literature with global similarity models. Specifically, we employed models of absolute letter position (slot codes and overlap models) and relative letter position (closed and open bigrams). Each of the representations was used to construct a global similarity model that made contact with responses and RTs at the individual word level using the linear ballistic accumulator (LBA) model (Brown & Heathcote Cognitive Psychology, 57 , 153-178, 2008). Relative position models were favored in three of the four datasets and parameter estimates suggested additional influence of the initial letters in the words. When semantic representations from the word2vec model were incorporated into the models, results indicated that orthographic representations were almost equally consequential as semantic representations in determining inter-item similarity and false recognition errors, which undermines previous suggestions that long-term memory is primarily driven by semantic representations. The model was able to modestly capture individual word variability in the false alarm rates, but there were limitations in capturing variability in the hit rates that suggest that the underlying representations require extension.

Semantic determinants of memorability

We examine why some words are more memorable than others by using predictive machine learning models applied to word recognition and recall datasets. Our approach provides more accurate out-of-sample predictions for recognition and recall than previous psychological models, and outperforms human participants in new studies of memorability prediction. Our approach's predictive power stems from its ability to capture the semantic determinants of memorability in a data-driven manner. We identify which semantic categories are important for memorability and show that, unlike features such as word frequency that influence recognition and recall differently, the memorability of semantic categories is consistent across recognition and recall. Our paper sheds light on the complex psychological drivers of memorability, and in doing so illustrates the power of machine learning methods for psychological theory development.

The organizational principles of impression formation

Trait impressions about others are a fundamental tool to navigate the rich social environment and yet a unitary model of its organizational principles is still lacking. The statistical properties of impression formation observed in previous studies are akin to processes that govern information encoding and storage in memory, suggesting similar cognitive and computational mechanisms. Here, in 2,780 participants, impression formation has been formalized with a computational model representing three organizational principles of memory (temporal, semantic and valence-related). The model specifically captured two main patterns of impression formation: (1) a negative valence effect that makes negative impressions loom longer than positive ones; (2) an interaction effect between the temporal and valence content that endorses more negative impressions when negative information is met first. This work shows that mechanisms of information encoding, storage and retrieval interact in ways that explain biased impression formation about social partners, thereby providing quantitative evidence for those mechanisms in individuals' impressions of others' social qualities. We discuss the implications of these results for social impressions in different, real-world contexts, and suggest how the proposed model might be extended to capture other kinds of effects, from negativity bias and pessimism to social discrimination.

Contextual reinstatement affects semantic organization

The Context Dependency Effect is the well-established finding in which memory performance is enhanced under conditions in which the encoding and retrieval contexts overlap (i.e., Same-Context) and diminished when the overlap between encoding and retrieval contexts is low (i.e., Different-Context). Despite much research on context-dependent memory, most prior work examined only mean performance levels. The current experiment examined the influence of context change, manipulated by using three different pieces of background music, on semantic organization during free recall. Recall driven by semantic organization captures an important, ecologically valid aspect of memory retrieval: because narratives of real-life events are typically comprised of semantically related concepts (e.g., "sea," "bathing suit," and "sand" when recalling a trip to the beach), their recall is likely driven by semantic organization. Participants in the current study were tested in the same or different context as the material was learned. The results showed that although the mean number of correctly recalled items was numerically greater in the Same-Context condition compared to the Different-Context condition, the Context Dependency Effect was not significant. In contrast, however, semantic clustering-an established measure of semantic organization-was greater in the Different-Context condition compared to the Same-Context condition. Together, these results suggest that when contextual cues at recall are relatively meager, participants instead use semantic information as cues to guide memory retrieval. In line with previous findings, temporal organization, patterns of errors, and serial position analyses showed no differences between the two context conditions. The present experiment provides novel evidence on how external context change affects recall organization.

Items Outperform Adjectives in a Computational Model of Binary Semantic Classification

Semantic memory encompasses one's knowledge about the world. Distributional semantic models, which construct vector spaces with embedded words, are a proposed framework for understanding the representational structure of human semantic knowledge. Unlike some classic semantic models, distributional semantic models lack a mechanism for specifying the properties of concepts, which raises questions regarding their utility for a general theory of semantic knowledge. Here, we develop a computational model of a binary semantic classification task, in which participants judged target words for the referent's size or animacy. We created a family of models, evaluating multiple distributional semantic models, and mechanisms for performing the classification. The most successful model constructed two composite representations for each extreme of the decision axis (e.g., one averaging together representations of characteristically big things and another of characteristically small things). Next, the target item was compared to each composite representation, allowing the model to classify more than 1,500 words with human-range performance and to predict response times. We propose that when making a decision on a binary semantic classification task, humans use task prompts to retrieve instances representative of the extremes on that semantic dimension and compare the probe to those instances. This proposal is consistent with the principles of the instance theory of semantic memory.

The spatiotemporal gradient of intrusion errors in continuous outcome source memory: Source retrieval is affected by both guessing and intrusions

Previous research has characterized source retrieval as a thresholded process, which fails on a proportion of trials and leads to guessing, as opposed to a continuous process, in which response precision varies across trials but is never zero. The thresholded view of source retrieval is largely based on the observation of heavy tailed distributions of response errors, thought to reflect a large proportion of "memoryless" trials. In this study, we investigate whether these errors might instead reflect systematic intrusions from other list items which can mimic source guessing. Using the circular diffusion model of decision making, which accounts for both response errors and RTs we found that intrusions account for some, but not all, errors in a continuous-report source memory task. We found that intrusion errors were more likely to come from items studied in nearby locations and times, and were well-described by a spatiotemporal gradient model, but not from semantically or perceptually similar cues. Our findings support a thresholded view of source retrieval but suggest that previous work has overestimated the proportion of guesses which have been conflated with intrusions.

Medial prefrontal cortex and hippocampal activity differentially contribute to ordinal and temporal context retrieval during sequence memory

Remembering sequences of events defines episodic memory, but retrieval can be driven by both ordinality and temporal contexts. Whether these modes of retrieval operate at the same time or not remains unclear. Theoretically, medial prefrontal cortex (mPFC) confers ordinality, while the hippocampus (HC) associates events in gradually changing temporal contexts. Here, we looked for evidence of each with BOLD fMRI in a sequence task that taxes both retrieval modes. To test ordinal modes, items were transferred between sequences but retained their position (e.g., AB3). Ordinal modes activated mPFC, but not HC. To test temporal contexts, we examined items that skipped ahead across lag distances (e.g., ABD). HC, but not mPFC, tracked temporal contexts. There was a mPFC and HC by retrieval mode interaction. These current results suggest that the mPFC and HC are concurrently engaged in different retrieval modes in support of remembering when an event occurred.

How do recall requirements affect decision-making in free recall initiation? A linear ballistic accumulator approach

Models of free recall describe free recall initiation as a decision-making process in which items compete to be retrieved. Recently, Osth and Farrell (Psychological Review, 126, 578–609, 2019) applied evidence accumulation models to complete RT distributions and serial positions of participants’ first recalls in free recall, which resulted in some novel conclusions about primacy and recency effects. Specifically, the results of the modeling favored an account in which primacy was due to reinstatement of the start-of-the-list, and recency was found to be exponential in shape. In this work, we examine what happens when participants are given alternative recall instructions. Prior work has demonstrated weaker primacy and greater recency when fewer items are required to report (Ward & Tan, Memory & Cognition, 2019), and a key question is whether this change in instructions qualitatively changes the nature of the recall process, or merely changes the parameters of the recall competition. We conducted an experiment where participants studied six- or 12-item lists and were post-cued as to whether to retrieve a single item, or as many items as possible. Subsequently, we applied LBA models with various assumptions about primacy and recency, implemented using hierarchical Bayesian techniques. While greater recency was observed when only one item was required for output, the model selection did not suggest that there were qualitative differences between the two conditions. Specifically, start-of-list reinstatement and exponential recency functions were favored in both conditions.

Multiple memories can be simultaneously reactivated during sleep as effectively as a single memory

Memory consolidation involves the reactivation of memory traces during sleep. If different memories are reactivated each night, how much do they interfere with one another? We examined whether reactivating multiple memories incurs a cost to sleep-related benefits by contrasting reactivation of multiple memories versus single memories during sleep. First, participants learned the on-screen location of different objects. Each object was part of a semantically coherent group comprised of either one, two, or six items (e.g., six different cats). During sleep, sounds were unobtrusively presented to reactivate memories for half of the groups (e.g., "meow"). Memory benefits for cued versus non-cued items were independent of the number of items in the group, suggesting that reactivation occurs in a simultaneous and promiscuous manner. Intriguingly, sleep spindles and delta-theta power modulations were sensitive to group size, reflecting the extent of previous learning. Our results demonstrate that multiple memories may be consolidated in parallel without compromising each memory's sleep-related benefit. These findings highlight alternative models for parallel consolidation that should be considered in future studies.

Explaining short-term memory phenomena with an integrated episodic/semantic framework of long-term memory

Current thinking about human memory is dominated by distinctions between episodic and semantic memory and between short-term memory (STM) and long-term memory (LTM). However, many memory phenomena seem to cut across these distinctions. This article attempts to set the groundwork for the issues that need to be resolved in generating an integrated model of long-term memory that incorporates semantic, episodic, and short-term memory. We contrast Nairne's (2002, Annual Review of Psychology) consensus account of short-term memory with a relatively generic theory of an integrated episodic-semantic memory. The later consists primarily of a list of principles which we and others argue are necessary to include in any theory of long-term memory. We then add some more specific assumptions to outline a modern theory of forgetting. We then turn to the issue of much of the phenomena thought to necessitate a dedicated short-term memory can be explained by an integrated theory of episodic and semantic memory. Our conclusion is that an integrated theory of long-term memory must be augmented to explain a small number of outstanding memory phenomena. Finally, we ask whether the augmentation needs to involve a dedicated mnemonic system, or sensory or language-based systems, which also have mnemonic capabilities.

Searching for Semantic Knowledge: A Vector Space Semantic Analysis of the Feature Generation Task

A recent neuropsychological study found that amnesic patients with hippocampal damage (HP) and severe declarative memory impairment produce markedly fewer responses than healthy comparison (CO) participants in a semantic feature generation task (Klooster and Duff, 2015), consistent with the idea that hippocampal damage is associated with semantic cognitive deficits. Participants were presented with a target word and asked to produce as many features of that word as possible (e.g., for target word "book," "read words on a page"). Here, we use the response sequences collected by Klooster and Duff (2015) to develop a vector space model of semantic search. We use this model to characterize the dynamics of semantic feature generation and consider the role of the hippocampus in this search process. Both HP and CO groups tended to initiate the search process with features close in semantic space to the target word, with a gradual decline in similarity to the target word over the first several responses. Adjacent features in the response sequence showed stronger similarity to each other than to non-adjacent features, suggesting that the search process follows a local trajectory in semantic space. Overall, HP patients generated features that were closer in semantic space to the representation of the target word, as compared to the features generated by the CO group, which ranged more widely in semantic space. These results are consistent with a model in which a compound retrieval cue (containing a representation of the target word and a representation of the previous response) is used to probe semantic memory. The model suggests that the HP group's search process is restricted from ranging as far in semantic space from the target word, relative to the CO group. These results place strong constraints on the structure of models of semantic memory search, and on the role of hippocampus in probing semantic memory.

New Perspectives on the Aging Lexicon

The field of cognitive aging has seen considerable advances in describing the linguistic and semantic changes that happen during the adult life span to uncover the structure of the mental lexicon (i.e., the mental repository of lexical and conceptual representations). Nevertheless, there is still debate concerning the sources of these changes, including the role of environmental exposure and several cognitive mechanisms associated with learning, representation, and retrieval of information. We review the current status of research in this field and outline a framework that promises to assess the contribution of both ecological and psychological aspects to the aging lexicon.

The role of control processes in temporal and semantic contiguity

Atkinson and Shiffrin (1968) argued that performance on any memory task reflects the combined influence of both the fixed structure of the memory system and control processes tailored to the specific task. We investigated the role of control processes in governing the temporal contiguity and semantic contiguity effects in free recall-tendencies to organize recall based on proximity in the study list and pre-existing semantic associations. Subjects studied lists that contained four "clusters", each composed of four semantically associated words but presented in random order such that associates were not in adjacent serial positions. Subjects were given either standard free-recall instructions, instructions to focus on order-based associations (i.e., the original order of presentation), or meaning-based associations (i.e., pre-existing semantic relationships). Replicating previous work, lists with strong semantic relationships resulted in a reduced overall temporal contiguity effect when recalled under standard free-recall instructions. However, under meaning-based recall instructions, the temporal contiguity effect was nearly eliminated. Detailed analyses of within-cluster transitions and an order reconstruction task revealed that temporal information was encoded, but control processes prevented it from dominating memory search. These results point to a need for more empirical work exploring how control processes change recall dynamics and for more theoretical work modeling the computational basis of these processes.

Neural activity reveals interactions between episodic and semantic memory systems during retrieval

Whereas numerous findings support a distinction between episodic and semantic memory, it is now widely acknowledged that these two forms of memory interact during both encoding and retrieval. The precise nature of this interaction, however, remains poorly understood. To examine the role of semantic organization during episodic encoding and retrieval, we recorded intracranial encephalographic signals as 69 neurosurgical patients studied and subsequently recalled categorized and unrelated word lists. Applying multivariate classifiers to neural recordings, we were able to reliably predict encoding success, retrieval success, and temporal and categorical clustering during recall. By assessing how these classifiers generalized across list types, we identified specific retrieval processes that predicted recall of categorized lists and distinguished between recall transitions within and between category clusters. These results particularly implicate retrieval (rather than encoding) processes in the categorical organization of episodic memories. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Temporal Proximity Links Unrelated News Events in Memory

Some memories are linked such that recalling one can trigger the retrieval of another. What determines which memories are linked? Some models predict that simply occurring close together in time is sufficient for links to form between memories. A competing theory suggests that temporal proximity is generally not sufficient, and existing evidence for such links is an artifact of using chainlike lists of items in artificial laboratory tasks. To test these competing accounts, we asked subjects to recall news stories that they had encountered over the past 2 years (Experiment 1) or 4 months (Experiment 2). In both experiments, subjects showed a strong bias to successively recall stories that appeared in the news within days of each other—even after accounting for the fact that stories that occur close in time tend to be semantically related. By moving beyond laboratory tasks, this research solidifies the foundation of contemporary memory theory.

Lingering representations of stimuli influence recall organization

Several prominent theories posit that information about recent experiences lingers in the brain and organizes memories for current experiences, by forming a temporal context that is linked to those memories at encoding. According to these theories, if the thoughts preceding an experience X resemble the thoughts preceding an experience Y, then X and Y should show an elevated probability of being recalled together. We tested this prediction by using multi-voxel pattern analysis (MVPA) of fMRI data to measure neural evidence for lingering processing of preceding stimuli. As predicted, memories encoded with similar lingering thoughts about the category of preceding stimuli were more likely to be recalled together. Our results demonstrate that the "fading embers" of previous stimuli help to organize recall, confirming a key prediction of computational models of episodic memory.

Temporal binding within and across events

Remembering the order in which events occur is a fundamental component of episodic memory. However, the neural mechanisms supporting serial recall remain unclear. Behaviorally, serial recall is greater for information encountered within the same event compared to across event boundaries, raising the possibility that contextual stability may modulate the cognitive and neural processes supporting serial encoding. In the present study, we used fMRI during the encoding of consecutive face and object stimuli to elucidate the neural encoding signatures supporting subsequent serial recall behavior both within and across events. We found that univariate BOLD activation in both the middle hippocampus and left ventrolateral prefrontal cortex (PFC) was associated with subsequent serial recall of items that occur across event boundaries. By contrast, successful serial encoding within events was associated with increased functional connectivity between the hippocampus and ventromedial PFC, but not with univariate activation in these or other regions. These findings build on evidence implicating hippocampal and PFC processes in encoding temporal aspects of memory. They further suggest that these encoding processes are influenced by whether binding occurs within a stable context or bridges two adjacent but distinct events.