Benefits of flexible prioritization in working memory can arise without costs

The self-bias in working memory: the favorability of self-referential stimuli in resource allocation

Self-representations guide and shape our thoughts and behaviour. People usually exhibit inherent biases in perception, attention, and memory to favour the information associated with themselves over that associated with others. The present study explored the phenomenon of self-bias in working memory (WM), specifically how self-referential processing impacts WM precision. Four precision-based experiments were conducted to assess the recall precision of self-referential items and items associated with other social agents. The findings revealed a robust self-prioritisation effect in WM precision, wherein self-referential items were recalled with greater precision than items associated with other social agents. Additionally, increased precision for self-referential items did not decrease the precision for simultaneously remembered items. This effect was limited by the total amount of WM resources and not influenced by a perceptual distractor. The inherent self-bias in WM can serve as a proxy to access the role self-representation in goal-oriented cognitive processing, providing a means of exploring the interaction between self-reference and high-level cognitive function.

Shifting attention between perception and working memory

Most everyday tasks require shifting the focus of attention between sensory signals in the external environment and internal contents in working memory. To date, shifts of attention have been investigated within each domain, but shifts between the external and internal domain remain poorly understood. We developed a combined perception and working-memory task to investigate and compare the consequences of shifting spatial attention within and between domains in the service of a common orientation-reproduction task. Participants were sequentially cued to attend to items either in working memory or to an upcoming sensory stimulation. Stay trials provided a baseline condition, while shift trials required participants to shift their attention to another item within the same or different domain. Validating our experimental approach, we found evidence that participants shifted attention effectively in either domain (Experiment 1). In addition, we observed greater costs when transitioning attention between as compared to within domains (Experiments 1, 2). Strikingly, these costs persisted even when participants were given more time to complete the attentional shift (Experiment 2). Biases in fixational gaze behaviour tracked attentional orienting in both domains, but revealed no latency or magnitude difference for within- versus between-domain shifts (Experiment 1). Collectively, the results from Experiments 1 and 2 suggest that shifting between attentional domains might be regulated by a unique control function. Our results break new ground for exploring the ubiquitous act of shifting attention between perception and working memory to guide adaptive behaviour in everyday cognition.

Presaccadic attention does not facilitate the detection of changes in the visual field

Planning a rapid eye movement (saccade) changes how we perceive our visual world. Even before we move the eyes visual discrimination sensitivity improves at the impending target of eye movements, a phenomenon termed "presaccadic attention." Yet, it is unknown if such presaccadic selection merely affects perceptual sensitivity, or also affects downstream decisional processes, such as choice bias. We report a surprising lack of presaccadic perceptual benefits in a common, everyday setting-detection of changes in the visual field. Despite the lack of sensitivity benefits, choice bias for reporting changes increased reliably for the saccade target. With independent follow-up experiments, we show that presaccadic change detection is rendered more challenging because percepts at the saccade target location are biased toward, and more precise for, only the most recent of two successive stimuli. With a Bayesian model, we show how such perceptual and choice biases are crucial to explain the effects of saccade plans on change detection performance. In sum, visual change detection sensitivity does not improve presaccadically, a result that is readily explained by teasing apart distinct components of presaccadic selection. The findings may have critical implications for real-world scenarios, like driving, that require rapid gaze shifts in dynamically changing environments.

The slow rate of working memory consolidation from vision is a structural limit

The speed with which information from vision is transformed into working memory (WM) representations that resist interference from ongoing perception and cognition is the subject of conflicting results. Using distinct paradigms, researchers have arrived at estimates of the consolidation time course ranging from 25 ms to 1 s - a range of more than an order of magnitude. However, comparisons of consolidation duration across very different estimation paradigms rely on the implicit assumption that WM consolidation speed is a stable, structural constraint of the WM system. The extremely large variation in WM consolidation speed estimates across measurement approaches motivated the current work's goal of determining whether consolidation speed truly is a stable structural constraint of WM encoding, or instead might be under strategic control as suggested by some accounts. By manipulating the relative task priority of WM encoding and a subsequent sensorimotor decision in a dual-task paradigm, the current experiments demonstrate that the long duration of WM consolidation does not change as a result of task-specific strategies. These results allow comparison of WM consolidation across estimation approaches, are consistent with recent multi-phase WM consolidation models, and are consistent with consolidation duration being an inflexible structural limit.

Differences of resource allocation to active and passive states in visual working memory

Scholars have long sought to understand the separation between the active and passive states in visual working memory. Results of recent behavioral studies have provided insight into the independence of storage resources in these two states. To explore how humans distribute these resources in the active and passive states in visual working memory, we adopted the classic double-retro-cue paradigm combined with a continuous reported color wheel to ascertain whether the precision of representations maintained in active and passive states are adjustable according to the frequency of spatial cues. The results showed that two distinct resource allocation mechanisms exist in these two states beyond traditional visual working memory theory and provide further support for the separation hypothesis.

Serial dependence in visual perception: A meta-analysis and review

Positive sequential dependencies are phenomena in which actions, perception, decisions, and memory of features or objects are systematically biased toward visual experiences from the recent past. Among many labels, serial dependencies have been referred to as priming, sequential dependencies, sequential effects, or serial effects. Despite extensive research on the topic, the field still lacks an operational definition of what counts as serial dependence. In this meta-analysis, we review the vast literature on serial dependence and quantitatively assess its key diagnostic characteristics across several different domains of visual perception. The meta-analyses fully characterize serial dependence in orientation, face, and numerosity perception. They show that serial dependence is defined by four main kinds of tuning: serial dependence decays with time (temporal-tuning), it depends on relative spatial location (spatial-tuning), it occurs only between similar features and objects (feature-tuning), and it is modulated by attention (attentional-tuning). We also review studies of serial dependence that report single observer data, highlighting the importance of individual differences in serial dependence. Finally, we discuss a range of outstanding questions and novel research avenues that are prompted by the meta-analyses. Together, the meta-analyses provide a full characterization of serial dependence as an operationally defined family of visual phenomena, and they outline several of the key diagnostic criteria for serial dependence that should serve as guideposts for future research.

Spatial Configuration Effects on the Dissociation between Active and Latent States in Visual Working Memory

Visual working memory includes both active, recallable items and latent items not directly available for recall. During the online manipulation of active working memory, latent items gain robust retention. According to the dissociation account, active and passive memories exist in independent states, both of which take up their own cognitive resources. However, it is unclear whether dissociation is a universal phenomenon during memory maintenance. Given that memory information is retained as a binding of identity and location, the spatial proximity of memory items might shield the dissociation property. To test this, we adopted a retro-cue memory task where the cued and uncued items were separated in the active and latent states. In Experiment 1, the memory items were presented at a relatively large spatial distance. The results supported the dissociation account for well-separated items. However, Experiment 2 rejected the dissociation for closer-in items, possibly because items in visual working memory were spatially labeled. These findings suggest that while memory maintenance generally conforms to the dissociation account, the spatial configuration of memory items can affect the dissociation property between the active and latent neural states in visual working memory.

Attention in flux

Selective attention comprises essential infrastructural functions supporting cognition-anticipating, prioritizing, selecting, routing, integrating, and preparing signals to guide adaptive behavior. Most studies have examined its consequences, systems, and mechanisms in a static way, but attention is at the confluence of multiple sources of flux. The world advances, we operate within it, our minds change, and all resulting signals progress through multiple pathways within the dynamic networks of our brains. Our aim in this review is to raise awareness of and interest in three important facets of how timing impacts our understanding of attention. These include the challenges posed to attention by the timing of neural processing and psychological functions, the opportunities conferred to attention by various temporal structures in the environment, and how tracking the time courses of neural and behavioral modulations with continuous measures yields surprising insights into the workings and principles of attention.

Turning Attention Inside Out: How Working Memory Serves Behavior

Flexible behavior requires guidance not only by sensations that are available immediately but also by relevant mental contents carried forward through working memory. Therefore, selective-attention functions that modulate the contents of working memory to guide behavior (inside-out) are just as important as those operating on sensory signals to generate internal contents (outside-in). We review the burgeoning literature on selective attention in the inside-out direction and underscore its functional, flexible, and future-focused nature. We discuss in turn the purpose (why), targets (what), sources (when), and mechanisms (how) of selective attention inside working memory, using visual working memory as a model. We show how the study of internal selective attention brings new insights concerning the core cognitive processes of attention and working memory and how considering selective attention and working memory together paves the way for a rich and integrated understanding of how mind serves behavior.

Opening Questions in Visual Working Memory

In this reflective piece on visual working memory, I depart from the laboriously honed skills of writing a review. Instead of integrating approaches, synthesizing evidence, and building a cohesive perspective, I scratch my head and share niggles and puzzlements. I expose where my scholarship and understanding are stumped by findings and standard views in the literature.

Effects of Attention Direction and Perceptual Distraction Within Visual Working Memory

Although substantial evidence demonstrates that directing attention to specific items is important for improving the performance of visual working memory (VWM), it is still not clear whether the attended items were better protected. The present study, thus, adopted a pre-cueing paradigm to examine the effect of attention direction and perceptual distractor on VWM. The results showed that a valid visual cue improved the individuals’ VWM performances and reduced their reaction time compared to the invalid and neutral cues. However, the VWM performances in the valid and neutral cue conditions were more disrupted by a post-stimuli distractor compared to the invalid cue condition. The findings suggest that although directing attention can improve the VWM performance, it is not efficient in protecting information from being distracted.

A Hierarchy of Functional States in Working Memory

Extensive research has examined how information is maintained in working memory (WM), but it remains unknown how WM is used to guide behavior. We addressed this question by combining human electrophysiology (50 subjects, male and female) with pattern analyses, cognitive modeling, and a task requiring the prolonged maintenance of two WM items and priority shifts between them. This enabled us to discern neural states coding for memories that were selected to guide the next decision from states coding for concurrently held memories that were maintained for later use, and to examine how these states contribute to WM-based decisions. Selected memories were encoded in a functionally active state. This state was reflected in spontaneous brain activity during the delay period, closely tracked moment-to-moment fluctuations in the quality of evidence integration, and also predicted when memories would interfere with each other. In contrast, concurrently held memories were encoded in a functionally latent state. This state was reflected only in stimulus-evoked brain activity, tracked memory precision at longer timescales, but did not engage with ongoing decision dynamics. Intriguingly, the two functional states were highly flexible, as priority could be dynamically shifted back and forth between memories without degrading their precision. These results delineate a hierarchy of functional states, whereby latent memories supporting general maintenance are transformed into active decision circuits to guide flexible behavior.SIGNIFICANCE STATEMENT Working memory enables maintenance of information that is no longer available in the environment. Abundant neuroscientific work has examined where in the brain working memories are stored, but it remains unknown how they are represented and used to guide behavior. Our study shows that working memories are represented in qualitatively different formats, depending on behavioral priorities. Memories that are selected for guiding behavior are encoded in an active state that transforms sensory input into decision variables, whereas other concurrently held memories are encoded in a latent state that supports precise maintenance without affecting ongoing cognition. These results dissociate mechanisms supporting memory storage and usage, and open the door to reveal not only where memories are stored but also how.

Training-induced improvement in working memory tasks results from switching to efficient strategies

It is debated whether training with a working memory (WM) task, particularly n-back, can improve general WM and reasoning skills. Most training studies found substantial improvement in the trained task, with little to no transfer to untrained tasks. We hypothesized that training does not increase WM capacity, but instead provides opportunities to develop an efficient task-specific strategy. We derived a strategy for the task that optimizes WM resources and taught it to participants. In two sessions, 14 participants who were taught this strategy performed as well as fourteen participants who trained for 40 sessions without strategy instructions. To understand the mechanisms underlying the no-instruction group's improvement, participants answered questionnaires during their training period. Their replies indicate that successful learners discovered the same strategy and their improvement was associated with this discovery. We conclude that n-back training allows the discovery of strategies that enable better performance with the same WM resources.

Frontoparietal action-oriented codes support novel instruction implementation

A key aspect of human cognitive flexibility concerns the ability to convert complex symbolic instructions into novel behaviors. Previous research proposes that this transformation is supported by two neurocognitive states: an initial declarative maintenance of task knowledge, and an implementation state necessary for optimal task execution. Furthermore, current models predict a crucial role of frontal and parietal brain regions in this process. However, whether declarative and procedural signals independently contribute to implementation remains unknown. We report the results of an fMRI experiment in which participants executed novel instructed stimulus-response associations. We then used a pattern-tracking procedure to quantify the contribution of format-unique signals during instruction implementation. This revealed independent procedural and declarative representations of novel S-Rs in frontoparietal areas, prior to execution. Critically, the degree of procedural activation predicted subsequent behavioral performance. Altogether, our results suggest an important contribution of frontoparietal regions to the neural architecture that regulates cognitive flexibility.

Prioritization within visual working memory reflects a flexible focus of attention

When motivated, people can keep nonrecent items in a list active during the presentation of new items, facilitating fast and accurate recall of the earlier items. It has been proposed that this occurs by flexibly orienting attention to a single prioritized list item, thus increasing the amount of attention-based maintenance directed toward this item at the expense of other items. This is manipulated experimentally by associating a single distinct feature with a higher reward value, such as a single red item in a list of black items. These findings may be more parsimoniously explained under a distinctiveness of encoding framework rather than a flexible attention allocation framework. The retrieval advantage for the prioritized list position may be because the incongruent feature stands out in the list perceptually and causes it to become better encoded. Across three visual working memory experiments, we contrast a flexible attention theory against a distinctiveness of encoding theory by manipulating the reward value associated with the incongruent feature. Findings from all three experiments show strong support in favor of the flexible attention theory and no support for the distinctiveness of encoding theory. We also evaluate and find no evidence that strategy use, motivation, or tiredness/fatigue associated with reward value can adequately explain flexible prioritization of attention. Flexible attentional prioritization effects may be best understood under the context of an online attentional refreshing mechanism.

Prioritized verbal working memory content biases ongoing action

Working memory (WM) holds information temporarily in mind, imparting the ability to guide behavior based on internal goals rather than external stimuli. However, humans often maintain WM content for a future task while performing more immediate actions. Consequently, transient WM representations may inadvertently influence ongoing (but unrelated) motor behavior. Here, we tested the impact of WM on adult human action execution and examined how the attentional or "activation" state of WM content modulates that impact. In 3 dual-task experiments, verbal WM for directional words influenced the trajectory and speed of hand movements performed during WM maintenance. This movement bias was also modulated by the attentional state of the WM content. Prioritized WM content strongly influenced actions during WM maintenance, while de-prioritized WM content was less influential. In summary, WM can unintentionally shape ongoing motor behavior, but the behavioral relevance of WM content determines the degree of influence on motor output. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Distraction biases working memory for faces

Working memory persists in the face of distraction, yet not without consequence. Previous research has shown that memory for low-level visual features is systematically influenced by the maintenance or presentation of a similar distractor stimulus. Responses are frequently biased in stimulus space towards a perceptual distractor, though this has yet to be determined for high-level stimuli. We investigated whether these influences are shared for complex visual stimuli such as faces. To quantify response accuracies for these stimuli, we used a delayed-estimation task with a computer-generated "face space" consisting of 80 faces that varied continuously as a function of age and sex. In a set of three experiments, we found that responses for a target face held in working memory were biased towards a distractor face presented during the maintenance period. The amount of response bias did not vary as a function of distance between target and distractor. Our data suggest that, similar to low-level visual features, high-level face representations in working memory are biased by the processing of related but task-irrelevant information.

Theoretical distinction between functional states in working memory and their corresponding neural states

Working memory (WM) is important for guiding behaviour, but not always for the next possible action. Here we define a WM item that is currently relevant for guiding behaviour as the functionally "active" item; whereas items maintained in WM, but not immediately relevant to behaviour, are defined as functionally "latent". Traditional neurophysiological theories of WM proposed that content is maintained via persistent neural activity (e.g., stable attractors); however, more recent theories have highlighted the potential role for "activity-silent" mechanisms (e.g., short-term synaptic plasticity). Given these somewhat parallel dichotomies, functionally active and latent cognitive states of WM have been associated with storage based on persistent-activity and activity-silent neural mechanisms, respectively. However, in this article we caution against a one-to-one correspondence between functional and activity states. We argue that the principal theoretical requirement for active and latent WM is that the corresponding neural states play qualitatively different functional roles. We consider a number of candidate solutions, and conclude that the neurophysiological mechanisms for functionally active and latent WM items are theoretically independent of the distinction between persistent activity-based and activity-silent forms of WM storage.

Premembering Experience: A Hierarchy of Time-Scales for Proactive Attention

Memories are about the past, but they serve the future. Memory research often emphasizes the former aspect: focusing on the functions that re-constitute (re-member) experience and elucidating the various types of memories and their interrelations, timescales, and neural bases. Here we highlight the prospective nature of memory in guiding selective attention, focusing on functions that use previous experience to anticipate the relevant events about to unfold-to "premember" experience. Memories of various types and timescales play a fundamental role in guiding perception and performance adaptively, proactively, and dynamically. Consonant with this perspective, memories are often recorded according to expected future demands. Using working memory as an example, we consider how mnemonic content is selected and represented for future use. This perspective moves away from the traditional representational account of memory toward a functional account in which forward-looking memory traces are informationally and computationally tuned for interacting with incoming sensory signals to guide adaptive behavior.

Working memory prioritization impacts neural recovery from distraction

The ability to protect goal-relevant information from disruption over short intervals is a hallmark of working memory. Recent behavioral data suggest that high-priority items in working memory are more vulnerable to disruption. We used functional magnetic resonance imaging to evaluate the hypothesis that prioritization of working memories might impact the recovery of their neural representation(s) after distraction. A delay-period retrospective cue informed participants which of two memory items (a face or a scene) to prioritize during a first delay period. Consistent with prior work, and confirming successful prioritization, multivoxel pattern classifier evidence in perceptual brain regions was higher for cued versus uncued memory items. A distraction task was then imposed before a second retrospective cue informed participants to either "stay" remembering the previously cued item or "switch" to the previously uncued item. This allowed for the evaluation of recovery for high-priority items (on stay trials) and also low-priority items (on switch trials). Classifiers showed successful reinstatement of both high- and low-priority items after distraction, but only low-priority items recovered to their pre-distraction representational levels. Moreover, the degree of prioritization before distraction predicted the amount of disruption for high-priority items after distraction, suggesting that the more a participant prioritized the cued item, the greater the impact of distraction. Our data provide neural evidence that prioritizing working memory information in perceptual regions makes that information more vulnerable to disruption.

EEG dynamics reveal a dissociation between storage and selective attention within working memory

Selective attention plays a prominent role in prioritizing information in working memory (WM), improving performance for attended representations. However, it remains unclear whether unattended WM representations suffer from information loss. Here we tested the hypothesis that within WM, selectively attending to an item and stopping storing other items are independent mechanisms. We recorded EEG while participants performed a WM recall task in which the item most likely to be tested was cued retrospectively during retention. By manipulating retro-cue reliability (i.e., the ratio of valid to invalid cue trials), we varied the incentive to retain non-cued items. Storage and selective attention in WM were measured during the retention interval by contralateral delay activity (CDA) and contralateral alpha power suppression, respectively. Soon after highly reliable cues, the cued item was attended, and non-cued items suffered information loss. However, for less reliable cues, initially the cued item was attended, but unattended items were kept in WM. Later during the delay, previously unattended items suffered information loss despite now attention being reallocated to their locations, presumably to strengthen their weakening traces. These results show that storage and attention in WM are distinct processes that can behave differently depending on the relative importance of representations.

The offline stream of conscious representations

When do we become conscious of a stimulus after its presentation? We would all agree that this necessarily takes time and that it is not instantaneous. Here, I would like to propose not only that conscious access is delayed relative to the external stimulation, but also that it can flexibly desynchronize from external stimulation; it can process some information 'offline', if and when it becomes relevant. Thus, in contrast with initial sensory processing, conscious experience might not strictly follow the sequence of events in the environment. In this article, I will review gathering evidence in favour of this proposition. I will argue that it offers a coherent framework for explaining a great variety of observations in the domain of perception, sensory memory and working memory: the psychological refractory period, the attentional blink, post-dictive phenomena, iconic memory, latent working memory and the newly described retro-perception phenomenon. I will integrate this proposition to the global neuronal workspace model and consider possible underlying brain mechanisms. Finally, I will argue that this capacity to process information 'offline' might have made conscious processing evolutionarily advantageous in spite of its sluggishness and capacity limitations.This article is part of the theme issue 'Perceptual consciousness and cognitive access'.

Priority Switches in Visual Working Memory are Supported by Frontal Delta and Posterior Alpha Interactions

Visual working memory (VWM) distinguishes between representations relevant for imminent versus future perceptual goals. We investigated how the brain sequentially prioritizes visual working memory representations that serve consecutive tasks. Observers remembered two targets for a sequence of two visual search tasks, thus making one target currently relevant, and the other prospectively relevant. We show that during the retention interval prior to the first search, lateralized parieto-occipital EEG alpha (8-14 Hz) suppression is stronger for current compared with prospective search targets. Crucially, between the first and second search task, this difference in posterior alpha lateralization reverses, reflecting the change in priority states of the two target representations. Connectivity analyses indicate that this switch in posterior alpha lateralization is driven by frontal delta/low-theta (2-6 Hz) activity. Moreover, this frontal low-frequency signal also predicts task performance after the switch. We thus obtained evidence for large-scale network interactions underlying the flexible shifting between the priority states of multiple memory representations in VWM.

Are there multiple ways to direct attention in working memory?

In visual working memory tasks, memory for an item is enhanced if participants are told that the item is relatively more valuable than others presented within the same trial. Experiment 1 explored whether these probe value boosts (termed prioritization effects in previous literature) are affected by probe frequency (i.e., how often the more valuable item is tested). Participants were presented with four colored shapes sequentially and asked to recall the color of one probed item following a delay. They were informed that the first item was more valuable (differential probe value) or as valuable as the other items (equal probe value), and that this item would be tested more frequently (differential probe frequency) or as frequently (equal probe frequency) as the other items. Probe value and probe frequency boosts were observed at the first position, though both were accompanied by costs to other items. Probe value and probe frequency boosts were additive, suggesting the manipulations yield independent effects. Further supporting this, experiment 2 revealed that probe frequency boosts are not reliant on executive resources, directly contrasting with previous findings regarding probe value. Taken together, these outcomes suggest there may be several ways in which attention can be directed in working memory.