Decomposing neurophysiological underpinnings of age-related decline in visual working memory

Exploring the neural basis of age-related decline in working memory is vital in our aging society. Previous electroencephalographic studies suggested that the contralateral delay activity (CDA) may be insensitive to age-related decline in lateralized visual working memory (VWM) performance. Instead, recent evidence indicated that task-induced alpha power lateralization decreases in older age. However, the relationship between alpha power lateralization and age-related decline of VWM performance remains unknown, and recent studies have questioned the validity of these findings due to confounding factors of the aperiodic signal. Using a sample of 134 participants, we replicated the age-related decrease of alpha power lateralization after adjusting for the aperiodic signal. Critically, the link between task performance and alpha power lateralization was found only when correcting for aperiodic signal biases. Functionally, these findings suggest that age-related declines in VWM performance may be related to the decreased ability to prioritize relevant over irrelevant information. Conversely, CDA amplitudes were stable across age groups, suggesting a distinct neural mechanism possibly related to preserved VWM encoding or early maintenance.

Hearing fearful prosody impairs visual working memory maintenance

Interference by distractors has been associated multiple times with diminished visual and auditory working memory (WM) performance. Negative emotional distractors in particular lead to detrimental effects on WM. However, these associations have only been seen when distractors and items to maintain in WM are from the same sensory modality. In this study, we investigate cross-modal interference on WM. We invited 20 participants to complete a visual change-detection task, assessing visual WM (VWM), while hearing emotional (fearful) and neutral auditory distractors. Electrophysiological activity was recorded to measure contralateral delay activity (CDA) and auditory P2 event-related potentials (ERP), indexing WM maintenance and distractor salience respectively. At the behavioral level, fearful prosody didn't decrease significantly working memory accuracy, compared to neutral prosody. Regarding ERPs, fearful distractors evoked a greater P2 amplitude than neutral distractors. Correlations between the two ERP potentials indicated that P2 amplitude difference between the two types of prosody was associated with the difference in CDA amplitude for fearful and neutral trials. This association suggests that cognitive resources required to process fearful prosody detrimentally impact VWM maintenance. That result provides a piece of additional evidence that negative emotional stimuli produce greater interference than neutral stimuli and that the cognitive resources used to process stimuli from different modalities come from a common pool.

Distinct mechanisms of attentional suppression: exploration of trait factors underlying cued- and learned-suppression

Attention allows us to focus on relevant information while ignoring distractions. Effective suppression of distracting information is crucial for efficient visual search. Recent studies have developed two paradigms to investigate attentional suppression: cued-suppression which is based on top-down control, and learned-suppression which is based on selection history. While both types of suppression reportedly engage proactive control, it remains unclear whether they rely on shared mechanisms. This study aimed to determine the relationship between cued- and learned-suppression. In a within-subjects design, 54 participants performed a cued-suppression task where pre-cues indicated upcoming target or distractor colors, and a learned-suppression task where a salient color distractor was present or absent. No significant correlation emerged between performance in the two tasks, suggesting distinct suppression mechanisms. Cued-suppression correlated with visual working memory capacity, indicating reliance on explicit control. In contrast, learned-suppression correlated with everyday distractibility, suggesting implicit control based on regularities. These results provide evidence for heterogeneous proactive control mechanisms underlying cued- and learned-suppression. While both engage inhibition, cued-suppression relies on deliberate top-down control modulated by working memory, whereas learned-suppression involves implicit suppression shaped by selection history and distractibility traits.

The independent storage mechanisms of visual and vibrotactile working memory

Whether information in working memory (WM) is stored in a domain-independent or domain-specific system is still the subject of intense debate. This study used the delayed match-to-sample paradigm, the dual-task paradigm, and the selective interference paradigm to investigate the mechanism of cross-modal storage in visual and vibrotactile WM. We postulated that WM may store cross-modal data from haptics and vision independently, and we proposed domain-specific WM storage. According to the findings, the WM can store cross-modal information from vision and haptics independently, and the storage of visual and tactile WM may be domain-specific. This study provides early support for the hypothesis that haptic and visuospatial sketchpads are dissociated. In addition, the current study provides evidence to elucidate the mechanisms by which WM stores and processes data from different modalities and content. The results also indicate that a cross-modal approach can broaden the cognitive processing bandwidth of WM.

Multisensory subtypes of aphantasia: Mental imagery as supramodal perception in reverse

Cognitive neuroscience research on mental imagery has largely focused on the visual imagery modality in unimodal task contexts. Recent studies have uncovered striking individual differences in visual imagery capacity, with some individuals reporting a subjective absence of conscious visual imagery ability altogether ("aphantasia"). However, naturalistic mental imagery is often multi-sensory, and preliminary findings suggest that many individuals with aphantasia also report a subjective lack of mental imagery in other sensory domains (such as auditory or olfactory imagery). In this paper, we perform a series of cluster analyses on the multi-sensory imagery questionnaire scores of two large groups of aphantasic subjects, defining latent sub-groups in this sample population. We demonstrate that aphantasia is a heterogenous phenomenon characterised by dominant sub-groups of individuals with visual aphantasia (those who report selective visual imagery absence) and multi-sensory aphantasia (those who report an inability to generate conscious mental imagery in any sensory modality). We replicate our findings in a second large sample and show that more unique aphantasia sub-types also exist, such as individuals with selectively preserved mental imagery in only one sensory modality (e.g. intact auditory imagery). We outline the implications of our findings for network theories of mental imagery, discussing how unique aphantasia aetiologies with distinct self-report patterns might reveal alterations to various levels of the sensory processing hierarchy implicated in mental imagery.

Mapping visual working memory models to a theoretical framework

The body of research on visual working memory (VWM)-the system often described as a limited memory store of visual information in service of ongoing tasks-is growing rapidly. The discovery of numerous related phenomena, and the many subtly different definitions of working memory, signify a challenge to maintain a coherent theoretical framework to discuss concepts, compare models and design studies. A lack of robust theory development has been a noteworthy concern in the psychological sciences, thought to be a precursor to the reproducibility crisis (Oberauer & Lewandowsky, Psychonomic Bulletin & Review, 26, 1596-1618, 2019). I review the theoretical landscape of the VWM field by examining two prominent debates-whether VWM is object-based or feature-based, and whether discrete-slots or variable-precision best describe VWM limits. I share my concerns about the dualistic nature of these debates and the lack of clear model specification that prevents fully determined empirical tests. In hopes of promoting theory development, I provide a working theory map by using the broadly encompassing memory for latent representations model (Hedayati et al., Nature Human Behaviour, 6, 5, 2022) as a scaffold for relevant phenomena and current theories. I illustrate how opposing viewpoints can be brought into accordance, situating leading models of VWM to better identify their differences and improve their comparison. The hope is that the theory map will help VWM researchers get on the same page-clarifying hidden intuitions and aligning varying definitions-and become a useful device for meaningful discussions, development of models, and definitive empirical tests of theories.

Comparing memory capacity across stimuli requires maximally dissimilar foils: Using deep convolutional neural networks to understand visual working memory capacity for real-world objects

The capacity of visual working and visual long-term memory plays a critical role in theories of cognitive architecture and the relationship between memory and other cognitive systems. Here, we argue that before asking the question of how capacity varies across different stimuli or what the upper bound of capacity is for a given memory system, it is necessary to establish a methodology that allows a fair comparison between distinct stimulus sets and conditions. One of the most important factors determining performance in a memory task is target/foil dissimilarity. We argue that only by maximizing the dissimilarity of the target and foil in each stimulus set can we provide a fair basis for memory comparisons between stimuli. In the current work we focus on a way to pick such foils objectively for complex, meaningful real-world objects by using deep convolutional neural networks, and we validate this using both memory tests and similarity metrics. Using this method, we then provide evidence that there is a greater capacity for real-world objects relative to simple colors in visual working memory; critically, we also show that this difference can be reduced or eliminated when non-comparable foils are used, potentially explaining why previous work has not always found such a difference. Our study thus demonstrates that working memory capacity depends on the type of information that is remembered and that assessing capacity depends critically on foil dissimilarity, especially when comparing memory performance and other cognitive systems across different stimulus sets.

The influence of depth on object selection and manipulation in visual working memory within a 3D context

Recent studies have examined whether the internal selection mechanism functions similarly for perception and visual working memory (VWM). However, the process of how we access and manipulate object representations distributed in a 3D space remains unclear. In this study, we utilized a memory search task to investigate the effect of depth on object selection and manipulation within VWM. The memory display consisted of colored items half positioned at the near depth plane and the other half at the far plane. During memory maintenance, the participants were instructed to search for a target representation and update its color. The results showed that under object-based attention (Experiments 1, 3, and 5), the update time was faster for targets at the near plane than for those at the far plane. This effect was absent in VWM when deploying spatial attention (Experiment 2) and in visual search regardless of the type of attention deployed (Experiment 4). The differential effects of depth on spatial and object-based attention in VWM suggest that spatial attention primarily relied on 2D location information irrespective of depth, whereas object-based attention seemed to prioritize memory representations at the front plane before shifting to the back. Our findings shed light on the interaction between depth perception and the selection mechanisms within VWM in a 3D context, emphasizing the importance of ordinal, rather than metric, spatial information in guiding object-based attention in VWM.

Suppression on the basis of template for rejection is reactive: Evidence from human electrophysiology

According to most theories of attention, the selection of task-relevant visual information can be enhanced by holding them in visual working memory (VWM). However, there has been a long-standing debate concerning whether similar optimization can also be achieved for task-irrelevant information, known as a "template for rejection". The present study aimed to explore this issue by examining the consequence of cue distractors before visual search tasks. For this endeavor, we manipulated the display heterogeneity by using two distractor conditions, salient and non-salient, to explore the extent to which holding the distractor color in VWM might affect attentional selection. We measured the reaction times of participants while their EEG activity was recorded. The results showed that WM-matched distractors did not improve reaction times but rather slowed them down in both tasks. Event-related potential (ERP) results showed that the display heterogeneity had no modulatory effect on the degree of distractor suppression. Even in the salient distractor condition, the WM-matched distractor received no greater suppression. Furthermore, the WM-matched distractor but not the neutral distractor elicited an N2pc before the PD in salient distractor conditions. This suggests that the template for rejection operates reactively since suppression occurs after extra attentional processes to the distractor. Moreover, the presence of WM-matched distractors led to a reduction of P3b, indicating a competition between target processing and WM-matched distractor rejection. Our findings provide insights into the mechanisms underlying the optimization of attentional selection, and have implications for future studies aimed at understanding the role of VWM in cognition.

The impact of caregiver inhibitory control on infant visual working memory

Visual working memory (VWM) emerges in the first year of life and has far-reaching implications for academic and later life outcomes. Given that caregivers play a significant role in shaping cognitive function in children, it is important to understand how they might impact VWM development as early as infancy. The current study investigated whether caregivers' efficiency of regulating inhibitory control was associated with VWM function in their infants. Eighty-eight caregivers were presented with a Go-NoGo task to assess inhibitory control. An efficiency score was calculated using their behavioural responses. Eighty-six 6-to-10-month-old infants were presented with a preferential looking task to assess VWM function. VWM load was manipulated across one (low load), two (medium load) and three (high load) items. Functional near-infrared spectroscopy was used to record brain activation from caregivers and their infants. We found no direct association between caregiver efficiency and infant VWM behaviour. However, we found an indirect association - caregiver efficiency was linked to infant VWM through left-lateralized fronto-parietal engagement. Specifically, infants with low efficiency caregivers showed decreasing left-lateralized parietal engagement with increasing VWM performance at the medium and high loads compared to infants with high efficiency caregivers, who did not show any load- or performance-dependent modulation. Our findings contribute to a growing body of literature examining the role that caregivers play in early neurocognitive development.

Investigating the effects of perceptual complexity versus conceptual meaning on the object benefit in visual working memory

Previous research has demonstrated greater visual working memory (VWM) performance for real-world objects compared with simple features. Greater amplitudes of the contralateral delay activity (CDA)-a sustained event-related potential measured during the delay period of a VWM task-have also been noted for meaningful stimuli, despite being thought of as a neural marker of a fixed working memory capacity. The current study aimed to elucidate the factors underlying improved memory performance for real-world objects by isolating the relative contributions of perceptual complexity (i.e., number of visual features) and conceptual meaning (i.e., availability of semantic, meaningful features). Participants (N = 22) performed a lateralized VWM task to test their memory of intact real-world objects, scrambled real-world objects and colours. The CDA was measured during both encoding and WM retention intervals (600-1000 ms and 1300-1700 ms poststimulus onset, respectively), and behavioural performance was estimated by using d' (memory strength in a two-alternative forced choice task). Behavioural results revealed significantly better performance within-subjects for real-world objects relative to scrambled objects and colours, with no difference between colours and scrambled objects. The amplitude of the CDA was also largest for intact real-world objects, with no difference in magnitude for scrambled objects and colours, during working memory maintenance. However, during memory encoding, both the colours and intact real-world objects had significantly greater amplitudes than scrambled objects and were comparable in magnitude. Overall, findings suggest that conceptual meaning (semantics) supports the memory benefit for real-world objects.

The role of visual working memory in capacity-limited cross-modal ensemble coding

Ensemble coding refers to the brain's ability to rapidly extract summary statistics, such as average size and average cost, from a large set of visual stimuli. Although ensemble coding is thought to circumvent a capacity limit of visual working memory, we recently observed a VWM-like capacity limit in an ensemble task where observers extracted the average sweetness of groups of food pictures (i.e., they could only integrate information from four out of six available items), thus suggesting the involvement of VWM in this novel form of cross-modal ensemble coding. Therefore, across two experiments we investigated if this cross-modal ensemble capacity limit could be explained by individual differences in VWM processing. To test this, observers performed both an ensemble task and a VWM task, and we determined 1) how much information they integrated into their ensemble percepts, and 2) how much information they remembered from those displays. Interestingly, we found that individual differences in VWM capacity did not explain differences in performance on the ensemble coding task (i.e., high-capacity individuals did not have significantly higher "ensemble abilities" than low-capacity individuals). While our data cannot definitively state whether or not VWM is necessary to perform the ensemble task, we conclude that it is certainly not sufficient to support this cognitive process. We speculate that the capacity limit may be explained by 1) a bottleneck at the perceptual stage (i.e., a failure to process multiple visual features across multiple items, as there are no singular features that convey taste), or 2) the interaction of multiple cognitive systems (e.g., VWM, gustatory working memory, long term memory). Our results highlight the importance of examining ensemble perception across multiple sensory and cognitive domains to provide a clearer picture of the mechanisms underlying everyday behavior.

The Intensity of Internal and External Attention Assessed with Pupillometry

Not only is visual attention shifted to objects in the external world, attention can also be directed to objects in memory. We have recently shown that pupil size indexes how strongly items are attended externally, which was reflected in more precise encoding into visual working memory. Using a retro-cuing paradigm, we here replicated this finding by showing that stronger pupil constrictions during encoding were reflective of the depth of encoding. Importantly, we extend this previous work by showing that pupil size also revealed the intensity of internal attention toward content stored in visual working memory. Specifically, pupil dilation during the prioritization of one among multiple internally stored representations predicted the precision of the prioritized item. Furthermore, the dynamics of the pupillary responses revealed that the intensity of internal and external attention independently determined the precision of internalized visual representations. Our results show that both internal and external attention are not all-or-none processes, but should rather be thought of as continuous resources that can be deployed at varying intensities. The employed pupillometric approach allows to unravel the intricate interplay between internal and external attention and their effects on visual working memory.

Association of white matter microstructural alteration with non-suicidal self-injury behavior and visual working memory in adolescents with borderline personality disorder

BACKGROUND

Non-suicidal self-injurious behavior (NSSI) is the core characteristic of adolescent borderline personality disorder (BPD) and visual working memory is involved in the pathological processes of BPD. This study aimed to investigate alterations in white matter microstructure and their association with NSSI and visual working memory in adolescents with BPD.

METHODS

53 adolescents diagnosed with BPD and 39 healthy controls (HCs) were enrolled. White matter microstructure was assessed with the fractional anisotropy (FA) and mean diffusivity (MD) from diffusion tensor imaging (DTI). Correlation analysis was performed to assess the association between FA/MD and core features of BPD. A mediation analysis was performed to test whether the effects of white matter alterations on NSSI could be mediated by visual working memory.

RESULTS

Adolescents with BPD showed a reduced FA and an increased MD in the cortical-limbic and cortical-thalamus circuit when compared to the HCs (p < 0.05). Increased MD was positively correlated with NSSI, impulse control and identity disturbance (p < 0.05), and was negatively correlated with the score of visual reproduction. Reserved visual working memory masked the effects of white matter microstructural alterations on NSSI behavior.

CONCLUSIONS

White matter microstructural deficits in the cortical-limbic and cortical-thalamus circuits may be associated with NSSI and visual working memory in adolescents with BPD. Reserved visual working memory may protect against NSSI.

Limitations on flexible allocation of visual short-term memory resources with multiple levels of goal-directed attentional prioritization

Studies suggest that visual short-term memory (VSTM) is a continuous resource that can be flexibly allocated using probabilistic cues that indicate test likelihood (i.e., goal-directed attentional priority to those items). Previous studies using simultaneous cues have not examined this flexible allocation beyond two distinct levels of priority. Moreover, previous studies have not examined whether there are individual differences in the ability to flexibly allocate VSTM resources, as well as whether this ability benefits from practice. The current study used a continuous report procedure to examine whether participants can use up to three levels of attentional priority to allocate VSTM resources via simultaneous probabilistic spatial cues. Three experiments were performed with differing priority levels, cues, and cue presentation times. Group level analysis demonstrated flexible allocation of VSTM resources; however, there was limited evidence that participants could use three goal-directed priority levels. A temporal analysis suggested that task fatigue, rather than practice effects, may interact with item priority. A Bayesian individual-differences analysis revealed that a minority of participants were using three levels of attentional priority, demonstrating that, while possible, it is not the predominant pattern of behaviour. Thus, we provided evidence that flexible allocation to three attention levels is possible under simultaneous cuing conditions for a minority of participants. Flexible allocation to three categories may be interpreted as a skill of high-performing participants akin to high memory capacity.

Learned associations serve as target proxies during difficult but not easy visual search

The target template contains information in memory that is used to guide attention during visual search and is typically thought of as containing features of the actual target object. However, when targets are hard to find, it is advantageous to use other information in the visual environment that is predictive of the target's location to help guide attention. The purpose of these studies was to test if newly learned associations between face and scene category images lead observers to use scene information as a proxy for the face target. Our results showed that scene information was used as a proxy for the target to guide attention but only when the target face was difficult to discriminate from the distractor face; when the faces were easy to distinguish, attention was no longer guided by the scene unless the scene was presented earlier. The results suggest that attention is flexibly guided by both target features as well as features of objects that are predictive of the target location. The degree to which each contributes to guiding attention depends on the efficiency with which that information can be used to decode the location of the target in the current moment. The results contribute to the view that attentional guidance is highly flexible in its use of information to rapidly locate the target.

Sequential encoding aids working memory for meaningful objects' identities but not for their colors

Previous studies have found that real-world objects' identities are better remembered than simple features like colored circles, and this effect is particularly pronounced when these stimuli are encoded one by one in a serial, item-based way. Recent work has also demonstrated that memory for simple features like color is improved if these colors are part of real-world objects, suggesting that meaningful objects can serve as a robust memory scaffold for their associated low-level features. However, it is unclear whether the improved color memory that arises from the colors appearing on real-world objects is affected by encoding format, in particular whether items are encoded sequentially or simultaneously. We test this using randomly colored silhouettes of recognizable versus unrecognizable scrambled objects that offer a uniquely controlled set of stimuli to test color working memory of meaningful versus non-meaningful objects. Participants were presented with four stimuli (silhouettes of objects or scrambled shapes) simultaneously or sequentially. After a short delay, they reported either which colors or which shapes they saw in a two-alternative forced-choice task. We replicated previous findings that meaningful stimuli boost working memory performance for colors (Exp. 1). We found that when participants remembered the colors (Exp. 2) there was no difference in performance across the two encoding formats. However, when participants remembered the shapes and thus identity of the objects (Exp. 3), sequential presentation resulted in better performance than simultaneous presentation. Overall, these results show that different encoding formats can flexibly impact visual working memory depending on what the memory-relevant feature is.

Early top-down control of internal selection induced by retrospective cues in visual working memory: advantage of peripheral over central cues

Attention can be deployed among external sensory stimuli or internal working memory (WM) representations, and recent primate studies have revealed that these external and internal selections share a common neural basis in the prefrontal cortex (PFC). However, it remains to be elucidated how PFC implements these selections, especially in humans. The present study aimed to further investigate whether PFC responded differentially to the peripheral and central retrospective cues (retro-cues) that induced attention selection among WM representations. To achieve this, we combined magnetoencephalography (MEG, Experiment 1) and transcranial magnetic stimulation (TMS, Experiment 2) with an orientation-recall paradigm. Experiment 1 found that a peripheral retro-cue with 100% reliability had a greater benefit on WM performance than a central retro-cue, while this advantage of peripheral over central cues vanished when the cue reliability dropped to 50% (non-informative). MEG source analysis indicated that the 100% peripheral retro-cue elicited earlier (∼125 ms) PFC responses than the central retro-cue (∼275 ms). Meanwhile, Granger causality analysis showed that PFC had earlier (0-200 ms) top-down signals projecting to the superior parietal lobule (SPL) and the lateral occipital cortex (LOC) after the onset of peripheral retro-cues, while these top-down signals appeared later (300-500 ms) after the onset of central retro-cues. Importantly, PFC activity within this period of 300-500 ms correlated with the peripheral advantage in behavior. Moreover, Experiment 2 applied TMS at different time points to test the causal influence of brain activity on behavior and found that stimulating PFC at 100 ms abolished the behavioral benefit of the peripheral retro-cue, as well as its advantage over the central retro-cue. Taken together, our results suggested that the advantage of peripheral over central retro-cues in the mnemonic domain is realized through faster top-down control from PFC, which challenged traditional opinions that the top-down control of attention on WM required at least 300 ms to appear. The present study highlighted that in addition to the causal role of PFC in attention selection of WM representations, timing was critical as well and faster was better.

Similarity-based clustering of multifeature objects in visual working memory

This study investigated the similarity-based clustering mechanism of multifeature stimuli, wherein items are separated or grouped based on their similarity in visual working memory (VWM). In particular, we investigated whether clustering occurred at an individual feature level or at an integrated object level when participants encoded objects with multiple features for VWM. To test this, we conducted two experiments in which participants remembered and reconstructed a randomly chosen feature (either color or orientation) from one of five presented stimuli. As a key manipulation, we kept the distributions of the two feature dimensions constant while controlling the conjunction between the two dimensions in two different conditions: congruent conjunction (CC) and incongruent conjunction (IC). With this manipulation, we expected to observe the same number of clusters regardless of the conjunction condition when clustering occurred at the feature level. However, we expected a different number of clusters for CC and IC conditions when clustering occurred at the object level. Across two experiments, we consistently observed evidence that favored feature-level clustering. Nevertheless, we found that the swap error rates increased in the IC condition only when two features had to be encoded in VWM. These results suggest that clustering occurs at the feature level in VWM and that feature-level clustering influences item-level feature binding. Therefore, our study demonstrated the flexibility of representational units in VWM.

VWM-based bias in conscious access can be extended to a new sandwich masking task and real-life stimuli

Previous studies using breaking continuous flash suppression observed that the content of visual working memory (VWM) influences the priority for accessing visual awareness. However, most studies have used simple stimuli, whereas real-life objects are typically more meaningful and contain more perceptual information than simple objects. In this study, we intermixed a delayed match-to-sample task to manipulate the content of VWM, and a breaking repeated masking suppression (b-RMS) task to investigate whether this memory-based effect in conscious access can be extended to a new sandwich masking task and real-life stimuli. The results revealed that memory-congruent objects broke RMS faster than incongruent objects for both simple and real-life objects. Specifically, for simple objects, color-matching targets broke RMS faster than color-mismatching targets, whereas state-matching targets broke RMS faster than state-mismatching targets for real-life objects. These results suggest that the faster detection of VWM-matching over VWM mismatching stimuli-which has been mostly studied using only one type of task (b-CFS) and stimulus (colored shapes)-extends to (1) yet another masking technique (b-RMS) and to (2) a novel stimulus type (real-life objects), providing evidence that memory-based biases in conscious access are a ubiquitous phenomenon.

Serial dependence in facial identity perception and visual working memory

Serial dependence (SD) refers to the effect in which a person's current perceptual judgment is attracted toward recent stimulus history. Perceptual and memory processes, as well as response and decisional biases, are thought to contribute to SD effects. The current study examined the processing stages of SD facial identity effects in the context of task-related decision processes and how such effects may differ from visual working memory (VWM) interactions. In two experiments, participants were shown a series of two sequentially presented face images. In Experiment 1, the two faces were separated by an interstimulus interval (ISI) of 1, 3, 6, or 10 s, and participants were instructed to reproduce the second face after a varying response delay of 0, 1, 3, 6, or 10 s. Results showed that SD effects occurred most consistently at ISI of 1 s and response delays of 1 and 6 s consistent with early and late stages of processing. In Experiment 2, the ISI was held constant at 1 s, and to separate SD from VWM interactions participants were post-cued to reproduce either the first or the second face. When the second face was the target, SD effects again occurred at response delays of 1 and 6 s, but not when the first face was the target. Together, the results demonstrates that SD facial identity effects occur independently of task-related processes in a distinct temporal fashion and suggest that SD and VWM interactions may rely on separate underlying mechanisms.

Assessing the interaction between working memory and perception through time

Content maintained in visual working memory changes concurrent visual processing, suggesting that visual working memory may recruit an overlapping neural representation with visual perception. However, it remains unclear whether visual working memory representations persist as a sensory code through time, or are recoded later into an abstract code. Here, we directly contrasted a temporal decay + visual code account and a temporal decay + abstract code account within the temporal dynamics of the interaction between working memory and perception. By manipulating the ISI (inter-stimulus interval) between working memory encoding and a perceptual discrimination task, we found that task-relevant and therefore actively maintained perceptual information parametrically altered participants' ability to discriminate perceptual stimuli even 4 s after encoding, whereas task-irrelevant information caused only an acutely transient effect. While continuously present, the size of this shift in discrimination thresholds gradually decreased over time. Concomitantly, the size of the bias in working memory reports increased over time. The opposing directions of threshold and bias effects are consistent with the local maintenance of information in perceptual areas, explained by a temporal decay + visual code account. As the maintained representation decays over time, its ability to alter incoming perceptual signals decreases (reduced threshold effects) while its likelihood of being impacted by those same signals increases (increased bias effects). Altogether, these results suggest that the readout of working memory relies on a sensory representation at a cost of increased interference by ongoing perception.

A spatially continuous diffusion model of visual working memory

We present results from five visual working memory (VWM) experiments in which participants were briefly shown between 2 and 6 colored squares. They were then cued to recall the color of one of the squares and they responded by choosing the color on a continuous color wheel. The experiments provided response proportions and response time (RT) measures as a function of angle for the choices. Current VWM models for this task include discrete models that assume an item is either within working memory or not and resource models that assume that memory strength varies as a function of the number of items. Because these models do not include processes that allow them to account for RT data, we implemented them within the spatially continuous diffusion model (SCDM, Ratcliff, 2018) and use the experimental data to evaluate these combined models. In the SCDM, evidence retrieved from memory is represented as a spatially continuous normal distribution and this drives the decision process until a criterion (represented as a 1-D line) is reached, which produces a decision. Noise in the accumulation process is represented by continuous Gaussian process noise over spatial position. The models that fit best from the discrete and resource-based classes converged on a common model that had a guessing component and that allowed the height of the normal memory-strength distribution to vary with number of items. The guessing component was implemented as a regular decision process driven by a flat evidence distribution, a zero-drift process. The combination of choice and RT data allows models that were not identifiable based on choice data alone to be discriminated.

Tracing the emergence of the memorability benefit

Some visual stimuli are consistently better remembered than others across individuals, due to variations in memorability (the stimulus-intrinsic property that determines ease of encoding into visual long-term memory (VLTM)). However, it remains unclear what cognitive processes give rise to this mnemonic benefit. One possibility is that this benefit is imbued within the capacity-limited bottleneck of VLTM encoding, namely visual working memory (VWM). More precisely, memorable stimuli may be preferentially encoded into VLTM because fewer cognitive resources are required to store them in VWM (efficiency hypothesis). Alternatively, memorable stimuli may be more competitive in obtaining cognitive resources than forgettable stimuli, leading to more successful storage in VWM (competitiveness hypothesis). Additionally, the memorability benefit might emerge post-VWM, specifically, if memorable stimuli are less prone to be forgotten (i.e., are "stickier") than forgettable stimuli after they pass through the encoding bottleneck (stickiness hypothesis). To test this, we conducted two experiments to examine how memorability benefits emerge by manipulating the stimulus memorability, set size, and degree of competition among stimuli as participants encoded them in the context of a working memory task. Subsequently, their memory for the encoded stimuli was tested in a VLTM task. In the VWM task, performance was better for memorable stimuli compared to forgettable stimuli, supporting the efficiency hypothesis. In addition, we found that when in direct competition, memorable stimuli were also better at attracting limited VWM resources than forgettable stimuli, supporting the competitiveness hypothesis. However, only the efficiency advantage translated to a performance benefit in VLTM. Lastly, we found that memorable stimuli were less likely to be forgotten after they passed through the encoding bottleneck imposed by VWM, supporting the "stickiness" hypothesis. Thus, our results demonstrate that the memorability benefit develops across multiple cognitive processes.

Familiarity enhances mnemonic precision but impairs mnemonic accuracy in visual working memory

Prior stimulus familiarity has a variety of effects on visual working memory representations and processes. However, it is still unclear how familiarity interacts with the veridical correspondence between mnemonic representation and external stimuli. Here, we examined the effect of familiarity on two aspects of mnemonic correspondence, precision and accuracy, in visual working memory. Specifically, we used a hierarchical Bayesian method to model task performance in a change detection task with celebrity lookalikes (morphed faces between celebrities and noncelebrities with various ratios) as the memory stimuli. We found that familiarity improves memory precision by sharpening mnemonic representation but impairs memory accuracy by biasing mnemonic representation toward familiar faces (i.e., celebrity faces). These findings provide an integrated account of the puzzling celebrity sighting phenomena with the dissociable effects on mnemonic imprecision and bias and further highlight the importance of assessing these two aspects of memory correspondence in future research.

Change localization: A highly reliable and sensitive measure of capacity in visual working memory

The change detection paradigm has been a widely used approach for measuring capacity in visual working memory (WM). In this task, subjects see an array of visual items, followed by a short blank delay and a single test item. Their task is to indicate whether that test item changed relative to the item in the sample array. This task provides reliable measurements of WM capacity that exhibit robust correlations with many outcome variables of interest. Here, we offer a new variant of this task that we call "change localization." This task is closely modeled after the change detection task described above, except that the test array contains the same number of items as the sample array, and one item has always changed in each trial. The subject's task is to select the changed item in the test array. Using both color and shape stimuli, scores in the change localization task were highly correlated with those in the change detection task, suggesting that change localization taps into the same variance in WM ability. Moreover, the change localization task was far more reliable than change detection, such that only half the number of trials were required to achieve robust reliability. To further validate the approach, we replicated known effects from the literature, demonstrating that they could be detected with far fewer trials than with change detection. Thus, change localization provides a highly reliable and sensitive approach for measuring visual working memory capacity.

Control of bottom-up attention in scene cognition contributes to visual working memory performance

Several studies have investigated the relationship between working memory and attention. However, most of the relevant studies so far investigated top-down attention; only a few have examined possible interactions between bottom-up attention and visual working memory. In the present study, we focused on the visual saliency of different parts of pictures as an index of the degree to which one's bottom-up attention can be drawn towards each of them. We administered the Picture Span Test (PST) to investigate whether salient parts of pictures can influence the performance of visual working memory. The task required participants to judge the semantic congruency of objects in pictures and remember specific parts of pictures. In Experiment 1, we calculated a saliency map for the PST stimuli and found that salient but task-irrelevant parts of pictures could evoke intrusion errors. In Experiment 2, we demonstrated that longer gazing time at target areas results in a higher probability of making correct recognition. In addition, frequent gaze fixation and high normalized scan-path saliency values in task-irrelevant areas were associated with intrusion errors. These results suggest that visual information processed by bottom-up attention may affect working memory.

There Is no Theory-Free Measure of "Swaps" in Visual Working Memory Experiments

Visual working memory is highly limited, and its capacity is tied to many indices of cognitive function. For this reason, there is much interest in understanding its architecture and the sources of its limited capacity. As part of this research effort, researchers often attempt to decompose visual working memory errors into different kinds of errors, with different origins. One of the most common kinds of memory error is referred to as a "swap," where people report a value that closely resembles an item that was not probed (e.g., an incorrect, non-target item). This is typically assumed to reflect confusions, like location binding errors, which result in the wrong item being reported. Capturing swap rates reliably and validly is of great importance because it permits researchers to accurately decompose different sources of memory errors and elucidate the processes that give rise to them. Here, we ask whether different visual working memory models yield robust and consistent estimates of swap rates. This is a major gap in the literature because in both empirical and modeling work, researchers measure swaps without motivating their choice of swap model. Therefore, we use extensive parameter recovery simulations with three mainstream swap models to demonstrate how the choice of measurement model can result in very large differences in estimated swap rates. We find that these choices can have major implications for how swap rates are estimated to change across conditions. In particular, each of the three models we consider can lead to differential quantitative and qualitative interpretations of the data. Our work serves as a cautionary note to researchers as well as a guide for model-based measurement of visual working memory processes.

Mountains of memory in a sea of uncertainty: Sampling the external world despite useful information in visual working memory

A large part of research on visual working memory (VWM) has traditionally focused on estimating its maximum capacity. Yet, humans rarely need to load up their VWM maximally during natural behavior, since visual information often remains accessible in the external world. Recent work, using paradigms that take into account the accessibility of information in the outside world, has indeed shown that observers utilize only one or two items in VWM before sampling from the external world again. One straightforward interpretation of this finding is that, in daily behavior, much fewer items are memorized than the typically reported capacity limits. Here, we first investigate whether this lower reliance on VWM when information is externally accessible might instead reflect resampling before VWM is actually depleted. To this aim we devised an online task, in which participants copied a model (six items in a 4x4 grid; always accessible) in an adjacent empty 4x4 grid. A key aspect of our paradigm is that we (unpredictably) interrupted participants just before inspection of the model with a 2-alternative-forced-choice (2-AFC) question, probing their VWM content. Critically, we observed above-chance performance on probes appearing just before model inspection. This finding shows that the external world was resampled, despite VWM still containing relevant information. We then asked whether increasing the cost of sampling causes participants to load up more information in VWM or, alternatively, to squeeze out more information from VWM (at the cost of making more errors). To manipulate the cost of resampling, we made it more difficult (specifically, more time-consuming) to access the model. We show that with increased cost of accessing the model (which lead to fewer, but longer model inspections), participants could place more items correctly immediately after sampling, and they kept attempting to place items for longer after their first error. These findings demonstrate that participants both encoded more information in VWM and made attempts to squeeze out more information from VWM when sampling became more costly. We argue that human observers constantly evaluate how certain they are of their VWM contents, and only use that VWM content of which their certainty exceeds a context-dependent "action threshold". This threshold, in turn, depends on the trade-off between the cost of resampling and the benefits of making an action. We argue that considering the interplay between the available VWM contents and a context-dependent action threshold, is key for reconciling the traditional VWM literature with VWM use in our day-to-day behavior.

Contralateral delay activity, but not alpha lateralization, indexes prioritization of information for working memory storage

Working memory is inherently limited, which makes it important to select and maintain only task-relevant information and to protect it from distraction. Previous research has suggested the contralateral delay activity (CDA) and lateralized alpha oscillations as neural candidates for such a prioritization process. While most of this work focused on distraction during encoding, we examined the effect of external distraction presented during memory maintenance. Participants memorized the orientations of three lateralized objects. After an initial distraction-free maintenance interval, distractors appeared in the same location as the targets or in the opposite hemifield. This distraction was followed by another distraction-free interval. Our results show that CDA amplitudes were stronger in the interval before compared with the interval after the distraction (i.e., CDA amplitudes were stronger in response to targets compared with distractors). This amplitude reduction in response to distractors was more pronounced in participants with higher memory accuracy, indicating prioritization and maintenance of relevant over irrelevant information. In contrast, alpha lateralization did not change from the interval before distraction compared with the interval after distraction, and we found no correlation between alpha lateralization and memory accuracy. These results suggest that alpha lateralization plays no direct role in either selective maintenance of task-relevant information or inhibition of distractors. Instead, alpha lateralization reflects the current allocation of spatial attention to the most salient information regardless of task-relevance. In contrast, CDA indicates flexible allocation of working memory resources depending on task-relevance.

Difficulty limits of visual mental imagery

While past work has focused on the representational format of mental imagery, and the similarities of its operation and neural substrate to online perception, surprisingly little has tested the boundaries of the level of detail that mental imagery can generate. To answer this question, we take inspiration from the visual short-term memory literature, a related field which has found that memory capacity is affected by the number of items, whether they are unique, and whether and how they move. We test these factors of set size, color heterogeneity, and transformation in mental imagery through both subjective (Exp 1; Exp 2) and objective (Exp 2) measures - difficulty ratings and a change detection task, respectively - to determine the capacity limits of our mental imagery, and find that limits on mental imagery are similar to those for visual short-term memory. In Experiment 1, participants rated the difficulty of imagining 1-4 colored items as subjectively more difficult when there were more items, when the items had unique colors instead of an identical color, and when they scaled or rotated instead of merely linearly translating. Experiment 2 isolated these subjective difficulty ratings of rotation for uniquely colored items, and added a rotation distance manipulation (10° to 110°), again finding higher subjective difficulty for more items, and for when those items rotated farther; the objective measure showed a decrease in performance for more items, but not for rotational degree. Congruities between the subjective and objective results suggest similar costs, but some incongruities suggest that subjective reports can be overly optimistic, likely because they are biased by an illusion of detail.

Crows protect visual working memory against interference

Working memory, the ability to actively maintain and manipulate information across time, is key to intelligent behavior. Because of the limited capacity of working memory, relevant information needs to be protected against distracting representations. Whether birds can resist distractors and safeguard memorized relevant information is unclear. We trained carrion crows in a delayed match-to-sample task to memorize an image while resisting other, interfering stimuli. We found that the repetition of the sample stimulus during the memory delay improved performance accuracy and accelerated reaction time relative to a reference condition with a neutral interfering stimulus. In contrast, the presentation of the image that constituted the subsequent non-match test stimulus mildly weakened performance. However, the crows' robust performance in this most demanding distractor condition indicates that sample information was actively protected from being overwritten by the distractor. These data show that crows can cognitively control and safeguard behaviorally relevant working memory contents.

Behavioral and electrophysiological evidence for the flexible recruitment of feature- and object-based processing in visual working memory comparison

Previous research is inconclusive on when visual working memory (VWM) can be object-based or feature-based. Prior event-related potential (ERP) studies using change detection tasks have found that amplitudes of the N200-an ERP index of VWM comparison- are sensitive to changes in both relevant and irrelevant features, suggesting a bias toward object-based processing. To test whether VWM comparison processing can operate in a feature-based manner, we aimed to create circumstances that would support feature-based processing by: 1) using a strong task-relevance manipulation, and 2) repeating features within a display. Participants completed two blocks of a change detection task for four-item displays in which they were told to respond to color changes (task relevant) but not shape changes (task irrelevant). The first block contained only task-relevant changes to create a strong task-relevance manipulation. In the second block, both relevant and irrelevant changes were present. In both blocks, half of the arrays contained within-display feature repetitions (e.g. two items of the same color or shape). We found that during the second block, N200 amplitudes were sensitive to task-relevant but not irrelevant features regardless of repetition status, consistent with feature-based processing. However, analyses of behavioral data and N200 latencies suggested that object-based processing was occurring at some stages of VWM processing on task-irrelevant feature change trials. In particular, task-irrelevant changes may be processed after no task-relevant feature change is revealed. Overall, the results from the current study suggest that the VWM processing is flexible and can be either object- or feature-based.

Spatial and non-spatial feature binding impairments in visual working memory in schizophrenia

Working memory (WM) impairments are well recognized in schizophrenia patients (PSZ) and contribute to poor psycho-social outcomes in this population. Distinct neural networks underlay the ability to encode and recall visual and spatial information raising the possibility that profile of visual working memory performance may help pinpoint dysfunctional neural correlates in schizophrenia. This study assessed the resolution and associative aspects of visual working memory deficits in schizophrenia and whether these deficits arise during encoding or maintenance processes. A total of 60 participants (30 PSZ and 30 healthy controls) matched in age, gender and education assessed on a modified object in place (OiPT), a delayed non-match-to-sample (DNMST) and a delayed spatial estimation (DSET) task. Patients demonstrated lower accuracy than controls in binding visual features of the same object and recognizing novel objects as well as lower precision recalling the location of a memorized target. Moreover, response choice set size affected recognition accuracy more in PSZ than controls. However, delay duration affected spatial recall precisions, binding, and recognition accuracy equally in the two groups. Our results suggest that visual working memory (vWM) impairments in schizophrenia predominantly reflect spatial and non-spatial binding deficits, with largely preserved discrete feature information. Moreover, these impairments likely arise more during encoding than during maintenance. These binding deficits may reflect impaired effective neural functional connectivity observed in schizophrenia.

Visual working memory phenomena based on categorical tasks replicate using a continuous measure: A simple interpretation and some methodological considerations

An increasingly popular method for investigating visuospatial working memory assumes stored features of objects such as color and orientation vary along continua subject to internal noise. It adapts the stimulus adjustment procedure from perceptual psychophysics to assess the precision with which stored features are represented in memory. This contrasts with methods using discrete, categorical measures of feature retention. The current study examined the replicability of some phenomena documented using conventional methodology when assessed using a continuous measure of feature recall. These concern memory for a short series of objects and include effects of recency, prioritizing an individual object, and presenting an irrelevant additional object after the last item (a poststimulus 'suffix'). In two experiments we find broadly similar results using a continuous measure of color-orientation binding to those obtained previously using categorical measures, with small differences we regard as minor. We interpret the convergence between methods in terms of a simple analogy between categorical memory and categorical perception whereby categorical retrieval involves the application of a discrete criterion to an underlying continuum of stored feature information. We conclude by discussing some of the advantages and limitations of continuous and categorical measures of retention.

Retrospective cue benefits in visual working memory are limited to a single location at a time

Working memory (WM) performance can be improved by an informative cue presented during storage. This effect, termed a retrocue benefit, can be used to study limits on how human observers prioritize information stored in WM for behavioral output. There is disagreement about whether retrocue benefits extend to multiple WM locations. Here, we hypothesized that multiple retrocues may improve some aspects of memory performance (e.g., a reduction in random guessing) while worsening others (e.g., an increase in the probability of reporting a feature presented at a non-probed location). We tested this possibility in three experiments. Participants remembered arrays of four orientations or colors over a brief delay, and spatial retrocues instructed participants to prioritize zero, one, two, or all four remembered orientations for possible report. At the end of the trial, participants recalled the orientation that appeared at one location. The results of this study revealed that participants' recall errors were lower during cue-one relative to cue-two and cue-four trials, and this benefit was driven primarily by a reduction in random guessing during cue-one trials. We found no evidence suggesting that multiple spatial cues (i.e., during cue-two trials) induced a trade-off between memory precision, random guessing, and non-target reports compared to neutral trials (i.e., cue-zero or cue-four). Thus, cuing participants to prioritize information appearing at multiple unique spatial positions led to no improvement in memory performance compared to neutral or no-cue trials, providing additional support for the view that retrocue benefits on WM performance are limited to a single spatial location at a time.

Where and when matter in visual recognition

Our perceptual system processes only a selected subset of an incoming stream of stimuli due to sensory biases and limitations in spatial and temporal attention and working memory capacity. In this study, we investigated perceptual access to sensory information that was temporally predictable or unpredictable and spread across the visual field. In a visual recognition task, participants were presented with an array of different number of alphabetical stimuli that were followed by a probe with a delay. They had to indicate whether the probe was included in the stimulus-set or not. To test the impact of temporal attention, coloured cues that were displayed before the visual stimuli indicated the presentation onset of the stimulus-set. We found that temporal predictability of stimulus onset yields higher performance. In addition, recognition performance was biased across the visual field with higher performance for stimuli that were presented on the upper and right visual quadrants. Our findings demonstrate that recognition accuracy is enhanced by temporal cues and has an inherently asymmetric shape across the visual field.

Sequential encoding paradigm reliably captures the individual differences from a simultaneous visual working memory task

Converging behavioral and neural evidence have suggested that visual stimuli could be attached to existing visual working memory sequentially in time. However, whether individual differences in sequential visual working memory paradigm are similar to those measured by the classical simultaneous change detection paradigm remain unknown. Here, we first show that sequentially presented visual stimuli exhibit similar working memory capacity bottlenecks as previous research using simultaneously presented items. We further reveal that within the same subject, the accuracy and capacity estimates using sequential and simultaneous paradigm were comparable across four different set sizes. Also, we discover that the individual differences measured by the sequential paradigm were highly correlated to those by the simultaneous paradigm within the same subject across all four set sizes of interest. Finally, we find that in a larger sample of subjects (n = 200), the variance and higher-order statistics were similar for sequential and simultaneous paradigms with set size of 6. Collectively, these findings indicate that individual differences measured by the sequential presentation of visual items rely on the similar working memory resources as those by the simultaneous form of presentation.

Visual-cortical enhancement by acoustic distractors: The effects of endogenous spatial attention and visual working memory load

Past work has shown that when a peripheral sound captures our attention, it activates the contralateral visual cortex as revealed by an event-related potential component labelled the auditory-evoked contralateral occipital positivity (ACOP). This cross-modal activation of the visual cortex has been observed even when the sounds were not relevant to the ongoing task (visual or auditory), suggesting that peripheral sounds automatically activate the visual cortex. However, it is unclear whether top-down factors such as visual working memory (VWM) load and endogenous attention, which modulate the impact of task-irrelevant information, may modulate this spatially-specific component. Here, we asked participants to perform a lateralized VWM task (change detection), whose performance is supported by both endogenous spatial attention and VWM storage. A peripheral sound that was unrelated to the ongoing task was delivered during the retention interval. The amplitude of sound-elicited ACOP was analyzed as a function of the spatial correspondence with the cued hemifield, and of the memory array set-size. The typical ACOP modulation was observed over parieto-occipital sites in the 280-500 ms time window after sound onset. Its amplitude was not affected by VWM load but was modulated when the location of the sound did not correspond to the hemifield (right or left) that was cued for the change detection task. Our results suggest that sound-elicited activation of visual cortices, as reflected in the ACOP modulation, is unaffected by visual working memory load. However, endogenous spatial attention affects the ACOP, challenging the hypothesis that it reflects an automatic process.

Context consistency improves ensemble perception of facial expressions

Humans have developed the capacity to rapidly extract summary statistics from the facial expressions of a crowd, such as computing the average facial expression. Although dual-task paradigms involving memory and ensemble tasks have recently found that this ensemble coding ability is biased by visual working memory, few studies have examined whether the context-dependent nature of memory itself can influence the perceptual averaging process. In two experiments, participants made forced-choice judgments about mean facial expressions that were paired with task-irrelevant background images, and the background images either matched or mismatched across encoding and response phases. When the backgrounds matched, it was at either the perceptual level (uniformly oriented lines with the same orientation in encoding and response phases, in Experiment 1), or at the summary statistics level (uniformly oriented lines in the response phase that had the same orientation as the mean of randomly oriented lines that were seen in the encoding phase, in Experiment 2). Participants in Experiment 1 showed a higher ensemble precision and better discrimination sensitivity when the backgrounds matched than when they mismatched, which is consistent with the kind of robust contextual memory effect that has been seen in prior research. We further demonstrated that the context-matching facilitation effect occurred at both the perceptual level (Experiment 1) and at the summary statistics level (Experiment 2). These results demonstrate that the effects of visual working memory on perceptual averaging are obligatory, and they highlight the importance of memory-related context dependency in perceptual averaging.

Perceptual comparisons modulate memory biases induced by new visual inputs

It is well-established that stimulus-specific information in visual working memory (VWM) can be systematically biased by new perceptual inputs. These memory biases are commonly attributed to interference that arises when perceptual inputs are physically similar to VWM contents. However, recent work has suggested that explicitly comparing the similarity between VWM contents and new perceptual inputs modulates the size of memory biases above and beyond stimulus-driven effects. Here, we sought to directly investigate this modulation hypothesis by comparing the size of memory biases following explicit comparisons to those induced when new perceptual inputs are ignored (Experiment 1) or maintained in VWM alongside target information (Experiment 2). We found that VWM reports showed larger attraction biases following explicit perceptual comparisons than when new perceptual inputs were ignored or maintained in VWM. An analysis of participants' perceptual comparisons revealed that memory biases were amplified after perceptual inputs were endorsed as similar-but not dissimilar-to one's VWM representation. These patterns were found to persist even after accounting for variability in the physical similarity between the target and perceptual stimuli across trials, as well as the baseline memory precision between the distinct task demands. Together, these findings illustrate a causal role of perceptual comparisons in modulating naturally-occurring memory biases.

Interference between items stored for distinct tasks in visual working memory

The action perspective on working memory suggests that memory representations are coded according to their specific temporal and behavioral task demands. This stands in contrast to theories that assume representations are stored in a task-agnostic format within a "common workspace". Here, we tested whether visual items that are memorized for different tasks are stored separately from one another or show evidence of inter-item interference during concurrent maintenance, indicating a common storage. In two experiments, we combined a framing memory task (memorize a motion direction for continuous direction report) with an embedded memory task (memorize a motion direction for a binary direction discrimination) that was placed within the retention period of the framing task. Even though the temporal and action demands were item specific, we observed two types of interference effects between the items: The embedded motion direction was (1) repulsed away and (2) degraded in precision by the motion direction of the item in the framing task. Repulsion and precision degradation increased with item similarity when both items were concurrently held in working memory. In contrast, perceptual and iconic memory control conditions revealed weaker repulsion overall and no interference effect on precision during the stimulus processing stages prior to working memory consolidation. Thus, additional inter-item interference arose uniquely within working memory. Together, our results present evidence that items that are stored for distinct tasks to be performed at distinct points in time, reside in a common workspace in working memory.

The role of motion in visual working memory for dynamic stimuli: More lagged but more precise representations of moving objects

While most visual working memory studies use static stimuli with unchanging features, objects in the real world are often dynamic, introducing significant differences in the surface feature information hitting the retina from the same object over time (e.g., changes in orientation, lighting, shadows). Previous research on dynamic stimuli has shown that change detection is improved if objects obey rules of physical motion, but it is unclear how memory for visual features interacts with object motion. In the current study, we investigated whether object motion facilitates greater temporal integration of continuously changing surface feature information. In a series of experiments, participants were asked to report the final color of continuously changing colored dots that were either moving or stationary on the screen. We found that the reported colors "lagged behind" the physical states of the dots when they were in motion. We also observed that the precision of memory responses was significantly higher for stimuli in the moving condition compared to the stationary condition. Together, these findings suggest that memory representation is improved - but lagged - for moving objects, consistent with the idea that object motion may facilitate integration of object information over longer intervals.

Perceptual bias contextualized in visually ambiguous stimuli

The visual appearance of an object is a function of stimulus properties as well as perceptual biases imposed by the observer. The context-specific trade-off between both can be measured accurately in a perceptual judgment task, involving grouping by proximity in ambiguous dot lattices. Such grouping depends lawfully on a stimulus parameter of the dot lattices known as their aspect ratio (AR), whose effect is modulated by a perceptual bias representing the preference for a cardinal orientation. In two experiments, we investigated how preceding context can lead to bias modulation, either in a top-down fashion via visual working memory (VWM) or bottom-up via sensory priming. In Experiment 1, we embedded the perceptual judgment task in a change detection paradigm and studied how the factors of VWM load (complexity of the memory array) and content (congruency in orientation to the ensuing dot lattice) affect the prominence of perceptual bias. A robust vertical orientation bias was observed, which was increased by VWM load and modulated by congruent VWM content. In Experiment 2, dot lattices were preceded by oriented primes. Here, primes regardless of orientation elicited a vertical orientation bias in dot lattices compared to a neutral baseline. Taken together, the two experiments demonstrate that top-down context (VWM load and content) effectively controls orientation bias modulation, while bottom-up context (i.e., priming) merely acts as an undifferentiated trigger to perceptual bias. These findings characterize the temporal context sensitivity of Gestalt perception, shed light on the processes responsible for different perceptual outcomes of ambiguous stimuli, and identify some of the mechanisms controlling perceptual bias.

Visual working memory representations bias attention more when they are the target of an action plan

Attention has frequently been regarded as an emergent property of linking sensory representations to action plans. It has recently been proposed that similar mechanisms may operate within visual working memory (VWM), such that linking an object in VWM to an action plan strengthens its sensory memory representation, which then expresses as an attentional bias. Here we directly tested this hypothesis by comparing attentional biases induced by VWM representations which were the target of a future action, to those induced by VWM representations that were equally task-relevant, but not the direct target of action. We predicted that the first condition would result in a more prioritized memory state and hence stronger attentional biases. Specifically, participants memorized a geometric shape for a subsequent memory test. At test, in case of a match, participants either had to perform a grip movement on the matching object (action condition), or perform the same movement, but on an unrelated object (control condition). To assess any attentional biases, during the delay period between memorandum and test, participants performed a visual selection task in which either the target was surrounded by the memorized shape (congruent trials) or a distractor (incongruent trials). Eye movements were measured as a proxy for attentional priority. We found a significant interaction for saccade latencies between action condition and shape congruency, reflecting more pronounced VWM-based attentional biases in the action condition. Our results are consistent with the idea that action plans prioritize sensory representations in VWM.

The interaction of internal and external attention

The internal/external framework of attention characterizes attention focused to perceptual stimuli and internal representations as highly similar processes. While much research on external attention examines how attention may be broadened or narrowed (attentional zoom lens), it is unclear if internal attention functions in a similar way. In the present study, we manipulate both internal and external attentions to be either broad or narrow. Participants first encoded either a broad or narrow working memory array containing three differently colored items. This array was maintained as they performed an Eriksen flanker task that was either distributed broadly or narrowly, followed by a memory test for a random memory item. We found that regardless of whether the flanker fell inside or outside of the internal breadth of attention, flanker congruency effects did not change. The exception to this was when internal breadth was manipulated with retrocuing, which resulted in greater congruency effects when flankers aligned with the span of internal breadth rather than falling outside of it. Overall, this study shows that internal breadth information is unlikely to alter processing of external distractors until some of the information is cued internally after encoding, suggesting limitations in the internal/external attention framework.

Distinguishing guesses from fuzzy memories: Further evidence for item limits in visual working memory

There is consistent debate over whether capacity in working memory (WM) is subject to an item limit, or whether an unlimited number of items can be held in this online memory system. The item limit hypothesis clearly predicts guessing responses when capacity is exceeded, and proponents of this view have highlighted evidence for guessing in visual working memory tasks. Nevertheless, various models that deny item limits can explain the same empirical patterns by asserting extremely low fidelity representations that cannot be distinguished from guesses. To address this ambiguity, we employed a task for which guess responses elicited a qualitatively distinct pattern from low fidelity memories. Inspired by work from Rouder et al. (2014), we employed an orientation WM task that required subjects to recall the precise orientation of each of six memoranda presented 1 s earlier. The orientation stimuli were created by rotating the position of a "clock hand" inside a circular region that was demarcated by four colored quadrants. Critically, when observers guess with these stimuli, the distribution of responses is biased towards the center of these quadrants, creating a "banded" pattern that cannot be explained by a low precision memory. We confirmed the presence of this guessing pattern using formal model comparisons, and we show that the prevalence of this pattern matches observers' own reports of when they thought they were guessing. Thus, these findings provide further evidence for guessing behaviors predicted by item limit models of WM capacity.

Eye movements dissociate between perceiving, sensing, and unconscious change detection in scenes

Detecting visual changes can be based on perceiving, whereby one can identify a specific detail that has changed, on sensing, whereby one knows that there is a change but is unable to identify what changed, or on unconscious change detection, whereby one is unaware of any change even though the change influences one's behavior. Prior work has indicated that the processes underlying these different types of change detection are functionally and neurally distinct, but the attentional mechanisms that are related to these different types of change detection remain largely unknown. In the current experiment, we examined eye movements during a change detection task in globally manipulated scenes, and participants indicated their change detection confidence on a scale that allowed us to isolate perceiving, sensing, and unconscious change detection. For perceiving-based change detection, but not sensing-based or unconscious change detection, participants were more likely to preferentially revisit highly changed scene regions across the first and second presentation of the scene (i.e., resampling). This increase in resampling started within 250 ms of the test scene onset, suggesting that the effect began within the first two fixations. In addition, changed scenes were related to more clustered (i.e., less dispersed) eye movements than unchanged scenes, particularly when the subjects were highly confident that no change had occurred - providing evidence for change detection outside of conscious awareness. The results indicate that perceiving, sensing, and unconscious change detection responses are related to partially distinct patterns of eye movements.

Working memory is updated by reallocation of resources from obsolete to new items

Visual working memory (VWM) resources are limited, placing constraints on how much visual information can be simultaneously retained. During visually guided activity, stored information can quickly become outdated, so updating mechanisms are needed to ensure the contents of memory remain relevant to current task goals. In particular, successful deallocation of resources from items that become obsolete is likely to be critical for maintaining the precision of those representations still in memory. The experiments in this study involved presenting two memory arrays of coloured disks in sequence. The appearance of the second array was a cue to replace, rehearse, or add a new colour to the colours in memory. We predicted that successful resource reallocation should result in comparable recall precision when an item was replaced or rehearsed, owing to the removal of pre-replacement features. In contrast, a failure to update WM should lead to comparable precision with a condition in which a new colour was added to memory. We identified a very small proportion (∼5%) of trials in which participants incorrectly reported a feature from the first array in place of its replacement in the second, which we interpreted as a failure to incorporate the information from the second display into memory. Once these trials were discounted, precision estimates were consistent with complete redistribution of resources in the case of updating a single item. We conclude that working memory can be efficiently updated when previous information becomes obsolete, but that this is a demanding active process that occasionally fails.

Is Categorization in Visual Working Memory a Way to Reduce Mental Effort? A Pupillometry Study

Recent studies on visual working memory (VWM) have shown that visual information can be stored in VWM as continuous (e.g., a specific shade of red) as well as categorical representations (e.g., the general category red). It has been widely assumed, yet never directly tested, that continuous representations require more VWM mental effort than categorical representations; given limited VWM capacity, this would mean that fewer continuous, as compared to categorical, representations can be maintained simultaneously. We tested this assumption by measuring pupil size, as a proxy for mental effort, in a delayed estimation task. Participants memorized one to four ambiguous (boundaries between adjacent color categories) or prototypical colors to encourage continuous or categorical representations, respectively; after a delay, a probe indicated the location of the to‐be‐reported color. We found that, for memory load 1, pupil size was larger while maintaining ambiguous as compared to prototypical colors, but without any difference in memory precision; this suggests that participants relied on an effortful continuous representation to maintain a single ambiguous color, thus resulting in pupil dilation while preserving precision. Strikingly, this effect gradually inverted, such that for memory load 4, pupil size was smaller while maintaining ambiguous and prototypical colors, but memory precision was now substantially reduced for ambiguous colors; this suggests that with increased memory load participants increasingly relied on categorical representations for ambiguous colors (which are by definition a poor fit to any category). Taken together, our results suggest that continuous representations are more effortful than categorical representations and that very few continuous representations (perhaps only one) can be maintained simultaneously.