Drifting codes within a stable coding scheme for working memory

Representation and computation in visual working memory

The ability to sustain internal representations of the sensory environment beyond immediate perception is a fundamental requirement of cognitive processing. In recent years, debates regarding the capacity and fidelity of the working memory (WM) system have advanced our understanding of the nature of these representations. In particular, there is growing recognition that WM representations are not merely imperfect copies of a perceived object or event. New experimental tools have revealed that observers possess richer information about the uncertainty in their memories and take advantage of environmental regularities to use limited memory resources optimally. Meanwhile, computational models of visuospatial WM formulated at different levels of implementation have converged on common principles relating capacity to variability and uncertainty. Here we review recent research on human WM from a computational perspective, including the neural mechanisms that support it.

A Neural Decision Signal during Internal Sampling from Working Memory in Humans

How humans transform sensory information into decisions that steer purposeful behavior is a central question in psychology and neuroscience that is traditionally investigated during the sampling of external environmental signals. The decision-making framework of gradual information sampling toward a decision has also been proposed to apply when sampling internal sensory evidence from working memory. However, neural evidence for this proposal remains scarce. Here we show (using scalp EEG in male and female human volunteers) that sampling internal visual representations from working memory elicits a scalp EEG potential associated with gradual evidence accumulation-the central parietal positivity. Consistent with an evolving decision process, we show how this signal (1) scales with the time participants require to reach a decision about the cued memory content and (2) is amplified when having to decide among multiple contents in working memory. These results bring the electrophysiology of decision-making into the domain of working memory and suggest that variability in memory-guided behavior may be driven (at least in part) by variations in the sampling of our inner mental contents.

A dynamic neural resource model bridges sensory and working memory

Probing memory of a complex visual image within a few hundred milliseconds after its disappearance reveals significantly greater fidelity of recall than if the probe is delayed by as little as a second. Classically interpreted, the former taps into a detailed but rapidly decaying visual sensory or 'iconic' memory (IM), while the latter relies on capacity-limited but comparatively stable visual working memory (VWM). While iconic decay and VWM capacity have been extensively studied independently, currently no single framework quantitatively accounts for the dynamics of memory fidelity over these time scales. Here, we extend a stationary neural population model of VWM with a temporal dimension, incorporating rapid sensory-driven accumulation of activity encoding each visual feature in memory, and a slower accumulation of internal error that causes memorized features to randomly drift over time. Instead of facilitating read-out from an independent sensory store, an early cue benefits recall by lifting the effective limit on VWM signal strength imposed when multiple items compete for representation, allowing memory for the cued item to be supplemented with information from the decaying sensory trace. Empirical measurements of human recall dynamics validate these predictions while excluding alternative model architectures. A key conclusion is that differences in capacity classically thought to distinguish IM and VWM are in fact contingent upon a single resource-limited WM store.

Maintenance of color memoranda in activity-quiescent working memory states: Evidence from impulse perturbation

In the present study, we used an impulse perturbation method to probe working memory maintenance of colors in neurally active and activity-quiescent states, focusing on a set of pre-registered analyses. We analyzed the electroencephalograph (EEG) data of 30 participants who completed a delayed match-to-sample working memory task, in which one of the two items that were presented was retro-cued as task relevant. The analyses revealed that both cued and uncued colors were decodable from impulse-evoked activity, the latter in contrast to previous reports of working memory for orientation gratings. Decoding of colors from oscillations in the alpha band showed that cued items could be decoded therein whereas uncued items could not. Overall, the outcomes suggest that subtle differences exist between the representation of colors, and that of stimuli with spatial properties, but the present results also demonstrate that regardless of their specific neural state, both are accessible through visual impulse perturbation.

Distinct neural signatures underlying information maintenance and manipulation in working memory

Previous working memory research has demonstrated robust stimulus representations during memory maintenance in both voltage and alpha-band activity in electroencephalography. However, the exact functions of these 2 neural signatures have remained controversial. Here we systematically investigated their respective contributions to memory manipulation. Human participants either maintained a previously seen spatial location, or manipulated the location following a mental rotation cue over a delay. Using multivariate decoding, we observed robust location representations in low-frequency voltage and alpha-band oscillatory activity with distinct spatiotemporal dynamics: location representations were most evident in posterior channels in alpha-band activity, but were most prominent in the more anterior, central channels in voltage signals. Moreover, the temporal emergence of manipulated representation in central voltage preceded that in posterior alpha-band activity, suggesting that voltage might carry stimulus-specific source signals originated internally from anterior cortex, whereas alpha-band activity might reflect feedback signals in posterior cortex received from higher-order cortex. Lastly, while location representations in both signals were coded in a low-dimensional neural subspace, location representation in central voltage was higher-dimensional and underwent a representational transformation that exclusively predicted memory behavior. Together, these results highlight the crucial role of central voltage in working memory, and support functional distinctions between voltage and alpha-band activity.

Geometry of visuospatial working memory information in miniature gaze patterns

Stimulus-dependent eye movements have been recognized as a potential confound in decoding visual working memory information from neural signals. Here we combined eye-tracking with representational geometry analyses to uncover the information in miniature gaze patterns while participants (n = 41) were cued to maintain visual object orientations. Although participants were discouraged from breaking fixation by means of real-time feedback, small gaze shifts (<1°) robustly encoded the to-be-maintained stimulus orientation, with evidence for encoding two sequentially presented orientations at the same time. The orientation encoding on stimulus presentation was object-specific, but it changed to a more object-independent format during cued maintenance, particularly when attention had been temporarily withdrawn from the memorandum. Finally, categorical reporting biases increased after unattended storage, with indications of biased gaze geometries already emerging during the maintenance periods before behavioural reporting. These findings disclose a wealth of information in gaze patterns during visuospatial working memory and indicate systematic changes in representational format when memory contents have been unattended.

Electrophysiological hallmarks for event relations and event roles in working memory

The ability to maintain events (i.e., interactions between/among objects) in working memory is crucial for our everyday cognition, yet the format of this representation is poorly understood. The current ERP study was designed to answer two questions: How is maintaining events (e.g., the tiger hit the lion) neurally different from maintaining item coordinations (e.g., the tiger and the lion)? That is, how is the event relation (present in events but not coordinations) represented? And how is the agent, or initiator of the event encoded differently from the patient, or receiver of the event during maintenance? We used a novel picture-sentence match-across-delay approach in which the working memory representation was "pinged" during the delay, replicated across two ERP experiments with Chinese and English materials. We found that maintenance of events elicited a long-lasting late sustained difference in posterior-occipital electrodes relative to non-events. This effect resembled the negative slow wave reported in previous studies of working memory, suggesting that the maintenance of events in working memory may impose a higher cost compared to coordinations. Although we did not observe significant ERP differences associated with pinging the agent vs. the patient during the delay, we did find that the ping appeared to dampen the ongoing sustained difference, suggesting a shift from sustained activity to activity silent mechanisms. These results suggest a new method by which ERPs can be used to elucidate the format of neural representation for events in working memory.

Neural population dynamics of human working memory

The activity of neurons in macaque prefrontal cortex (PFC) persists during working memory (WM) delays, providing a mechanism for memory.1,2,3,4,5,6,7,8,9,10,11 Although theory,11,12 including formal network models,13,14 assumes that WM codes are stable over time, PFC neurons exhibit dynamics inconsistent with these assumptions.15,16,17,18,19 Recently, multivariate reanalyses revealed the coexistence of both stable and dynamic WM codes in macaque PFC.20,21,22,23 Human EEG studies also suggest that WM might contain dynamics.24,25 Nonetheless, how WM dynamics vary across the cortical hierarchy and which factors drive dynamics remain unknown. To elucidate WM dynamics in humans, we decoded WM content from fMRI responses across multiple cortical visual field maps.26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48 We found coexisting stable and dynamic neural representations of WM during a memory-guided saccade task. Geometric analyses of neural subspaces revealed that early visual cortex exhibited stronger dynamics than high-level visual and frontoparietal cortex. Leveraging models of population receptive fields, we visualized and made the neural dynamics interpretable. We found that during WM delays, V1 population initially encoded a narrowly tuned bump of activation centered on the peripheral memory target. Remarkably, this bump then spread inward toward foveal locations, forming a vector along the trajectory of the forthcoming memory-guided saccade. In other words, the neural code transformed into an abstraction of the stimulus more proximal to memory-guided behavior. Therefore, theories of WM must consider both sensory features and their task-relevant abstractions because changes in the format of memoranda naturally drive neural dynamics.

Pinging the brain to reveal the hidden attentional priority map using encephalography

Attention has been usefully thought of as organized in priority maps - putative maps of space where attentional priority is weighted across spatial regions in a winner-take-all competition for attentional deployment. Recent work has highlighted the influence of past experiences on the weighting of spatial priority - called selection history. Aside from being distinct from more well-studied, top-down forms of attentional enhancement, little is known about the neural substrates of history-mediated attentional priority. Using a task known to induce statistical learning of target distributions, in an EEG study we demonstrate that this otherwise invisible, latent attentional priority map can be visualized during the intertrial period using a 'pinging' technique in conjunction with multivariate pattern analyses. Our findings not only offer a method of visualizing the history-mediated attentional priority map, but also shed light on the underlying mechanisms allowing our past experiences to influence future behavior.

Cognitive Dynamics Estimation: A whole-brain spatial regression paradigm for extracting the temporal dynamics of cognitive processes

Functional MRI (fMRI) has been instrumental in understanding how cognitive processes are spatially mapped in the brain, yielding insights about brain regions and functions. However, in case the orthogonality of behavioral or stimulus timing is not guaranteed, the estimated brain maps fail to dissociate each cognitive process, and the resultant maps become unstable. Also, the brain mapping exercise can not provide temporal information on the cognitive process. Here we propose a qualitatively different approach to fMRI analysis, named Cognitive Dynamics Estimation (CDE), that estimates how multiple cognitive processes change over time even when behavior or stimulus logs are unavailable. This method transposes the conventional brain mapping; the brain activity pattern at each time point is subject to regression analysis with data-driven maps of cognitive processes as regressors, resulting in the time series of cognitive processes. The estimated time series captured the fluctuation of intensity and timing of cognitive processes on a trial-by-trial basis, which conventional analysis could not capture. Notably, the estimated time series predicted participants' cognitive ability to perform each psychological task. As an addition to our fMRI analytic toolkit, these results suggest the potential for CDE to elucidate underexplored cognitive phenomena, especially in the temporal domain.

A gradual transition from veridical to categorical representations along the visual hierarchy during working memory, but not perception

The ability to stably maintain visual information over brief delays is central to cognitive functioning. One possible way to achieve robust working memory maintenance is by having multiple concurrent mnemonic representations across multiple cortical loci. For example, early visual cortex might contribute to storage by representing information in a "sensory-like" format, while intraparietal sulcus uses a format transformed away from sensory driven responses. As an explicit test of mnemonic code transformations along the visual hierarchy, we quantitatively modeled the progression of veridical-to-categorical orientation representations in human participants. Participants directly viewed, or held in mind, an oriented grating pattern, and the similarity between fMRI activation patterns for different orientations was calculated throughout retinotopic cortex. During direct perception, similarity was clustered around cardinal orientations, while during working memory the obliques were represented more similarly. We modeled these similarity patterns based on the known distribution of orientation information in the natural world: The "veridical" model uses an efficient coding framework to capture hypothesized representations during visual perception. The "categorical" model assumes that different "psychological distances" between orientations result in orientation categorization relative to cardinal axes. During direct perception, the veridical model explained the data well in early visual areas, while the categorical model did worse. During working memory, the veridical model only explained some of the data, while the categorical model gradually gained explanatory power for increasingly anterior retinotopic regions. These findings suggest that directly viewed images are represented veridically, but once visual information is no longer tethered to the sensory world, there is a gradual progression to more categorical mnemonic formats along the visual hierarchy.

Impulse perturbation reveals cross-modal access to sensory working memory through learned associations

We investigated if learned associations between visual and auditory stimuli can afford full cross-modal access to working memory. Previous research using the impulse perturbation technique has shown that cross-modal access to working memory is one-sided; visual impulses reveal both auditory and visual memoranda, but auditory impulses do not seem to reveal visual memoranda (Wolff et al., 2020b). Our participants first learned to associate six auditory pure tones with six visual orientation gratings. Next, a delayed match-to-sample task for the orientations was completed, while EEG was recorded. Orientation memories were recalled either via their learned auditory counterpart, or were visually presented. We then decoded the orientation memories from the EEG responses to both auditory and visual impulses presented during the memory delay. Working memory content could always be decoded from visual impulses. Importantly, through recall of the learned associations, the auditory impulse also evoked a decodable response from the visual WM network, providing evidence for full cross-modal access. We also observed that after a brief initial dynamic period, the representational codes of the memory items generalized across time, as well as between perceptual maintenance and long-term recall conditions. Our results thus demonstrate that accessing learned associations in long-term memory provides a cross-modal pathway to working memory that seems to be based on a common coding scheme.

Working memory control dynamics follow principles of spatial computing

Working memory (WM) allows us to remember and selectively control a limited set of items. Neural evidence suggests it is achieved by interactions between bursts of beta and gamma oscillations. However, it is not clear how oscillations, reflecting coherent activity of millions of neurons, can selectively control individual WM items. Here we propose the novel concept of spatial computing where beta and gamma interactions cause item-specific activity to flow spatially across the network during a task. This way, control-related information such as item order is stored in the spatial activity independent of the detailed recurrent connectivity supporting the item-specific activity itself. The spatial flow is in turn reflected in low-dimensional activity shared by many neurons. We verify these predictions by analyzing local field potentials and neuronal spiking. We hypothesize that spatial computing can facilitate generalization and zero-shot learning by utilizing spatial component as an additional information encoding dimension.

Contralateral delay activity, but not alpha lateralization, indexes prioritization of information for working memory storage

Working memory is inherently limited, which makes it important to select and maintain only task-relevant information and to protect it from distraction. Previous research has suggested the contralateral delay activity (CDA) and lateralized alpha oscillations as neural candidates for such a prioritization process. While most of this work focused on distraction during encoding, we examined the effect of external distraction presented during memory maintenance. Participants memorized the orientations of three lateralized objects. After an initial distraction-free maintenance interval, distractors appeared in the same location as the targets or in the opposite hemifield. This distraction was followed by another distraction-free interval. Our results show that CDA amplitudes were stronger in the interval before compared with the interval after the distraction (i.e., CDA amplitudes were stronger in response to targets compared with distractors). This amplitude reduction in response to distractors was more pronounced in participants with higher memory accuracy, indicating prioritization and maintenance of relevant over irrelevant information. In contrast, alpha lateralization did not change from the interval before distraction compared with the interval after distraction, and we found no correlation between alpha lateralization and memory accuracy. These results suggest that alpha lateralization plays no direct role in either selective maintenance of task-relevant information or inhibition of distractors. Instead, alpha lateralization reflects the current allocation of spatial attention to the most salient information regardless of task-relevance. In contrast, CDA indicates flexible allocation of working memory resources depending on task-relevance.

Dynamics Are the Only Constant in Working Memory

In this short perspective, we reflect upon our tendency to use oversimplified and idiosyncratic tasks in a quest to discover general mechanisms of working memory. We discuss how the work of Mark Stokes and collaborators has looked beyond localized, temporally persistent neural activity and shifted focus toward the importance of distributed, dynamic neural codes for working memory. A critical lesson from this work is that using simplified tasks does not automatically simplify the neural computations supporting behavior (even if we wish it were so). Moreover, Stokes' insights about multidimensional dynamics highlight the flexibility of the neural codes underlying cognition and have pushed the field to look beyond static measures of working memory.

Role of time in binding features in visual working memory

Previous research on feature binding in visual working memory has supported a privileged role for location in binding an object's nonspatial features. However, humans are able to correctly recall feature conjunctions of objects that occupy the same location at different times. In a series of behavioral experiments, we investigated binding errors under these conditions, and specifically tested whether ordinal position can take the role of location in mediating feature binding. We performed two dual report experiments in which participants had to memorize three colored shapes presented sequentially at the screen center. When participants were cued with the ordinal position of one item and had to report its shape and color, report errors for the two features were largely uncorrelated. In contrast, when participants were cued, for example, with an item's shape and reported an incorrect ordinal position, they had a high chance of making a corresponding error in the color report. This pattern of error correlations closely matched the predictions of a model in which color and shape are bound to each other only indirectly via an item's ordinal position. In a third experiment, we directly compared the roles of location and sequential position in feature binding. Participants viewed a sequence of colored disks displayed at different locations and were cued either by a disk's location or its ordinal position to report its remaining properties. The pattern of errors supported a mixed strategy with individual variation, suggesting that binding via either time or space could be used for this task. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Reactivating ordinal position information from auditory sequence memory in human brains

Retaining a sequence of events in their order is a core ability of many cognitive functions, such as speech recognition, movement control, and episodic memory. Although content representations have been widely studied in working memory (WM), little is known about how ordinal position information of an auditory sequence is retained in the human brain as well as its coding characteristics. In fact, there is still a lack of an efficient approach to directly accessing the stored ordinal position code during WM retention. Here, 31 participants performed an auditory sequence WM task with their brain activities recorded using electroencephalography (EEG). We developed new triggering events that could successfully reactivate neural representations of ordinal position during the delay period. Importantly, the ordinal position reactivation is further related to recognition behavior, confirming its indexing of WM storage. Furthermore, the ordinal position code displays an intriguing “stable-dynamic” format, i.e. undergoing the same dynamic neutral trajectory in the multivariate neural space during both encoding and retention (whenever reactivated). Overall, our results provide an effective approach to accessing the behaviorally-relevant ordinal position information in auditory sequence WM and reveal its new temporal characteristics.

Task-dependent fractal patterns of information processing in working memory

We applied detrended fluctuation analysis, power spectral density, and eigenanalysis of detrended cross-correlations to investigate fMRI data representing a diurnal variation of working memory in four visual tasks: two verbal and two nonverbal. We show that the degree of fractal scaling is regionally dependent on the engagement in cognitive tasks. A particularly apparent difference was found between memorisation in verbal and nonverbal tasks. Furthermore, the detrended cross-correlations between brain areas were predominantly indicative of differences between resting state and other tasks, between memorisation and retrieval, and between verbal and nonverbal tasks. The fractal and spectral analyses presented in our study are consistent with previous research related to visuospatial and verbal information processing, working memory (encoding and retrieval), and executive functions, but they were found to be more sensitive than Pearson correlations and showed the potential to obtain other subtler results. We conclude that regionally dependent cognitive task engagement can be distinguished based on the fractal characteristics of BOLD signals and their detrended cross-correlation structure.

Neural signature of flexible coding in prefrontal cortex

The ability of prefrontal cortex to quickly encode novel associations is crucial for adaptive behavior and central to working memory. Fast Hebbian changes in synaptic strength permit forming new associations, but neuronal signatures of this have been elusive. We devised a trialwise index of pattern similarity to look for rapid changes in population codes. Based on a computational model of working memory, we hypothesized that synaptic strength-and consequently, the tuning of neurons-could change if features of a subsequent stimulus need to be "reassociated," i.e., if bindings between features need to be broken to encode the new item. As a result, identical stimuli might elicit different neural responses. As predicted, neural response similarity dropped following rebinding, but only in prefrontal cortex. The history-dependent changes were expressed on top of traditional, fixed selectivity and were not explainable by carryover of previous firing into the current trial or by neural adaptation.

Neural Reinstatement Tracks Spread of Attention between Object Features in Working Memory

Attention can be allocated in working memory (WM) to select and privilege relevant content. It is unclear whether attention selects individual features or whole objects in WM. Here, we used behavioral measures, eye-tracking, and EEG to test the hypothesis that attention spreads between an object's features in WM. Twenty-six participants completed a WM task that asked them to recall the angle of one of two oriented, colored bars after a delay while EEG and eye-tracking data were collected. During the delay, an orthogonal "incidental task" cued the color of one item for a match/mismatch judgment. On congruent trials (50%), the cued item was probed for subsequent orientation recall; on incongruent trials (50%), the other memory item was probed. As predicted, selecting the color of an object in WM brought other features of the cued object into an attended state as revealed by EEG decoding, oscillatory α-power, gaze bias, and improved orientation recall performance. Together, the results show that attentional selection spreads between an object's features in WM, consistent with object-based attentional selection. Analyses of neural processing at recall revealed that the selected object was automatically compared with the probe, whether it was the target for recall or not. This provides a potential mechanism for the observed benefits of nonpredictive cueing in WM, where a selected item is prioritized for subsequent decision-making.

Cortical feedback loops bind distributed representations of working memory

Working memory—the brain’s ability to internalize information and use it flexibly to guide behaviour—is an essential component of cognition. Although activity related to working memory has been observed in several brain regions^1–3, how neural populations actually represent working memory^4–7 and the mechanisms by which this activity is maintained^8–12 remain unclear^13–15. Here we describe the neural implementation of visual working memory in mice alternating between a delayed non-match-to-sample task and a simple discrimination task that does not require working memory but has identical stimulus, movement and reward statistics. Transient optogenetic inactivations revealed that distributed areas of the neocortex were required selectively for the maintenance of working memory. Population activity in visual area AM and premotor area M2 during the delay period was dominated by orderly low-dimensional dynamics^16,17 that were, however, independent of working memory. Instead, working memory representations were embedded in high-dimensional population activity, present in both cortical areas, persisted throughout the inter-stimulus delay period, and predicted behavioural responses during the working memory task. To test whether the distributed nature of working memory was dependent on reciprocal interactions between cortical regions^18–20, we silenced one cortical area (AM or M2) while recording the feedback it received from the other. Transient inactivation of either area led to the selective disruption of inter-areal communication of working memory. Therefore, reciprocally interconnected cortical areas maintain bound high-dimensional representations of working memory.

Experiments in mice alternating between a visual working memory task and a task that is independent of working memory provide insight into the neural representation of working memory and the distributed nature of its maintenance.

Pinging the brain with visual impulses reveals electrically active, not activity-silent, working memories

Persistently active neurons during mnemonic periods have been regarded as the mechanism underlying working memory maintenance. Alternatively, neuronal networks could instead store memories in fast synaptic changes, thus avoiding the biological cost of maintaining an active code through persistent neuronal firing. Such "activity-silent" codes have been proposed for specific conditions in which memories are maintained in a nonprioritized state, as for unattended but still relevant short-term memories. A hallmark of this "activity-silent" code is that these memories can be reactivated from silent, synaptic traces. Evidence for "activity-silent" working memory storage has come from human electroencephalography (EEG), in particular from the emergence of decodability (EEG reactivations) induced by visual impulses (termed pinging) during otherwise "silent" periods. Here, we reanalyze EEG data from such pinging studies. We find that the originally reported absence of memory decoding reflects weak statistical power, as decoding is possible based on more powered analyses or reanalysis using alpha power instead of raw voltage. This reveals that visual pinging EEG "reactivations" occur in the presence of an electrically active, not silent, code for unattended memories in these data. This crucial change in the evidence provided by this dataset prompts a reinterpretation of the mechanisms of EEG reactivations. We provide 2 possible explanations backed by computational models, and we discuss the relationship with TMS-induced EEG reactivations.

Temporal dynamics of implicit memory underlying serial dependence

Serial dependence is the effect in which the immediately preceding trial influences participants' responses to the current stimulus. But for how long does this bias last in the absence of interference from other stimuli? Here, we had 20 healthy young adult participants (12 women) perform a coincident timing task using different inter-trial intervals to characterize the serial dependence effect as the time between trials increases. Our results show that serial dependence abruptly decreases from 0.1 s to 1 s inter-trial interval, but it remains pronounced after that for up to 8 s. In addition, participants' response variability slightly decreases over longer intervals. We discuss these results in light of recent models suggesting that serial dependence might rely on a short-term memory trace kept through changes in synaptic weights, which might explain its long duration and apparent stability over time.

Persistent Activity During Working Memory From Front to Back

Working memory (WM) extends the duration over which information is available for processing. Given its importance in supporting a wide-array of high level cognitive abilities, uncovering the neural mechanisms that underlie WM has been a primary goal of neuroscience research over the past century. Here, we critically review what we consider the two major "arcs" of inquiry, with a specific focus on findings that were theoretically transformative. For the first arc, we briefly review classic studies that led to the canonical WM theory that cast the prefrontal cortex (PFC) as a central player utilizing persistent activity of neurons as a mechanism for memory storage. We then consider recent challenges to the theory regarding the role of persistent neural activity. The second arc, which evolved over the last decade, stemmed from sophisticated computational neuroimaging approaches enabling researchers to decode the contents of WM from the patterns of neural activity in many parts of the brain including early visual cortex. We summarize key findings from these studies, their implications for WM theory, and finally the challenges these findings pose. Our goal in doing so is to identify barriers to developing a comprehensive theory of WM that will require a unification of these two "arcs" of research.

A Hierarchy of Functional States in Working Memory

Extensive research has examined how information is maintained in working memory (WM), but it remains unknown how WM is used to guide behavior. We addressed this question by combining human electrophysiology (50 subjects, male and female) with pattern analyses, cognitive modeling, and a task requiring the prolonged maintenance of two WM items and priority shifts between them. This enabled us to discern neural states coding for memories that were selected to guide the next decision from states coding for concurrently held memories that were maintained for later use, and to examine how these states contribute to WM-based decisions. Selected memories were encoded in a functionally active state. This state was reflected in spontaneous brain activity during the delay period, closely tracked moment-to-moment fluctuations in the quality of evidence integration, and also predicted when memories would interfere with each other. In contrast, concurrently held memories were encoded in a functionally latent state. This state was reflected only in stimulus-evoked brain activity, tracked memory precision at longer timescales, but did not engage with ongoing decision dynamics. Intriguingly, the two functional states were highly flexible, as priority could be dynamically shifted back and forth between memories without degrading their precision. These results delineate a hierarchy of functional states, whereby latent memories supporting general maintenance are transformed into active decision circuits to guide flexible behavior.SIGNIFICANCE STATEMENT Working memory enables maintenance of information that is no longer available in the environment. Abundant neuroscientific work has examined where in the brain working memories are stored, but it remains unknown how they are represented and used to guide behavior. Our study shows that working memories are represented in qualitatively different formats, depending on behavioral priorities. Memories that are selected for guiding behavior are encoded in an active state that transforms sensory input into decision variables, whereas other concurrently held memories are encoded in a latent state that supports precise maintenance without affecting ongoing cognition. These results dissociate mechanisms supporting memory storage and usage, and open the door to reveal not only where memories are stored but also how.

Decoding visual colour from scalp electroencephalography measurements

Recent advances have made it possible to decode various aspects of visually presented stimuli from patterns of scalp EEG measurements. As of recently, such multivariate methods have been commonly used to decode visual-spatial features such as location, orientation, or spatial frequency. In the current study, we show that it is also possible to track visual colour processing by using Linear Discriminant Analysis on patterns of EEG activity. Building on other recent demonstrations, we show that colour decoding: (1) reflects sensory qualities (as opposed to, for example, verbal labelling) with a prominent contribution from posterior electrodes contralateral to the stimulus, (2) conforms to a parametric coding space, (3) is possible in multi-item displays, and (4) is comparable in magnitude to the decoding of visual stimulus orientation. Through subsampling our data, we also provide an estimate of the approximate number of trials and participants required for robust decoding. Finally, we show that while colour decoding can be sensitive to subtle differences in luminance, our colour decoding results are primarily driven by measured colour differences between stimuli. Colour decoding opens a relevant new dimension in which to track visual processing using scalp EEG measurements, while bypassing potential confounds associated with decoding approaches that focus on spatial features.

Visual and Semantic Contributions to Visual Short-Term Memory

Liu et al. recently demonstrated novel neural evidence for visual and semantic contributions to the encoding and maintenance of object information in a delayed match-to-sample task. Their data highlight the close interaction between sensory experience and prior semantic knowledge in human visual short-term memory for naturalistic stimuli.

Minimally dependent activity subspaces for working memory and motor preparation in the lateral prefrontal cortex

The lateral prefrontal cortex is involved in the integration of multiple types of information, including working memory and motor preparation. However, it is not known how downstream regions can extract one type of information without interference from the others present in the network. Here, we show that the lateral prefrontal cortex of non-human primates contains two minimally dependent low-dimensional subspaces: one that encodes working memory information, and another that encodes motor preparation information. These subspaces capture all the information about the target in the delay periods, and the information in both subspaces is reduced in error trials. A single population of neurons with mixed selectivity forms both subspaces, but the information is kept largely independent from each other. A bump attractor model with divisive normalization replicates the properties of the neural data. These results provide new insights into neural processing in prefrontal regions.

Tracking stimulus representation across a 2-back visual working memory task

How does the neural representation of visual working memory content vary with behavioural priority? To address this, we recorded electroencephalography (EEG) while subjects performed a continuous-performance 2-back working memory task with oriented-grating stimuli. We tracked the transition of the neural representation of an item (n) from its initial encoding, to the status of 'unprioritized memory item' (UMI), and back to 'prioritized memory item', with multivariate inverted encoding modelling. Results showed that the representational format was remapped from its initially encoded format into a distinctive 'opposite' representational format when it became a UMI and then mapped back into its initial format when subsequently prioritized in anticipation of its comparison with item n + 2. Thus, contrary to the default assumption that the activity representing an item in working memory might simply get weaker when it is deprioritized, it may be that a process of priority-based remapping helps to protect remembered information when it is not in the focus of attention.

A Bayesian and efficient observer model explains concurrent attractive and repulsive history biases in visual perception

Human perceptual decisions can be repelled away from (repulsive adaptation) or attracted towards recent visual experience (attractive serial dependence). It is currently unclear whether and how these repulsive and attractive biases interact during visual processing and what computational principles underlie these history dependencies. Here we disentangle repulsive and attractive biases by exploring their respective timescales. We find that perceptual decisions are concurrently attracted towards the short-term perceptual history and repelled from stimuli experienced up to minutes into the past. The temporal pattern of short-term attraction and long-term repulsion cannot be captured by an ideal Bayesian observer model alone. Instead, it is well captured by an ideal observer model with efficient encoding and Bayesian decoding of visual information in a slowly changing environment. Concurrent attractive and repulsive history biases in perceptual decisions may thus be the consequence of the need for visual processing to simultaneously satisfy constraints of efficiency and stability.