Ghosts in the machine: memory interference from the previous trial

Location- and feature-based selection histories make independent, qualitatively distinct contributions to urgent visuomotor performance

Attention mechanisms that guide visuomotor behaviors are classified into three broad types according to their reliance on stimulus salience, current goals, and selection histories (i.e., recent experience with events of many sorts). These forms of attentional control are clearly distinct and multifaceted, but what is largely unresolved is how they interact dynamically to determine impending visuomotor choices. To investigate this, we trained two macaque monkeys to perform an urgent version of an oddball search task in which a red target appears among three green distracters, or vice versa. By imposing urgency, performance can be tracked continuously as it transitions from uninformed guesses to informed choices, and this, in turn, permits assessment of attentional control as a function of time. We found that the probability of making a correct choice was strongly modulated by the histories of preceding target colors and target locations. Crucially, although both effects were gated by success (or reward), the two variables played dynamically distinct roles: whereas location history promoted an early motor bias, color history modulated the later perceptual evaluation. Furthermore, target color and location influenced performance independently of each other. The results show that, when combined, selection histories can give rise to enormous swings in visuomotor performance even in simple tasks with highly discriminable stimuli.

Unifying network model links recency and central tendency biases in working memory

The central tendency bias, or contraction bias, is a phenomenon where the judgment of the magnitude of items held in working memory appears to be biased toward the average of past observations. It is assumed to be an optimal strategy by the brain and commonly thought of as an expression of the brain's ability to learn the statistical structure of sensory input. On the other hand, recency biases such as serial dependence are also commonly observed and are thought to reflect the content of working memory. Recent results from an auditory delayed comparison task in rats suggest that both biases may be more related than previously thought: when the posterior parietal cortex (PPC) was silenced, both short-term and contraction biases were reduced. By proposing a model of the circuit that may be involved in generating the behavior, we show that a volatile working memory content susceptible to shifting to the past sensory experience - producing short-term sensory history biases - naturally leads to contraction bias. The errors, occurring at the level of individual trials, are sampled from the full distribution of the stimuli and are not due to a gradual shift of the memory toward the sensory distribution's mean. Our results are consistent with a broad set of behavioral findings and provide predictions of performance across different stimulus distributions and timings, delay intervals, as well as neuronal dynamics in putative working memory areas. Finally, we validate our model by performing a set of human psychophysics experiments of an auditory parametric working memory task.

Qualitatively Different Delay-Dependent Working Memory Distortions in People With Schizophrenia and Healthy Control Participants

BACKGROUND

Impairments in working memory (WM) have been well documented in people with schizophrenia (PSZ). However, these quantitative WM impairments can often be explained by nonspecific factors, such as impaired goal maintenance. Here, we used a spatial orientation delayed response task to explore a qualitative difference in WM dynamics between PSZ and healthy control participants (HCs). More specifically, we took advantage of the discovery that WM representations may drift either toward or away from previous trial targets (serial dependence). We tested the hypothesis that WM representations would drift toward the previous trial target in HCs but away from the previous trial target in PSZ.

METHODS

We assessed serial dependence in PSZ (n = 31) and HCs (n = 25) using orientation as the to-be-remembered feature and memory delays lasting from 0 to 8 seconds. Participants were asked to remember the orientation of a teardrop-shaped object and reproduce the orientation after a delay period of varying length.

RESULTS

Consistent with prior studies, we found that current trial memory representations were less precise in PSZ than in HCs. We also found that WM for the current trial orientation drifted toward the previous trial orientation in HCs (representational attraction) but drifted away from the previous trial orientation in PSZ (representational repulsion).

CONCLUSIONS

These results demonstrate a qualitative difference in WM dynamics between PSZ and HCs that cannot be easily explained by nuisance factors such as reduced effort. Most computational neuroscience models also fail to explain these results because they maintain information solely by means of sustained neural firing, which does not extend across trials. The results suggest a fundamental difference between PSZ and HCs in longer-term memory mechanisms that persist across trials, such as short-term potentiation and neuronal adaptation.

Serial dependence in timing at the perceptual level being modulated by working memory

Recent experiences bias the perception of following stimuli, as has been verified in various kinds of experiments in visual perception. This phenomenon, known as serial dependence, may reflect mechanisms to maintain perceptual stability. In the current study, we examined several key properties of serial dependence in temporal perception. Firstly, we examined the source of the serial dependence effect in temporal perception. We found that perception without motor reproduction is sufficient to induce the sequential effect; motor reproduction caused a stronger effect and is achieved by biasing the perception of the future target duration rather than directly influencing the subsequent movement. Secondly, we ask how working memory influences serial dependence in a temporal reproduction task. By varying the delay time between standard duration and the reproduction, we showed that the strength of serial dependence is enhanced as the delay increased. Those features of serial dependence are consistent with what has been observed in visual perceptual tasks, for example, orientation perception or location perception. The similarities between the visual and the timing tasks may suggest a similar neural coding mechanism of magnitude between the visual stimuli and the duration.

Learning efficient representations of environmental priors in working memory

Experience shapes our expectations and helps us learn the structure of the environment. Inference models render such learning as a gradual refinement of the observer's estimate of the environmental prior. For instance, when retaining an estimate of an object's features in working memory, learned priors may bias the estimate in the direction of common feature values. Humans display such biases when retaining color estimates on short time intervals. We propose that these systematic biases emerge from modulation of synaptic connectivity in a neural circuit based on the experienced stimulus history, shaping the persistent and collective neural activity that encodes the stimulus estimate. Resulting neural activity attractors are aligned to common stimulus values. Using recently published human response data from a delayed-estimation task in which stimuli (colors) were drawn from a heterogeneous distribution that did not necessarily correspond with reported population biases, we confirm that most subjects' response distributions are better described by experience-dependent learning models than by models with fixed biases. This work suggests systematic limitations in working memory reflect efficient representations of inferred environmental structure, providing new insights into how humans integrate environmental knowledge into their cognitive strategies.

Attractor dynamics with activity-dependent plasticity capture human working memory across time scales

Most cognitive functions require the brain to maintain immediately preceding stimuli in working memory. Here, using a human working memory task with multiple delays, we test the hypothesis that working memories are stored in a discrete set of stable neuronal activity configurations called attractors. We show that while discrete attractor dynamics can approximate working memory on a single time scale, they fail to generalize across multiple timescales. This failure occurs because at longer delay intervals the responses contain more information about the stimuli than can be stored in a discrete attractor model. We present a modeling approach that combines discrete attractor dynamics with activity-dependent plasticity. This model successfully generalizes across all timescales and correctly predicts intertrial interactions. Thus, our findings suggest that discrete attractor dynamics are insufficient to model working memory and that activity-dependent plasticity improves durability of information storage in attractor systems.

Spatial correspondence in relative space regulates serial dependence

Our perception is often attracted to what we have seen before, a phenomenon called 'serial dependence.' Serial dependence can help maintain a stable perception of the world, given the statistical regularity in the environment. If serial dependence serves this presumed utility, it should be pronounced when consecutive elements share the same identity when multiple elements spatially shift across successive views. However, such preferential serial dependence between identity-matching elements in dynamic situations has never been empirically tested. Here, we hypothesized that serial dependence between consecutive elements is modulated more effectively by the spatial correspondence in relative space than by that in absolute space because spatial correspondence in relative coordinates can warrant identity matching invariantly to changes in absolute coordinates. To test this hypothesis, we developed a task where two targets change positions in unison between successive views. We found that serial dependence was substantially modulated by the correspondence in relative coordinates, but not by that in absolute coordinates. Moreover, such selective modulation by the correspondence in relative space was also observed even for the serial dependence defined by previous non-target elements. Our findings are consistent with the view that serial dependence subserves object-based perceptual stabilization over time in dynamic situations.

Serial dependence in visual perception: A meta-analysis and review

Positive sequential dependencies are phenomena in which actions, perception, decisions, and memory of features or objects are systematically biased toward visual experiences from the recent past. Among many labels, serial dependencies have been referred to as priming, sequential dependencies, sequential effects, or serial effects. Despite extensive research on the topic, the field still lacks an operational definition of what counts as serial dependence. In this meta-analysis, we review the vast literature on serial dependence and quantitatively assess its key diagnostic characteristics across several different domains of visual perception. The meta-analyses fully characterize serial dependence in orientation, face, and numerosity perception. They show that serial dependence is defined by four main kinds of tuning: serial dependence decays with time (temporal-tuning), it depends on relative spatial location (spatial-tuning), it occurs only between similar features and objects (feature-tuning), and it is modulated by attention (attentional-tuning). We also review studies of serial dependence that report single observer data, highlighting the importance of individual differences in serial dependence. Finally, we discuss a range of outstanding questions and novel research avenues that are prompted by the meta-analyses. Together, the meta-analyses provide a full characterization of serial dependence as an operationally defined family of visual phenomena, and they outline several of the key diagnostic criteria for serial dependence that should serve as guideposts for future research.

Stimulation along the anterior-posterior axis of lateral frontal cortex reduces visual serial dependence

Serial dependence is an attractive pull that recent perceptual history exerts on current judgments. Theory suggests that this bias is due to a form of short-term plasticity prevalent specifically in the frontal lobe. We sought to test the importance of the frontal lobe to serial dependence by disrupting neural activity along its lateral surface during two tasks with distinct perceptual and motor demands. In our first experiment, stimulation of the lateral prefrontal cortex (LPFC) during an oculomotor delayed response task decreased serial dependence only in the first saccade to the target, whereas stimulation posterior to the LPFC decreased serial dependence only in adjustments to eye position after the first saccade. In our second experiment, which used an orientation discrimination task, stimulation anterior to, in, and posterior to the LPFC all caused equivalent decreases in serial dependence. In this experiment, serial dependence occurred only between stimuli at the same location; an alternation bias was observed across hemifields. Frontal stimulation had no effect on the alternation bias. Transcranial magnetic stimulation to parietal cortex had no effect on serial dependence in either experiment. In summary, our experiments provide evidence for both functional differentiation (Experiment 1) and redundancy (Experiment 2) in frontal cortex with respect to serial dependence.

Qualitatively different delay-dependent working memory distortions in people with schizophrenia and healthy control subjects

Background

Impairments in working memory(WM) have been well-documented in people with schizophrenia(PSZ). However, these quantitative WM impairments can often be explained by nonspecific factors, such as impaired goal maintenance. Here, we used a spatial orientation delayed-response task to explore a qualitative difference in WM dynamics between PSZ and healthy control subjects(HCS). Specifically, we took advantage of the discovery that WM representations may drift either toward or away from previous-trial targets(serial dependence). We tested the hypothesis that WM representations drift toward the previous-trial target in HCS but away from the previous-trial target in PSZ.

Methods

We assessed serial dependence in PSZ(N=31) and HCS(N=25), using orientation as the to-be-remembered feature and memory delays from 0 to 8s. Participants were asked to remember the orientation of a teardrop-shaped object and reproduce the orientation after a varying delay period.

Results

Consistent with prior studies, we found that current-trial memory representations were less precise in PSZ than in HCS. We also found that WM for the current-trial orientation drifted toward the previous-trial orientation in HCS(representational attraction) but drifted away from the previous-trial orientation in PSZ(representational repulsion).

Conclusions

These results demonstrate a qualitative difference in WM dynamics between PSZ and HCS that cannot easily be explained by nuisance factors such as reduced effort. Most computational neuroscience models also fail to explain these results, because they maintain information solely by means of sustained neural firing, which does not extend across trials. The results suggest a fundamental difference between PSZ and HCS in longer-term memory mechanisms that persist across trials, such as short-term potentiation and neuronal adaptation.

Reconstructing a disambiguation sequence that forms perceptual memory of multistable displays via reverse correlation method: Bias onset perception but gently

When multistable displays are presented intermittently with long blank intervals, their onset perception is determined by perceptual memory of multistable displays. We investigated when and how it is formed using a reverse correlation method and bistable kinetic depth effect displays. Each experimental block consisted of interleaved fully ambiguous probe and exogenously disambiguated prime displays. The purpose of the former was to "read out" the perceptual memory, whereas the latter contained purely random disambiguation sequences that were presented at the beginning of the prime display, throughout the entire presentation, or at the beginning and the end of the presentation. For each experiment and condition, we selected a subset of trials with disambiguation sequences that led to a change in perception of either the prime itself (sequences that modified perception) or the following fully ambiguous probe (sequences that modified perceptual memory). We estimated average disambiguation sequences for each participant using additive linear models. We found that an optimal sequence started at the onset with a moderate disambiguation against the previously dominant state (dominant perception for the previous probe) that gradually reduced until the display is fully ambiguous. We also show that the same sequence leads to an altered perception of the prime, indicating that perception and perceptual memory form at the same time. We suggest that perceptual memory is a consequence of an earlier evidence accumulation process and is informative about how the visual system treated ambiguity in the past rather than how it anticipates an uncertain future.

A key role of orientation in the coding of visual motion direction

Despite the fundamental importance of visual motion processing, our understanding of how the brain represents basic aspects of motion is incomplete. While it is generally believed that direction is the main representational feature of motion, motion processing is also influenced by nondirectional orientation signals that are present in most motion stimuli. Here, we aimed to test whether this nondirectional motion axis contributes motion perception even when orientation is completely absent from the stimulus. Using stimuli with and without orientation signals, we found that serial dependence in a simple motion direction estimation task was predominantly determined by the orientation of the previous motion stimulus. Moreover, the observed attraction profiles closely matched the characteristic pattern of serial attraction found in orientation perception. Evidently, the sequential integration of motion signals strongly depends on the orientation of motion, indicating a fundamental role of nondirectional orientation in the coding of visual motion direction.

Attractive serial dependence overcomes repulsive neuronal adaptation

Sensory responses and behavior are strongly shaped by stimulus history. For example, perceptual reports are sometimes biased toward previously viewed stimuli (serial dependence). While behavioral studies have pointed to both perceptual and postperceptual origins of this phenomenon, neural data that could elucidate where these biases emerge is limited. We recorded functional magnetic resonance imaging (fMRI) responses while human participants (male and female) performed a delayed orientation discrimination task. While behavioral reports were attracted to the previous stimulus, response patterns in visual cortex were repelled. We reconciled these opposing neural and behavioral biases using a model where both sensory encoding and readout are shaped by stimulus history. First, neural adaptation reduces redundancy at encoding and leads to the repulsive biases that we observed in visual cortex. Second, our modeling work suggest that serial dependence is induced by readout mechanisms that account for adaptation in visual cortex. According to this account, the visual system can simultaneously improve efficiency via adaptation while still optimizing behavior based on the temporal structure of natural stimuli.

The coding principals of early sensory regions are in constant flux due to adaptation, but how does the brain interpret these labile signals from early sensory areas? This study finds that a visual illusion known as serial dependence can be explained by a model where readout of these early areas also changes dynamically; the model is supported by neuroimaging and behavioral data.

Magnitude and sources of proactive interference in visual memory

Proactive interference - the disruptive effect of old memories on new learning - is a long-established forgetting mechanism, yet there are doubts about its impact on visual working memory and uncertainty about the kinds of information that cause proactive interference. The present study aimed to assess these issues in three experiments using a modified recent probes task. Participants encoded four target images on each trial and determined whether a probe matched one of those targets. In Experiment 1, probes matching targets from trial N-1 or N-3 damaged responding in relation to a novel probe. Proactive interference was also produced by probes differing in state to a previously experienced target. This was further assessed in Experiments 2 and 3. Here, probes differing in colour to a previous target, or matching the general target category only, produced little proactive interference. Conversely, probes directly matching a prior target, or differing in state information, hindered task performance. This study found robust proactive interference in visual working memory that could endure over multiple trials, but it was also produced by stimuli closely resembling an old target. This challenges the notion that proactive interference is produced by an exact representation of a previously encoded image.

Working Memory: From Neural Activity to the Sentient Mind

Working memory (WM) is the ability to maintain and manipulate information in the conscious mind over a timescale of seconds. This ability is thought to be maintained through the persistent discharges of neurons in a network of brain areas centered on the prefrontal cortex, as evidenced by neurophysiological recordings in nonhuman primates, though both the localization and the neural basis of WM has been a matter of debate in recent years. Neural correlates of WM are evident in species other than primates, including rodents and corvids. A specialized network of excitatory and inhibitory neurons, aided by neuromodulatory influences of dopamine, is critical for the maintenance of neuronal activity. Limitations in WM capacity and duration, as well as its enhancement during development, can be attributed to properties of neural activity and circuits. Changes in these factors can be observed through training-induced improvements and in pathological impairments. WM thus provides a prototypical cognitive function whose properties can be tied to the spiking activity of brain neurons. © 2021 American Physiological Society. Compr Physiol 11:1-41, 2021.

Across-Area Synchronization Supports Feature Integration in a Biophysical Network Model of Working Memory

Working memory function is severely limited. One key limitation that constrains the ability to maintain multiple items in working memory simultaneously is so-called swap errors. These errors occur when an inaccurate response is in fact accurate relative to a non-target stimulus, reflecting the failure to maintain the appropriate association or "binding" between the features that define one object (e.g., color and location). The mechanisms underlying feature binding in working memory remain unknown. Here, we tested the hypothesis that features are bound in memory through synchrony across feature-specific neural assemblies. We built a biophysical neural network model composed of two one-dimensional attractor networks - one for color and one for location - simulating feature storage in different cortical areas. Within each area, gamma oscillations were induced during bump attractor activity through the interplay of fast recurrent excitation and slower feedback inhibition. As a result, different memorized items were held at different phases of the network's oscillation. These two areas were then reciprocally connected via weak cortico-cortical excitation, accomplishing binding between color and location through the synchronization of pairs of bumps across the two areas. Encoding and decoding of color-location associations was accomplished through rate coding, overcoming a long-standing limitation of binding through synchrony. In some simulations, swap errors arose: "color bumps" abruptly changed their phase relationship with "location bumps." This model, which leverages the explanatory power of similar attractor models, specifies a plausible mechanism for feature binding and makes specific predictions about swap errors that are testable at behavioral and neurophysiological levels.

Temporal dynamics of implicit memory underlying serial dependence

Serial dependence is the effect in which the immediately preceding trial influences participants' responses to the current stimulus. But for how long does this bias last in the absence of interference from other stimuli? Here, we had 20 healthy young adult participants (12 women) perform a coincident timing task using different inter-trial intervals to characterize the serial dependence effect as the time between trials increases. Our results show that serial dependence abruptly decreases from 0.1 s to 1 s inter-trial interval, but it remains pronounced after that for up to 8 s. In addition, participants' response variability slightly decreases over longer intervals. We discuss these results in light of recent models suggesting that serial dependence might rely on a short-term memory trace kept through changes in synaptic weights, which might explain its long duration and apparent stability over time.

Serial dependence revealed in history-dependent perceptual templates

In any given perceptual task, the visual system selectively weighs or filters incoming information. The particular set of weights or filters form a kind of template, which reveals the regions or types of information that are particularly useful for a given perceptual decision.1,2 Unfortunately, sensory input is noisy and ever changing. To compensate for these fluctuations, the visual system could adopt a strategy of biasing the templates such that they reflect a temporal smoothing of input, which would be a form of serial dependence.3-5 Here, we demonstrate that perceptual templates are, in fact, altered by serial dependence. Using a simple orientation detection task and classification-image technique, we found that perceptual templates are systematically biased toward previously seen, task-irrelevant orientations. The results of an orientation discrimination task suggest that this shift in perceptual template derives from a change in the perceptual appearance of orientation. Our study reveals how serial dependence biases internal templates of orientation and suggests that the sensitivity of classification-image techniques in general could be improved by taking into account history-dependent fluctuations in templates.

Balancing Flexibility and Interference in Working Memory

Working memory is central to cognition, flexibly holding the variety of thoughts needed for complex behavior. Yet, despite its importance, working memory has a severely limited capacity, holding only three to four items at once. In this article, I review experimental and computational evidence that the flexibility and limited capacity of working memory reflect the same underlying neural mechanism. I argue that working memory relies on interactions between high-dimensional, integrative representations in the prefrontal cortex and structured representations in the sensory cortex. Together, these interactions allow working memory to flexibly maintain arbitrary representations. However, the distributed nature of working memory comes at the cost of causing interference between items in memory, resulting in a limited capacity. Finally, I discuss several mechanisms used by the brain to reduce interference and maximize the effective capacity of working memory. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Motor learning by selection in visual working memory

Motor adaptation maintains movement accuracy over the lifetime. Saccadic eye movements have been used successfully to study the mechanisms and neural basis of adaptation. Using behaviorally irrelevant targets, it has been shown that saccade adaptation is driven by errors only in a brief temporal interval after movement completion. However, under natural conditions, eye movements are used to extract information from behaviorally relevant objects and to guide actions manipulating these objects. In this case, the action outcome often becomes apparent only long after movement completion, outside the supposed temporal window of error evaluation. Here, we show that saccade adaptation can be driven by error signals long after the movement when using behaviorally relevant targets. Adaptation occurred when a task-relevant target appeared two seconds after the saccade, or when a retro-cue indicated which of two targets, stored in visual working memory, was task-relevant. Our results emphasize the important role of visual working memory for optimal movement control.

Frontotemporal coordination predicts working memory performance and its local neural signatures

Neurons in some sensory areas reflect the content of working memory (WM) in their spiking activity. However, this spiking activity is seldom related to behavioral performance. We studied the responses of inferotemporal (IT) neurons, which exhibit object-selective activity, along with Frontal Eye Field (FEF) neurons, which exhibit spatially selective activity, during the delay period of an object WM task. Unlike the spiking activity and local field potentials (LFPs) within these areas, which were poor predictors of behavioral performance, the phase-locking of IT spikes and LFPs with the beta band of FEF LFPs robustly predicted successful WM maintenance. In addition, IT neurons exhibited greater object-selective persistent activity when their spikes were locked to the phase of FEF LFPs. These results reveal that the coordination between prefrontal and temporal cortex predicts the successful maintenance of visual information during WM.

Extracting the dynamics of behavior in sensory decision-making experiments

Decision-making strategies evolve during training and can continue to vary even in well-trained animals. However, studies of sensory decision-making tend to characterize behavior in terms of a fixed psychometric function that is fit only after training is complete. Here, we present PsyTrack, a flexible method for inferring the trajectory of sensory decision-making strategies from choice data. We apply PsyTrack to training data from mice, rats, and human subjects learning to perform auditory and visual decision-making tasks. We show that it successfully captures trial-to-trial fluctuations in the weighting of sensory stimuli, bias, and task-irrelevant covariates such as choice and stimulus history. This analysis reveals dramatic differences in learning across mice and rapid adaptation to changes in task statistics. PsyTrack scales easily to large datasets and offers a powerful tool for quantifying time-varying behavior in a wide variety of animals and tasks.

Perceptual history propagates down to early levels of sensory analysis

One function of perceptual systems is to construct and maintain a reliable representation of the environment. A useful strategy intrinsic to modern “Bayesian” theories of perception1, 2, 3, 4, 5, 6 is to take advantage of the relative stability of the input and use perceptual history (priors) to predict current perception. This strategy is efficient1, 2, 3, 4, 5, 6, 7 but can lead to stimuli being biased toward perceptual history, clearly revealed in a phenomenon known as serial dependence.8, 9, 10, 11, 12, 13, 14 However, it is still unclear whether serial dependence biases sensory encoding or only perceptual decisions.^15,16 We leveraged on the “surround tilt illusion”—where tilted flanking stimuli strongly bias perceived orientation—to measure its influence on the pattern of serial dependence, which is typically maximal for similar orientations of past and present stimuli.^7,10 Maximal serial dependence for a neutral stimulus preceded by an illusory one occurred when the perceived, not the physical, orientations of the two stimuli matched, suggesting that the priors biasing current perception incorporate the effect of the illusion. However, maximal serial dependence of illusory stimuli induced by neutral stimuli occurred when their physical (not perceived) orientations were matched, suggesting that priors interact with incoming sensory signals before they are biased by flanking stimuli. The evidence suggests that priors are high-level constructs incorporating contextual information, which interact directly with early sensory signals, not with highly processed perceptual representations.

•

Perception is heavily biased by perceptual history and expectations

•

Perceptual history includes illusory effects driven by spatial context

•

This representation propagates back to sensory areas preceding context effects

•

The results point to a neural architecture consistent with predictive coding

Delay-period activity in frontal, parietal, and occipital cortex tracks noise and biases in visual working memory

Working memory is imprecise, and these imprecisions can be explained by the combined influences of random diffusive error and systematic drift toward a set of stable states ("attractors"). However, the neural correlates of diffusion and drift remain unknown. Here, we investigated how delay-period activity in frontal and parietal cortex, which is known to correlate with the decline in behavioral memory precision observed with increasing memory load, might relate to diffusion and drift. We analyzed data from an existing experiment in which subjects performed delayed recall for line orientation, at different loads, during functional magnetic resonance imaging (fMRI) scanning. To quantify the influence of drift and diffusion, we modeled subjects' behavior using a discrete attractor model and calculated within-subject correlation between frontal and parietal delay-period activity and whole-trial estimates of drift and diffusion. We found that although increases in frontal and parietal activity were associated with increases in both diffusion and drift, diffusion explained the most variance in frontal and parietal delay-period activity. In comparison, a subsequent whole-brain regression analysis showed that drift, rather than diffusion, explained the most variance in delay-period activity in lateral occipital cortex. These results are consistent with a model of the differential recruitment of general frontoparietal mechanisms in response to diffusive noise and of stimulus-specific biases in occipital cortex.

Interplay between persistent activity and activity-silent dynamics in the prefrontal cortex underlies serial biases in working memory

Persistent neuronal spiking has long been considered the mechanism underlying working memory, but recent proposals argue for alternative 'activity-silent' substrates. Using monkey and human electrophysiology data, we show here that attractor dynamics that control neural spiking during mnemonic periods interact with activity-silent mechanisms in the prefrontal cortex (PFC). This interaction allows memory reactivations, which enhance serial biases in spatial working memory. Stimulus information was not decodable between trials, but remained present in activity-silent traces inferred from spiking synchrony in the PFC. Just before the new stimulus, this latent trace was reignited into activity that recapitulated the previous stimulus representation. Importantly, the reactivation strength correlated with the strength of serial biases in both monkeys and humans, as predicted by a computational model that integrates activity-based and activity-silent mechanisms. Finally, single-pulse transcranial magnetic stimulation applied to the human PFC between successive trials enhanced serial biases, thus demonstrating the causal role of prefrontal reactivations in determining working-memory behavior.

Build-up of serial dependence in color working memory

Serial dependence, how immediately preceding experiences bias our current estimations, has been described experimentally during delayed-estimation of many different visual features, with subjects tending to make estimates biased towards previous ones. It has been proposed that these attractive biases help perception stabilization in the face of correlated natural scene statistics, although this remains mostly theoretical. Color, which is strongly correlated in natural scenes, has never been studied with regard to its serial dependencies. Here, we found significant serial dependence in 7 out of 8 datasets with behavioral data of humans (total n = 760) performing delayed-estimation of color with uncorrelated sequential stimuli. Moreover, serial dependence strength built up through the experimental session, suggesting metaplastic mechanisms operating at a slower time scale than previously proposed (e.g. short-term synaptic facilitation). Because, in contrast with natural scenes, stimuli were temporally uncorrelated, this build-up casts doubt on serial dependencies being an ongoing adaptation to the stable statistics of the environment.

A Bayesian and efficient observer model explains concurrent attractive and repulsive history biases in visual perception

Human perceptual decisions can be repelled away from (repulsive adaptation) or attracted towards recent visual experience (attractive serial dependence). It is currently unclear whether and how these repulsive and attractive biases interact during visual processing and what computational principles underlie these history dependencies. Here we disentangle repulsive and attractive biases by exploring their respective timescales. We find that perceptual decisions are concurrently attracted towards the short-term perceptual history and repelled from stimuli experienced up to minutes into the past. The temporal pattern of short-term attraction and long-term repulsion cannot be captured by an ideal Bayesian observer model alone. Instead, it is well captured by an ideal observer model with efficient encoding and Bayesian decoding of visual information in a slowly changing environment. Concurrent attractive and repulsive history biases in perceptual decisions may thus be the consequence of the need for visual processing to simultaneously satisfy constraints of efficiency and stability.

Context information supports serial dependence of multiple visual objects across memory episodes

Serial dependence is thought to promote perceptual stability by compensating for small changes of an object’s appearance across memory episodes. So far, it has been studied in situations that comprised only a single object. The question of how we selectively create temporal stability of several objects remains unsolved. In a memory task, objects can be differentiated by their to-be-memorized feature (content) as well as accompanying discriminative features (context). We test whether congruent context features, in addition to content similarity, support serial dependence. In four experiments, we observe a stronger serial dependence between objects that share the same context features across trials. Apparently, the binding of content and context features is not erased but rather carried over to the subsequent memory episode. As this reflects temporal dependencies in natural settings, our findings reveal a mechanism that integrates corresponding content and context features to support stable representations of individualized objects over time.

Visual cognition compensates for small changes in an object’s appearance to ensure its perceived continuity. We show that in situations with multiple objects, context features like color, temporal or spatial position are used as anchors to selectively integrate corresponding objects over time.

Serial dependence in vision: Merely encoding the previous-trial target is not enough

The reported perception of a visual stimulus on one trial can be biased by the stimulus that was presented on the previous trial. In the present study we asked whether encoding the previous-trial stimulus is sufficient to produce this serial dependence effect, or whether the effect also depends on postencoding processes. To distinguish between these possibilities, we designed a task in which participants reported either the color or the direction of a set of colored moving dots on each trial. The to-be-reported dimension was indicated by a postcue after stimulus offset, so participants were required to encode both features of every stimulus. We assessed serial dependence for motion perception as a function of which feature dimension had been reported on the previous trial. In Experiment 1, we found a serial dependence effect for motion only when participants had reported the direction of motion on the previous trial, and not when they had encoded the direction of motion but reported the color of the stimulus. Experiment 2 confirmed that this pattern of results was not driven by the difficulty of the color task. When we used the same response modality for both motion and color reports in Experiment 3, we found significant serial dependence effects following both color-report and motion-report trials, but the effect was significantly weaker following color-report trials. Together, these findings indicate that postperceptual processes play a critical role in serial dependence and that the mere encoding of the previous-trial target is not sufficient to produce the serial dependence effect.

Error-correcting dynamics in visual working memory

Working memory is critical to cognition, decoupling behavior from the immediate world. Yet, it is imperfect; internal noise introduces errors into memory representations. Such errors have been shown to accumulate over time and increase with the number of items simultaneously held in working memory. Here, we show that discrete attractor dynamics mitigate the impact of noise on working memory. These dynamics pull memories towards a few stable representations in mnemonic space, inducing a bias in memory representations but reducing the effect of random diffusion. Model-based and model-free analyses of human and monkey behavior show that discrete attractor dynamics account for the distribution, bias, and precision of working memory reports. Furthermore, attractor dynamics are adaptive. They increase in strength as noise increases with memory load and experiments in humans show these dynamics adapt to the statistics of the environment, such that memories drift towards contextually-predicted values. Together, our results suggest attractor dynamics mitigate errors in working memory by counteracting noise and integrating contextual information into memories.

Neural representations in working memory are susceptible to internal noise, which scales with memory load. Here, the authors show that attractor dynamics mitigate the influence of internal noise by pulling memories towards a few stable representations.

Working Memory Maintenance Modulates Serial Dependence Effects of Perceived Emotional Expression

The stability of face perception is vital in interpersonal interactions. Recent studies have revealed the mechanism of the stability in the perception of stable attributes of faces (such as facial identity) by serial dependence, a phenomenon in which perception of current stimuli is pulled toward recently viewed stimuli. However, whether serial dependence of perceived emotional expression (a changeable attribute of faces) exists remains controversial, and its exact nature has not been examined yet. To address these issues, we used the methods of constant stimuli and two-interval forced choice tasks in three psychophysical experiments. Participants compared two successive facial expressions selected from a continuum with 50 morphed faces ranging from sad to happy. Experiment 1a and 1b showed that a perceived facial expression pulled toward previously seen facial expressions (i.e., a significant serial dependence effect), independent of response instructions. Furthermore, a stronger serial dependence effect was found when the first facial expression was retained in working memory for a longer delay duration (Experiment 2), and yet a weaker serial dependence effect was observed when a longer delay between decision and response was performed (Experiment 3). These findings indicate that serial dependence facilitates the stability of facial expression perception and is modulated by working memory representations.

Use of cognitive correction training improves learning for children with mathematics learning disability

Children with normal intelligence may experience varying degrees of mathematics learning disabilities (MD). This study aims to conduct training to improve the brain's cognitive ability for mathematics learning by focusing on two important mathematical cognitive abilities. This was a prospective study of 70 children in grades 2-5 from two primary schools in Changzhou and with MD enrolled from June 2015 to February 2017. The children were randomized 1:1 to the training and control groups. A training cycle included 40 sessions (5/weeks) (30 min each session). The efficacy of learning was assessed by assessing number learning and graph reasoning, and by using the Raven standard reasoning test score. In the training group, backward number memory (from 6.1 ± 1.8 to 6.7 ± 1.3, P = 0.02), number sequential connection (from 54.4 ± 14.5 to 47.1 ± 12.1, P < 0.01), and rapid graph judgment (from 531.9 ± 76.3 to 557.8 ± 85.7, P = 0.04) were improved by training, while there was no effect on forward number memory (P = 0.13). There were significant differences in total score and scores of b, c, and e series before and after training (all P < 0.05). The children in the control group had no improvement after 8 weeks. There was a correlation between the ability of rapid graphic judgment before and after training and the score of the Raven E series (r = 0.384, P = 0.024), and between the score of the Raven C series and the score of the Raven D series (r = 0.468, P = 0.013). Cognitive correction training improved the sensitivity to numbers and mathematics learning in children with MD.

Optimizing sequential decisions in the drift-diffusion model

To make decisions organisms often accumulate information across multiple timescales. However, most experimental and modeling studies of decision-making focus on sequences of independent trials. On the other hand, natural environments are characterized by long temporal correlations, and evidence used to make a present choice is often relevant to future decisions. To understand decision-making under these conditions we analyze how a model ideal observer accumulates evidence to freely make choices across a sequence of correlated trials. We use principles of probabilistic inference to show that an ideal observer incorporates information obtained on one trial as an initial bias on the next. This bias decreases the time, but not the accuracy of the next decision. Furthermore, in finite sequences of trials the rate of reward is maximized when the observer deliberates longer for early decisions, but responds more quickly towards the end of the sequence. Our model also explains experimentally observed patterns in decision times and choices, thus providing a mathematically principled foundation for evidence-accumulation models of sequential decisions.

Serial dependence in position occurs at the time of perception

Observers perceive objects in the world as stable over space and time, even though the visual experience of those objects is often discontinuous and distorted due to masking, occlusion, camouflage, or noise. How are we able to easily and quickly achieve stable perception in spite of this constantly changing visual input? It was previously shown that observers experience serial dependence in the perception of features and objects, an effect that extends up to 15 seconds back in time. Here, we asked whether the visual system utilizes an object's prior physical location to inform future position assignments in order to maximize location stability of an object over time. To test this, we presented subjects with small targets at random angular locations relative to central fixation in the peripheral visual field. Subjects reported the perceived location of the target on each trial by adjusting a cursor's position to match its location. Subjects made consistent errors when reporting the perceived position of the target on the current trial, mislocalizing it toward the position of the target in the preceding two trials (Experiment 1). This pull in position perception occurred even when a response was not required on the previous trial (Experiment 2). In addition, we show that serial dependence in perceived position occurs immediately after stimulus presentation, and it is a fast stabilization mechanism that does not require a delay (Experiment 3). This indicates that serial dependence occurs for position representations and facilitates the stable perception of objects in space. Taken together with previous work, our results show that serial dependence occurs at many stages of visual processing, from initial position assignment to object categorization.

Reference repulsion is not a perceptual illusion

Perceptual decisions are often influenced by contextual factors. For instance, when engaged in a visual discrimination task against a reference boundary, subjective reports about the judged stimulus feature are biased away from the boundary - a phenomenon termed reference repulsion. Until recently, this phenomenon has been thought to reflect a perceptual illusion regarding the appearance of the stimulus, but new evidence suggests that it may rather reflect a post-perceptual decision bias. To shed light on this issue, we examined whether and how orientation judgments affect perceptual appearance. In a first experiment, we confirmed that after judging a grating stimulus against a discrimination boundary, the subsequent reproduction response was indeed repelled from the boundary. To investigate the perceptual nature of this bias, in a second experiment we measured the perceived orientation of the grating stimulus more directly, in comparison to a reference stimulus visible at the same time. Although we did observe a small repulsive bias away from the boundary, this bias was explained by random trial-by-trial fluctuations in sensory representations together with classical stimulus adaptation effects and did not reflect a systematic bias due to the discrimination judgment. Overall, the current study indicates that discrimination judgments do not elicit a perceptual illusion and points towards a post-perceptual locus of reference repulsion.

Synaptic mechanisms of interference in working memory

Information from preceding trials of cognitive tasks can bias performance in the current trial, a phenomenon referred to as interference. Subjects performing visual working memory tasks exhibit interference in their responses: the recalled target location is biased in the direction of the target presented on the previous trial. We present modeling work that develops a probabilistic inference model of this history-dependent bias, and links our probabilistic model to computations of a recurrent network wherein short-term facilitation accounts for the observed bias. Network connectivity is reshaped dynamically during each trial, generating predictions from prior trial observations. Applying timescale separation methods, we obtain a low-dimensional description of the trial-to-trial bias based on the history of target locations. Furthermore, we demonstrate task protocols for which our model with facilitation performs better than a model with static connectivity: repetitively presented targets are better retained in working memory than targets drawn from uncorrelated sequences.

Synaptic augmentation in a cortical circuit model reproduces serial dependence in visual working memory

Recent work has established that visual working memory is subject to serial dependence: current information in memory blends with that from the recent past as a function of their similarity. This tuned temporal smoothing likely promotes the stability of memory in the face of noise and occlusion. Serial dependence accumulates over several seconds in memory and deteriorates with increased separation between trials. While this phenomenon has been extensively characterized in behavior, its neural mechanism is unknown. In the present study, we investigate the circuit-level origins of serial dependence in a biophysical model of cortex. We explore two distinct kinds of mechanisms: stable persistent activity during the memory delay period and dynamic “activity-silent” synaptic plasticity. We find that networks endowed with both strong reverberation to support persistent activity and dynamic synapses can closely reproduce behavioral serial dependence. Specifically, elevated activity drives synaptic augmentation, which biases activity on the subsequent trial, giving rise to a spatiotemporally tuned shift in the population response. Our hybrid neural model is a theoretical advance beyond abstract mathematical characterizations, offers testable hypotheses for physiological research, and demonstrates the power of biological insights to provide a quantitative explanation of human behavior.

Serial dependence is absent at the time of perception but increases in visual working memory

Recent experiments have shown that visual cognition blends current input with that from the recent past to guide ongoing decision making. This serial dependence appears to exploit the temporal autocorrelation normally present in visual scenes to promote perceptual stability. While this benefit has been assumed, evidence that serial dependence directly alters stimulus perception has been limited. In the present study, we parametrically vary the delay between stimulus and response in a spatial delayed response task to explore the trajectory of serial dependence from the moment of perception into post-perceptual visual working memory. We find that behavioral responses made immediately after viewing a stimulus show evidence of adaptation, but not attractive serial dependence. Only as the memory period lengthens is a blending of past and present information apparent in behavior, reaching its maximum with a delay of six seconds. These results dovetail with other recent findings to bolster the interpretation that serial dependence is a phenomenon of mnemonic rather than perceptual processes. However, even while this pattern of effects in group-averaged data has now been found consistently, we show that the relative strengths of adaptation and serial dependence vary widely across individuals. Finally, we demonstrate that when leading mathematical models of working memory are adjusted to account for these trial-history effects, their fit to behavioral data is substantially improved.

Control of the strength of visual-motor transmission as the mechanism of rapid adaptation of priors for Bayesian inference in smooth pursuit eye movements

Bayesian inference provides a cogent account of how the brain combines sensory information with "priors" based on past experience to guide many behaviors, including smooth pursuit eye movements. We now demonstrate very rapid adaptation of the pursuit system's priors for target direction and speed. We go on to leverage that adaptation to outline possible neural mechanisms that could cause pursuit to show features consistent with Bayesian inference. Adaptation of the prior causes changes in the eye speed and direction at the initiation of pursuit. The adaptation appears after a single trial and accumulates over repeated exposure to a given history of target speeds and directions. The influence of the priors depends on the reliability of visual motion signals: priors are more effective against the visual motion signals provided by low-contrast vs. high-contrast targets. Adaptation of the direction prior generalizes to eye speed and vice versa, suggesting that both priors could be controlled by a single neural mechanism. We conclude that the pursuit system can learn the statistics of visual motion rapidly and use those statistics to guide future behavior. Furthermore, a model that adjusts the gain of visual-motor transmission predicts the effects of recent experience on pursuit direction and speed, as well as the specifics of the generalization between the priors for speed and direction. We suggest that Bayesian inference in pursuit behavior is implemented by distinctly non-Bayesian internal mechanisms that use the smooth eye movement region of the frontal eye fields to control of the gain of visual-motor transmission.NEW & NOTEWORTHY Bayesian inference can account for the interaction between sensory data and past experience in many behaviors. Here, we show, using smooth pursuit eye movements, that the priors based on past experience can be adapted over a very short time frame. We also show that a single model based on direction-specific adaptation of the strength of visual-motor transmission can explain the implementation and adaptation of priors for both target direction and target speed.

Ghosts in the Machine II: Neural Correlates of Memory Interference from the Previous Trial

Previous memoranda interfere with working memory. For example, spatial memories are biased toward locations memorized on the previous trial. We predicted, based on attractor network models of memory, that activity in the frontal eye fields (FEFs) encoding a previous target location can persist into the subsequent trial and that this ghost will then bias the readout of the current target. Contrary to this prediction, we find that FEF memory representations appear biased away from (not toward) the previous target location. The behavioral and neural data can be reconciled by a model in which receptive fields of memory neurons converge toward remembered locations, much as receptive fields converge toward attended locations. Convergence increases the resources available to encode the relevant memoranda and decreases overall error in the network, but the residual convergence from the previous trial can give rise to an attractive behavioral bias on the next trial.

Neural circuit basis of visuo-spatial working memory precision: a computational and behavioral study

The amount of information that can be retained in working memory (WM) is limited. Limitations of WM capacity have been the subject of intense research, especially in trying to specify algorithmic models for WM. Comparatively, neural circuit perspectives have barely been used to test WM limitations in behavioral experiments. Here we used a neuronal microcircuit model for visuo-spatial WM (vsWM) to investigate memory of several items. The model assumes that there is a topographic organization of the circuit responsible for spatial memory retention. This assumption leads to specific predictions, which we tested in behavioral experiments. According to the model, nearby locations should be recalled with a bias, as if the two memory traces showed attraction or repulsion during the delay period depending on distance. Another prediction is that the previously reported loss of memory precision for an increasing number of memory items (memory load) should vanish when the distances between items are controlled for. Both predictions were confirmed experimentally. Taken together, our findings provide support for a topographic neural circuit organization of vsWM, they suggest that interference between similar memories underlies some WM limitations, and they put forward a circuit-based explanation that reconciles previous conflicting results on the dependence of WM precision with load.