Attractive and repulsive effects of sensory history concurrently shape visual perception

Working memory flips the direction of serial bias through memory-based decision

Reported perception of a new stimulus is either attracted toward or repelled away from task-irrelevant prior stimuli. While prevailing theories propose that the opposing serial biases may stem from distinct stages of information processing, the exact role of working memory (WM) in the serial bias remains unclear despite its consistent involvement in nearly all pertinent studies. Additionally, it is not well understood whether this bias is primarily driven by the biased representation itself or by the decision-making process for the new stimulus. In the present study, we used an orientation delayed estimation paradigm with an attention-demanding intervening task, designed to disrupt the maintenance of stimulus information to investigate the role of WM in serial bias. In the analysis, we scrutinized the trajectory of mouse reports and response time to investigate how the response unfolds over time. Our findings indicate that the serial bias went from repulsive to attractive when WM maintenance was interrupted by the intervening task, and that the associated response trajectories and response time exhibited patterns that cannot be explained by the biased representation alone. These results demonstrate that the task-irrelevant prior stimulus influences the decision for the new stimulus, with the direction of the bias being determined by attentional demand during WM maintenance, thereby placing significant constraints on existing theories on the serial bias effect.

Serial dependence requires visual awareness: Evidence from continuous flash suppression

The visual system often undergoes a relatively stable perception even in a noisy visual environment. This crucial function was reflected in a visual perception phenomenon-serial dependence, in which recent stimulus history systematically biases current visual decisions. Although serial dependence effects have been revealed in numerous studies, few studies examined whether serial dependence would require visual awareness. By using the continuous flash suppression (CFS) technique to render grating stimuli invisible, we investigated whether serial dependence effects could emerge at the unconscious levels. In an orientation adjustment task, subjects viewed a randomly oriented grating and reported their orientation perception via an adjustment response. Subjects performed a series of three type trial pairs. The first two trial pairs, in which subjects were instructed to make a response or no response toward the first trial of the pairs, respectively, were used to measure serial dependence at the conscious levels; the third trial pair, in which the grating stimulus in the first trial of the pair was masked by a CFS stimulus, was used to measure the serial dependence at the unconscious levels. One-back serial dependence effects for the second trial of the pairs were evaluated. We found significant serial dependence effects at the conscious levels, whether absence (Experiment 1) or presence (Experiment 2) of CFS stimuli, but failed to find the effects at the unconscious levels, corroborating the view that serial dependence requires visual awareness.

Cardinal bias interacts with the stimulus history bias in orientation working memory

Reports in a visual working memory(WM) task exhibit biases related to the categorical structure of the stimulus space (e.g., cardinal bias) as well as biases related to previously seen stumuli (e.g., serial bias). While these biases are common and can occur simultaneously, the extent to which they interact in WM remains unknown. In the present study, I used orientation delayed estimation tasks known to produce both cardinal and serial biases and found that the serial bias systematically varied based on the relative positions of the cardinal axis and the preceding stimulus in orientation space. When they were positioned in a way that generated cardinal and serial biases in the same direction (i.e., on the same side of the target orientation), reports for the target orientation exhibited a regular repulsive serial bias. However, when their positions resulted in the biases in the opposite directions (i.e., on the opposite side of the target orientation), no serial bias occurred. This absence of serial bias was replicated in a follow-up experiment where the locations of the stimulus orientation and the response probe were completely randomized, suggesting that the interaction occurs independently from location-based response preparation processes. Together, these results demonstrate that the prior stimulus and the cardinal axis impose interactive impact on the processing of new stimulus, producing differential patterns of serial bias depending on the specific stimulus being processed. These findings place significant implications on computational models addressing the nature of the stimulus history effect and its underlying mechanisms.

The distinct development of stimulus and response serial dependence

Serial dependence (SD) is a phenomenon wherein current perceptions are biased by the previous stimulus and response. This helps to attenuate perceptual noise and variability in sensory input and facilitates stable ongoing perceptions of the environment. However, little is known about the developmental trajectory of SD. This study investigates how the stimulus and response biases of the SD effect develop across three age groups. Conventional analyses, in which previous stimulus and response biases were assessed separately, revealed significant changes in the biases over time. Previous stimulus bias shifted from repulsion to attraction, while previous response bias evolved from attraction to greater attraction. However, there was a strong correlation between stimulus and response orientations. Therefore, a generalized linear mixed-effects (GLME) analysis that simultaneously considered both previous stimulus and response, outperformed separate analyses. This revealed that previous stimulus and response resulted in two distinct biases with different developmental trajectories. The repulsion bias of previous stimulus remained relatively stable across all age groups, whereas the attraction bias of previous response was significantly stronger in adults than in children and adolescents. These findings demonstrate that the repulsion bias towards preceding stimuli is established early in the developing brain (at least by around 10 years old), while the attraction bias towards responses is not fully developed until adulthood. Our findings provide new insights into the development of the SD phenomenon and how humans integrate two opposing mechanisms into their perceptual responses to external input during development.

Attention influences the effects of the previous form orientation on the current motion direction estimation

Recent studies have found that the estimates of motion directions are biased toward the previous form orientations, showing serial dependence, and the serial dependence does not involve cognitive abilities. In the current study, we conducted two experiments to investigate whether and how attention-a cognitive ability-affected the serial dependence. The results showed that serial dependence was present in the current study, reproducing the previous findings. Importantly, when the attentional load reduced the reliability (i.e., estimation accuracy and precision) of previous form orientations (Experiment 1), the serial dependence decreased, meaning that the biases of motion direction estimates toward previous form orientations were reduced; in contrast, when the attentional load reduced the reliability of current motion directions (Experiment 2), the serial dependence increased, meaning that the biases of motion direction estimates toward previous form orientations were increased. These trends were well consistent with the prediction of the Bayesian inference theory. Therefore, the current study revealed the involvement of attention in the serial dependence of current motion direction estimation on the previous form orientation, demonstrating that the serial dependence was cognitive and the attentional effect can be a Bayesian inference process, initially revealing its computational mechanism.

Spatial correspondence in relative space regulates serial dependence

Our perception is often attracted to what we have seen before, a phenomenon called 'serial dependence.' Serial dependence can help maintain a stable perception of the world, given the statistical regularity in the environment. If serial dependence serves this presumed utility, it should be pronounced when consecutive elements share the same identity when multiple elements spatially shift across successive views. However, such preferential serial dependence between identity-matching elements in dynamic situations has never been empirically tested. Here, we hypothesized that serial dependence between consecutive elements is modulated more effectively by the spatial correspondence in relative space than by that in absolute space because spatial correspondence in relative coordinates can warrant identity matching invariantly to changes in absolute coordinates. To test this hypothesis, we developed a task where two targets change positions in unison between successive views. We found that serial dependence was substantially modulated by the correspondence in relative coordinates, but not by that in absolute coordinates. Moreover, such selective modulation by the correspondence in relative space was also observed even for the serial dependence defined by previous non-target elements. Our findings are consistent with the view that serial dependence subserves object-based perceptual stabilization over time in dynamic situations.

Tuning perception and decisions to temporal context

Recent work suggests that serial dependence, where perceptual decisions are biased toward previous stimuli, arises from the prior that sensory input is temporally correlated. However, existing studies have mostly used random stimulus sequences that do not involve such temporal consistencies. Here, we manipulated the temporal statistics of visual stimuli to examine the role of true temporal correlations in serial dependence. In two experiments, observers reproduced the orientation of the last stimulus in a sequence, while we varied temporal correlations in the stimulus features at two timescales: stimulus history within the trial and decision history across trials. We found a clear dissociation: increasing temporal correlation in the stimulus history led to adaptation-like repulsive biases, whereas increasing temporal correlation in the decision history reduced attractive biases. Thus, we suggest that temporal correlation enhances the discriminative ability of the visual system, revealing the fundamental role of the broader temporal context.

The effects of visual distractors on serial dependence

Attractive serial dependence occurs when perceptual decisions are attracted toward previous stimuli. This effect is mediated by spatial attention and is most likely to occur when similar stimuli are attended at nearby locations. Attention, however, also involves the suppression of distracting information and of spatial locations where distracting stimuli have frequently appeared. Although distractors form an integral part of our visual experience, how they affect the processing of subsequent stimuli is unknown. Here, in two experiments, we tested serial dependence from distractor stimuli during an orientation adjustment task. We interleaved adjustment trials with a discrimination task requiring observers to ignore a peripheral distractor randomly appearing on half of the trials. Distractors were either similar to the adjustment probe (Experiment 1) or differed in spatial frequency and contrast (Experiment 2) and were shown at predictable or random locations in separate blocks. The results showed that the distractor caused considerable attentional capture in the discrimination task, with observers likely using proactive strategies to anticipate distractors at predictable locations. However, there was no evidence that the distractors affected the perceptual stream leading to positive serial dependence. Instead, they left a weak repulsive trace in Experiment 1 and more generally interfered with the effect of the previous adjustment probe in the serial dependence task. We suggest that this repulsive bias may reflect the operation of mechanisms involved in attentional suppression.

Serial dependence improves performance and biases confidence-based decisions

Perception depends on both the current sensory input and on the preceding stimuli history, a mechanism referred to as serial dependence (SD). One interesting, and somewhat controversial, question is whether serial dependence originates at the perceptual stage, which should lead to a sensory improvement, or at a subsequent decisional stage, causing solely a bias. Here, we studied the effects of SD in a novel manner by leveraging on the human capacity to spontaneously assess the quality of sensory information. Two noisy-oriented Gabor stimuli were simultaneously presented along with two bars of the same orientation as the Gabor stimuli. Participants were asked to choose which Gabor stimulus to judge and then make a forced-choice judgment of its orientation by selecting the appropriate response bar. On all trials, one of the Gabor stimuli had the same orientation as the Gabor in the same position on the previous trial. We explored whether continuity in orientation and position affected choice and accuracy. Results show that continuity of orientation leads to a persistent (up to four back) accuracy advantage and a higher preference in the selection of stimuli with the same orientation, and this advantage accumulates over trials. In contrast, analysis of the continuity of the selected position indicated that participants had a strong tendency to choose stimuli in the same position, but this behavior did not lead to an improvement in accuracy.

Attractive and repulsive effects of sensory history concurrently shape visual perception

Background

Sequential effects of environmental stimuli are ubiquitous in most behavioral tasks involving magnitude estimation, memory, decision making, and emotion. The human visual system exploits continuity in the visual environment, which induces two contrasting perceptual phenomena shaping visual perception. Previous work reported that perceptual estimation of a stimulus may be influenced either by attractive serial dependencies or repulsive aftereffects, with a number of experimental variables suggested as factors determining the direction and magnitude of sequential effects. Recent studies have theorized that these two effects concurrently arise in perceptual processing, but empirical evidence that directly supports this hypothesis is lacking, and it remains unclear whether and how attractive and repulsive sequential effects interact in a trial. Here we show that the two effects concurrently modulate estimation behavior in a typical sequence of perceptual tasks.

Results

We first demonstrate that observers’ estimation error as a function of both the previous stimulus and response cannot be fully described by either attractive or repulsive bias but is instead well captured by a summation of repulsion from the previous stimulus and attraction toward the previous response. We then reveal that the repulsive bias is centered on the observer’s sensory encoding of the previous stimulus, which is again repelled away from its own preceding trial, whereas the attractive bias is centered precisely on the previous response, which is the observer’s best prediction about the incoming stimuli.

Conclusions

Our findings provide strong evidence that sensory encoding is shaped by dynamic tuning of the system to the past stimuli, inducing repulsive aftereffects, and followed by inference incorporating the prediction from the past estimation, leading to attractive serial dependence.