Serial Dependence in Perception

Much evidence has shown that perception is biased towards previously presented similar stimuli, an effect recently termed serial dependence. Serial dependence affects nearly every aspect of perception, often causing gross perceptual distortions, especially for weak and ambiguous stimuli. Despite unwanted side-effects, empirical evidence and Bayesian modeling show that serial dependence acts to improve efficiency and is generally beneficial to the system. Consistent with models of predictive coding, the Bayesian priors of serial dependence are generated at high levels of cortical analysis, incorporating much perceptual experience, but feed back to lower sensory areas. These feedback loops may drive oscillations in the alpha range, linked strongly with serial dependence. The discovery of top-down predictive perceptual processes is not new, but the new, more quantitative approach characterizing serial dependence promises to lead to a deeper understanding of predictive perceptual processes and their underlying neural mechanisms.

Serial dependence in timing at the perceptual level being modulated by working memory

Recent experiences bias the perception of following stimuli, as has been verified in various kinds of experiments in visual perception. This phenomenon, known as serial dependence, may reflect mechanisms to maintain perceptual stability. In the current study, we examined several key properties of serial dependence in temporal perception. Firstly, we examined the source of the serial dependence effect in temporal perception. We found that perception without motor reproduction is sufficient to induce the sequential effect; motor reproduction caused a stronger effect and is achieved by biasing the perception of the future target duration rather than directly influencing the subsequent movement. Secondly, we ask how working memory influences serial dependence in a temporal reproduction task. By varying the delay time between standard duration and the reproduction, we showed that the strength of serial dependence is enhanced as the delay increased. Those features of serial dependence are consistent with what has been observed in visual perceptual tasks, for example, orientation perception or location perception. The similarities between the visual and the timing tasks may suggest a similar neural coding mechanism of magnitude between the visual stimuli and the duration.

Serial dependence bias can predict the overall estimation error in visual perception

Although visual feature estimations are accurate and precise, overall estimation errors (i.e., the difference between estimates and actual values) tend to show systematic patterns. For example, estimates of orientations are systematically biased away from horizontal and vertical orientations, showing an oblique illusion. Additionally, many recent studies have demonstrated that estimations of current visual features are systematically biased toward previously seen features, showing a serial dependence. However, no study examined whether the overall estimation errors were correlated with the serial dependence bias. To address this question, we enrolled three groups of participants to estimate orientation, motion speed, and point-light-walker direction. The results showed that the serial dependence bias explained over 20% of overall estimation errors in the three tasks, indicating that we could use the serial dependence bias to predict the overall estimation errors. The current study first demonstrated that the serial dependence bias was not independent from the overall estimation errors. This finding could inspire researchers to investigate the neural bases underlying the visual feature estimation and serial dependence.

The effects of visual distractors on serial dependence

Attractive serial dependence occurs when perceptual decisions are attracted toward previous stimuli. This effect is mediated by spatial attention and is most likely to occur when similar stimuli are attended at nearby locations. Attention, however, also involves the suppression of distracting information and of spatial locations where distracting stimuli have frequently appeared. Although distractors form an integral part of our visual experience, how they affect the processing of subsequent stimuli is unknown. Here, in two experiments, we tested serial dependence from distractor stimuli during an orientation adjustment task. We interleaved adjustment trials with a discrimination task requiring observers to ignore a peripheral distractor randomly appearing on half of the trials. Distractors were either similar to the adjustment probe (Experiment 1) or differed in spatial frequency and contrast (Experiment 2) and were shown at predictable or random locations in separate blocks. The results showed that the distractor caused considerable attentional capture in the discrimination task, with observers likely using proactive strategies to anticipate distractors at predictable locations. However, there was no evidence that the distractors affected the perceptual stream leading to positive serial dependence. Instead, they left a weak repulsive trace in Experiment 1 and more generally interfered with the effect of the previous adjustment probe in the serial dependence task. We suggest that this repulsive bias may reflect the operation of mechanisms involved in attentional suppression.

Serial dependencies between form orientation and motion direction are asymmetric

Much work has been done to uncover the mechanisms underlying form and motion information integration. However, no study examined the symmetry of the integration of form and motion across the temporal domain (i.e., serial dependence). In Experiment 1, we presented form and motion displays sequentially. In the form displays, dot pairs were oriented toward one screen position, indicating the form orientation; in the motion displays, dots moved radially outward. Their motion trajectories were oriented toward one screen position, indicating the motion direction. In each trial, participants reported their perceived form orientation after the form display or their perceived motion direction after the motion display. We found that the current trial's perceived motion direction was biased toward the previous trial's form orientation and vice versa, indicating serial dependencies between form orientation and motion direction. In Experiment 2, we changed the form and motion displays' reliability by varying the two displays' dot densities. The results showed that the serial dependence of form orientation on motion direction perception decreased only with increasing the current motion display's reliability; neither the reliability of the previous motion display nor that of the current form display significantly affected the serial dependence of motion direction on form orientation perception. Hence, serial dependencies between form orientation and motion direction were asymmetric. Our across-temporal integrations between form and motion, together with the simultaneous integration of form and motion revealed in the previous studies, depict a comprehensive mechanism underlying the integration of the two pieces of information.

Task feedback suggests a post-perceptual component to serial dependence

Decisions across a range of perceptual tasks are biased toward past stimuli. Such serial dependence is thought to be an adaptive low-level mechanism that promotes perceptual stability across time. However, recent studies suggest post-perceptual mechanisms may also contribute to serially biased responses, calling into question a single locus of serial dependence and the nature of integration of past and present sensory inputs. We measured serial dependence in the context of a three-dimensional (3D) motion perception task where uncertainty in the sensory information varied substantially from trial to trial. We found that serial dependence varied with stimulus properties that impact sensory uncertainty on the current trial. Reduced stimulus contrast was associated with an increased bias toward the stimulus direction of the previous trial. Critically, performance feedback, which reduced sensory uncertainty, abolished serial dependence. These results provide clear evidence for a post-perceptual locus of serial dependence in 3D motion perception and support the role of serial dependence as a response strategy in the face of substantial sensory uncertainty.

Stable perceptual phenotype of the magnitude of history biases even in the face of global task complexity

According to a Bayesian framework, visual perception requires active interpretation of noisy sensory signals in light of prior information. One such mechanism, serial dependence, is thought to promote perceptual stability by assimilating current percepts with recent stimulus history. Combining a delayed orientation-adjustment paradigm with predictable (study 1) or unpredictable (study 2) task structure, we test two key predictions of this account in a novel context: first, that serial dependence should persist even in variable environments, and, second, that, within a given observer and context, this behavioral bias should be stable from one occasion to the next. Relying on data of 41 human volunteers and two separate experimental sessions, we confirm both hypotheses. Group-level, attractive serial dependence remained strong even in the face of volatile settings with multiple, unpredictable types of tasks, and, despite considerable interindividual variability, within-subject patterns of attractive and repulsive stimulus-history biases were highly stable from one experimental session to the next. In line with the hypothesized functional role of serial dependence, we propose that, together with previous work, our findings suggest the existence of a more general individual-specific fingerprint with which the past shapes current perception. Congruent with the Bayesian account, interindividual differences may then result from differential weighting of sensory evidence and prior information.

Serial dependence in visual perception: A meta-analysis and review

Positive sequential dependencies are phenomena in which actions, perception, decisions, and memory of features or objects are systematically biased toward visual experiences from the recent past. Among many labels, serial dependencies have been referred to as priming, sequential dependencies, sequential effects, or serial effects. Despite extensive research on the topic, the field still lacks an operational definition of what counts as serial dependence. In this meta-analysis, we review the vast literature on serial dependence and quantitatively assess its key diagnostic characteristics across several different domains of visual perception. The meta-analyses fully characterize serial dependence in orientation, face, and numerosity perception. They show that serial dependence is defined by four main kinds of tuning: serial dependence decays with time (temporal-tuning), it depends on relative spatial location (spatial-tuning), it occurs only between similar features and objects (feature-tuning), and it is modulated by attention (attentional-tuning). We also review studies of serial dependence that report single observer data, highlighting the importance of individual differences in serial dependence. Finally, we discuss a range of outstanding questions and novel research avenues that are prompted by the meta-analyses. Together, the meta-analyses provide a full characterization of serial dependence as an operationally defined family of visual phenomena, and they outline several of the key diagnostic criteria for serial dependence that should serve as guideposts for future research.

Serial dependencies in visual stability during self-motion

Every time we move our head, the brain must decide whether the displacement of the visual scene is the result of external or self-produced motion. Gaze shifts generate the biggest and most frequent disturbance of vision. Visual stability during gaze shifts is necessary for both, dissociating self-produced from external motion and retaining bodily balance. Here, we asked participants to perform an eye-head gaze shift to a target that was briefly presented in a head-mounted display. We manipulated the velocity of the scene displacement across trials such that the background moved either too fast or too slow in relation to the head movement speed. Participants were required to report whether they perceived the gaze-contingent visual motion as faster or slower than what they would expect from their head movement velocity. We found that the point of visual stability was attracted to the velocity presented in the previous trial. Our data reveal that serial dependencies in visual stability calibrate the mapping between motor-related signals coding head movement velocity and visual motion velocity. This process is likely to aid in visual stability as the accuracy of this mapping is crucial to maintain visual stability during self-motion.NEW & NOTEWORTHY We report that visual stability during self-motion is maintained by serial dependencies between the current and the previous gaze-contingent visual velocity that was experienced during a head movement. The gaze-contingent scene displacement velocity that appears normal to us thus depends on what we have registered in the recent history of gaze shifts. Serial dependencies provide an efficient means to maintain visual stability during self-motion.

Serial dependence improves performance and biases confidence-based decisions

Perception depends on both the current sensory input and on the preceding stimuli history, a mechanism referred to as serial dependence (SD). One interesting, and somewhat controversial, question is whether serial dependence originates at the perceptual stage, which should lead to a sensory improvement, or at a subsequent decisional stage, causing solely a bias. Here, we studied the effects of SD in a novel manner by leveraging on the human capacity to spontaneously assess the quality of sensory information. Two noisy-oriented Gabor stimuli were simultaneously presented along with two bars of the same orientation as the Gabor stimuli. Participants were asked to choose which Gabor stimulus to judge and then make a forced-choice judgment of its orientation by selecting the appropriate response bar. On all trials, one of the Gabor stimuli had the same orientation as the Gabor in the same position on the previous trial. We explored whether continuity in orientation and position affected choice and accuracy. Results show that continuity of orientation leads to a persistent (up to four back) accuracy advantage and a higher preference in the selection of stimuli with the same orientation, and this advantage accumulates over trials. In contrast, analysis of the continuity of the selected position indicated that participants had a strong tendency to choose stimuli in the same position, but this behavior did not lead to an improvement in accuracy.

Stimulation along the anterior-posterior axis of lateral frontal cortex reduces visual serial dependence

Serial dependence is an attractive pull that recent perceptual history exerts on current judgments. Theory suggests that this bias is due to a form of short-term plasticity prevalent specifically in the frontal lobe. We sought to test the importance of the frontal lobe to serial dependence by disrupting neural activity along its lateral surface during two tasks with distinct perceptual and motor demands. In our first experiment, stimulation of the lateral prefrontal cortex (LPFC) during an oculomotor delayed response task decreased serial dependence only in the first saccade to the target, whereas stimulation posterior to the LPFC decreased serial dependence only in adjustments to eye position after the first saccade. In our second experiment, which used an orientation discrimination task, stimulation anterior to, in, and posterior to the LPFC all caused equivalent decreases in serial dependence. In this experiment, serial dependence occurred only between stimuli at the same location; an alternation bias was observed across hemifields. Frontal stimulation had no effect on the alternation bias. Transcranial magnetic stimulation to parietal cortex had no effect on serial dependence in either experiment. In summary, our experiments provide evidence for both functional differentiation (Experiment 1) and redundancy (Experiment 2) in frontal cortex with respect to serial dependence.

Serial dependence in perception across naturalistic generative adversarial network-generated mammogram

Purpose

Human perception and decisions are biased toward previously seen stimuli. This phenomenon is known as serial dependence and has been extensively studied for the last decade. Recent evidence suggests that clinicians' judgments of mammograms might also be impacted by serial dependence. However, the stimuli used in previous psychophysical experiments on this question, consisting of artificial geometric shapes and healthy tissue backgrounds, were unrealistic. We utilized realistic and controlled generative adversarial network (GAN)-generated radiographs to mimic images that clinicians typically encounter.

Approach

Mammograms from the digital database for screening mammography (DDSM) were utilized to train a GAN. This pretrained GAN was then adopted to generate a large set of authentic-looking simulated mammograms: 20 circular morph continuums, each with 147 images, for a total of 2940 images. Using these stimuli in a standard serial dependence experiment, participants viewed a random GAN-generated mammogram on each trial and subsequently matched the GAN-generated mammogram encountered using a continuous report. The characteristics of serial dependence from each continuum were analyzed.

Results

We found that serial dependence affected the perception of all naturalistic GAN-generated mammogram morph continuums. In all cases, the perceptual judgments of GAN-generated mammograms were biased toward previously encountered GAN-generated mammograms. On average, perceptual decisions had 7% categorization errors that were pulled in the direction of serial dependence.

Conclusions

Serial dependence was found even in the perception of naturalistic GAN-generated mammograms created by a GAN. This supports the idea that serial dependence could, in principle, contribute to decision errors in medical image perception tasks.

Attractive and repulsive serial dependence: The role of task relevance, the passage of time, and the number of stimuli

Visual decisions are attracted toward features of previous stimuli. This phenomenon, termed serial dependence, has been related to a mechanism that integrates present visual input with stimuli seen up to 10 to 15 s in the past. It is believed that this mechanism is "temporally tuned" and the effect of prior stimuli fades with time. Here, we investigated whether the temporal window of serial dependence is influenced by the number of stimuli shown. Observers performed an orientation adjustment task where the interval between the past and present stimulus and the number of intervening stimuli varied. First, we found that the direction-repulsive or attractive-and duration of the effect of a past stimulus depends on whether the past stimulus was relevant to behavior. Second, we show that the number of stimuli, and not only the passage of time, plays a role: The effect of a stimulus at a fixed interval depends on the number of other stimuli shown after. Our results demonstrate that neither a single mechanism nor a general tuning window can fully capture the complexity of serial dependence.

Serial Dependence in Dermatological Judgments

Serial Dependence is a ubiquitous visual phenomenon in which sequentially viewed images appear more similar than they actually are, thus facilitating an efficient and stable perceptual experience in human observers. Although serial dependence is adaptive and beneficial in the naturally autocorrelated visual world, a smoothing perceptual experience, it might turn maladaptive in artificial circumstances, such as medical image perception tasks, where visual stimuli are randomly sequenced. Here, we analyzed 758,139 skin cancer diagnostic records from an online app, and we quantified the semantic similarity between sequential dermatology images using a computer vision model as well as human raters. We then tested whether serial dependence in perception occurs in dermatological judgments as a function of image similarity. We found significant serial dependence in perceptual discrimination judgments of lesion malignancy. Moreover, the serial dependence was tuned to the similarity in the images, and it decayed over time. The results indicate that relatively realistic store-and-forward dermatology judgments may be biased by serial dependence. These findings help in understanding one potential source of systematic bias and errors in medical image perception tasks and hint at useful approaches that could alleviate the errors due to serial dependence.

The Representational Similarity between Visual Perception and Recent Perceptual History

From moment to moment, the visual properties of objects in the world fluctuate because of external factors like ambient lighting, occlusion and eye movements, and internal (proximal) noise. Despite this variability in the incoming information, our perception is stable. Serial dependence, the behavioral attraction of current perceptual responses toward previously seen stimuli, may reveal a mechanism underlying stability: a spatiotemporally tuned operator that smooths over spurious fluctuations. The current study examined the neural underpinnings of serial dependence by recording the electroencephalographic (EEG) brain response of female and male human observers to prototypical objects (faces, cars, and houses) and morphs that mixed properties of two prototypes. Behavior was biased toward previously seen objects. Representational similarity analysis (RSA) revealed that responses evoked by visual objects contained information about the previous stimulus. The trace of previous representations in the response to the current object occurred immediately on object appearance, suggesting that serial dependence arises from a brain state or set that precedes processing of new input. However, the brain response to current visual objects was not representationally similar to the trace they leave on subsequent object representations. These results reveal that while past stimulus history influences current representations, this influence does not imply a shared neural code between the previous trial (memory) and the current trial (perception).SIGNIFICANCE STATEMENT The perception of visual objects is pulled toward instances of that object seen in the recent past. The neural underpinnings of this serial dependence remain to be fully investigated. The present study examined electroencephalographic (EEG) responses to faces, cars, and houses, and ambiguous between-category morphs. With representational similarity analysis (RSA), we showed (1) object-specific neural patterns that differentiate the three categories; (2) that the response to the current object contains information about the previous object, mirroring behavioral serial dependence; (3) that the object-specific neural pattern about the past was different from that in the current response, revealing that while past stimulus history influences current representations, this does not imply a shared neural code between the previous trial (memory) and the current trial (perception).

The influence of the tested item on serial dependence in perceptual decisions

Serial dependence in vision reflects how perceptual decisions can be biased by what we have recently perceived. Serial dependence studies test single items' effects on perceptual decisions. However, our visual world contains multiple objects at any given moment, so it's important to understand how past experiences affect not only a single object but also perception in a more general sense. Here we asked the question: What effect does a single item have when there is more than one subsequently presented test item? We displayed a single line (inducer) at the screen center, then either a single test-line or two simultaneous test-lines, varying in orientation space to the inducer. Next, participants reported test-line orientation using a left or right located response circle (to indicate which test-line should be reported). The results demonstrated that the inducer influenced subsequent perceptual judgments of a test-line even when two test-lines were presented. Distant items caused repulsive serial dependence, while close items caused attractive serial dependence. This shows how a single inducer can influence test-line judgments, even when two test-lines are presented, and can produce attractive and repulsive serial dependence biases when the item to report is revealed after it has disappeared.

Effects of satisfying and violating expectations on serial dependence

Serial dependence refers to the phenomenon that observers tend to report stimuli as being more similar to previous stimuli than they really are (attractive dependence) or, in some cases, as more different than they really are (repulsive dependence). Numerous experiments have demonstrated serial dependence for a range of modalities and stimulus features, highlighting the role of bottom-up sensory interactions. However, comparatively less research has focused on how higher-level cognitive factors, such as expectations, might influence serial dependence. Here, we manipulated expectations by having observers respond to target luminance gratings that occurred at the end of a sequence of non-target gratings. The sequence either rotated predictably (inducing an expectation), varied randomly (inducing no expectation), or rotated predictably but had a random target orientation (violating expectations). We found that observers produced less errors and indicated less uncertainty in their estimations of expected stimuli but their responses were biased away from the penultimate stimulus in the sequence (repulsive dependence). In contrast, following random sequences, responses showed an attractive bias to the penultimate stimulus in the sequence. Unexpected targets showed a mixture of both biases, such that when targets happened (by chance) to appear as expected, responses were repulsed, but responses to target orientations that more clearly violated expectations were attracted. These results indicate that, whereas attraction to previous stimuli may be a default strategy employed in response to random and unexpected events, certain expectations can reverse the default bias into a repulsive one.

Serial dependence in visual perception: A review

How does the visual system represent continuity in the constantly changing visual input? A recent proposal is that vision is serially dependent: Stimuli seen a moment ago influence what we perceive in the present. In line with this, recent frameworks suggest that the visual system anticipates whether an object seen at one moment is the same as the one seen a moment ago, binding visual representations across consecutive perceptual episodes. A growing body of work supports this view, revealing signatures of serial dependence in many diverse visual tasks. Yet, the variety of disparate findings and interpretations calls for a more general picture. Here, we survey the main paradigms and results over the past decade. We also focus on the challenge of finding a relationship between serial dependence and the concept of "object identity," taking centuries-long history of research into account. Among the seemingly contrasting findings on serial dependence, we highlight common patterns that may elucidate the nature of this phenomenon and attempt to identify questions that are unanswered.

Evidence of Serial Dependence from Decoding of Visual Evoked Potentials

It is well known that recent sensory experience influences perception, recently demonstrated by a phenomenon termed "serial dependence." However, its underlying neural mechanisms are poorly understood. We measured ERP responses to pairs of stimuli presented randomly to the left or right hemifield. Seventeen male and female adults judged whether the upper or lower half of the grating had higher spatial frequency, independent of the horizontal position of the grating. This design allowed us to trace the memory signal modulating task performance and also the implicit memory signal associated with hemispheric position. Using classification techniques, we decoded the position of the current and previous stimuli and the response from voltage scalp distributions of the current trial. Classification of previous responses reached full significance only 700 ms after presentation of the current stimulus, consistent with retrieval of an activity-silent memory trace. Cross-condition classification accuracy of past responses (trained on current responses) correlated with the strength of serial dependence effects of individual participants. Overall, our data provide evidence for a silent memory signal that can be decoded from the EEG potential, which interacts with the neural processing of the current stimulus. This silent memory signal could be the physiological substrate subserving at least one type of serial dependence.SIGNIFICANCE STATEMENT The neurophysiological underpinnings of how past perceptual experience affects current perception are poorly understood. Here, we show that recent experience is reactivated when a new stimulus is presented and that the strength of this reactivation correlates with serial biases in individual participants, suggesting that serial dependence is established on the basis of a silent memory signal.

Causal evidence for the higher-order origin of serial dependence suggests a multi-area account

There has been recent interest in whether serial dependence, a temporal bias in working memory and perception, is generated by higher-order cognitive or sensory areas. Based on the literature suggesting prefrontal areas combined with a recent article by de Azevedo Neto & Bartels (1), providing causal evidence for serial dependence relying on premotor cortex but not the visual area hV5/MT+, I here argue for a higher-order and multi-area origin of serial dependence.

Serial dependence for oculomotor control depends on early sensory signals

To create an accurate percept of the world, the visual system relies on past experience and prior assumptions.¹ For example, although the retinal projection of an object moving in depth changes drastically, we still perceive the object at a constant size and velocity.^2,3 Consequently, if we see the same object with a constant retinal size at two different depth levels, the perceived size differs (illustrated by the Ponzo illusion). Past experience also directly influences perceptual judgments, an effect known as serial dependence.^4,5 Such sequential effects have also been reported for oculomotor behavior, even on the trial-by-trial level.^6-10 An integration of past experiences seems like a smart and sophisticated mechanism to reduce uncertainty and improve behavior in a world full of statistical regularities. By leveraging the Ponzo illusion to dissociate perceived size and speed from retinal signals, we show that serial-dependence effects for oculomotor control are mediated by retinal error signals. These sequential effects likely take place in early sensory processing because they transfer to different visual stimuli. In contrast to recently reported history effects for perceptual decisions,¹¹ sequential effects for oculomotor control deviate from perceptual mechanisms by not integrating spatial context and by ignoring size and velocity constancy. Although this dissociation might appear suboptimal, we argue that this effect reveals the different goals of the oculomotor and perceptual systems. The oculomotor system tries to reduce retinal error signals to bring and keep the target close to the fovea, whereas the visual system interprets retinal input to achieve an accurate representation of the world.¹².

Disrupting Short-Term Memory Maintenance in Premotor Cortex Affects Serial Dependence in Visuomotor Integration

Human behavior is biased by past experience. For example, when intercepting a moving target, the speed of previous targets will bias responses in future trials. Neural mechanisms underlying this so-called serial dependence are still under debate. Here, we tested the hypothesis that the previous trial leaves a neural trace in brain regions associated with encoding task-relevant information in visual and/or motor regions. We reasoned that injecting noise by means of transcranial magnetic stimulation (TMS) over premotor and visual areas would degrade such memory traces and hence reduce serial dependence. To test this hypothesis, we applied bursts of TMS pulses to right visual motion processing region hV5/MT+ and to left dorsal premotor cortex (PMd) during intertrial intervals of a coincident timing task performed by twenty healthy human participants (15 female). Without TMS, participants presented a bias toward the speed of the previous trial when intercepting moving targets. TMS over PMd decreased serial dependence in comparison to the control Vertex stimulation, whereas TMS applied over hV5/MT+ did not. In addition, TMS seems to have specifically affected the memory trace that leads to serial dependence, as we found no evidence that participants' behavior worsened after applying TMS. These results provide causal evidence that an implicit short-term memory mechanism in premotor cortex keeps information from one trial to the next, and that this information is blended with current trial information so that it biases behavior in a visuomotor integration task with moving objects.SIGNIFICANCE STATEMENT Human perception and action are biased by the recent past. The origin of such serial bias is still not fully understood, but a few components seem to be fundamental for its emergence: the brain needs to keep previous trial information in short-term memory and blend it with incoming information. Here, we present evidence that a premotor area has a potential role in storing previous trial information in short-term memory in a visuomotor task and that this information is responsible for biasing ongoing behavior. These results corroborate the perspective that areas associated with processing information of a stimulus or task also participate in maintaining that information in short-term memory even when this information is no longer relevant for current behavior.

Perceptual History Acts in World-Centred Coordinates

Serial dependence effects have been observed using a variety of stimuli and tasks, revealing that the recent past can bias current percepts, leading to increased similarity between two. The aim of this study is to determine whether this temporal integration occurs in egocentric or allocentric coordinates. We asked participants to perform an orientation reproduction task using grating stimuli while the head was kept at a fixed position, or after a 40° yaw rotation between trials, from left (-20°) to right (+20°), putting the egocentric and allocentric cues in conflict. Under these conditions, allocentric cues prevailed.

Serial dependence revealed in history-dependent perceptual templates

In any given perceptual task, the visual system selectively weighs or filters incoming information. The particular set of weights or filters form a kind of template, which reveals the regions or types of information that are particularly useful for a given perceptual decision.1,2 Unfortunately, sensory input is noisy and ever changing. To compensate for these fluctuations, the visual system could adopt a strategy of biasing the templates such that they reflect a temporal smoothing of input, which would be a form of serial dependence.3-5 Here, we demonstrate that perceptual templates are, in fact, altered by serial dependence. Using a simple orientation detection task and classification-image technique, we found that perceptual templates are systematically biased toward previously seen, task-irrelevant orientations. The results of an orientation discrimination task suggest that this shift in perceptual template derives from a change in the perceptual appearance of orientation. Our study reveals how serial dependence biases internal templates of orientation and suggests that the sensitivity of classification-image techniques in general could be improved by taking into account history-dependent fluctuations in templates.

A shared mechanism for facial expression in human faces and face pareidolia

Facial expressions are vital for social communication, yet the underlying mechanisms are still being discovered. Illusory faces perceived in objects (face pareidolia) are errors of face detection that share some neural mechanisms with human face processing. However, it is unknown whether expression in illusory faces engages the same mechanisms as human faces. Here, using a serial dependence paradigm, we investigated whether illusory and human faces share a common expression mechanism. First, we found that images of face pareidolia are reliably rated for expression, within and between observers, despite varying greatly in visual features. Second, they exhibit positive serial dependence for perceived facial expression, meaning an illusory face (happy or angry) is perceived as more similar in expression to the preceding one, just as seen for human faces. This suggests illusory and human faces engage similar mechanisms of temporal continuity. Third, we found robust cross-domain serial dependence of perceived expression between illusory and human faces when they were interleaved, with serial effects larger when illusory faces preceded human faces than the reverse. Together, the results support a shared mechanism for facial expression between human faces and illusory faces and suggest that expression processing is not tightly bound to human facial features.

Increased influence of prior choices on perceptual decisions in autism

Autism spectrum disorder (ASD) manifests sensory and perceptual atypicalities. Recent theories suggest that these may reflect a reduced influence of prior information in ASD. Some studies have found reduced adaptation to recent sensory stimuli in ASD. However, the effects of prior stimuli and prior perceptual choices can counteract one-another. Here, we investigated this using two different tasks (in two different cohorts): (i) visual location discrimination and (ii) multisensory (visual-vestibular) heading discrimination. We fit the data using a logistic regression model to dissociate the specific effects of prior stimuli and prior choices. In both tasks, perceptual decisions were biased toward recent choices. Notably, the 'attractive' effect of prior choices was significantly larger in ASD (in both tasks and cohorts), while there was no difference in the influence of prior stimuli. These results challenge theories of reduced priors in ASD, and rather suggest an increased consistency bias for perceptual decisions in ASD.

Analyzing categorical time series in the presence of missing observations

In real applications, time series often exhibit missing observations such that standard analytical tools cannot be applied. While there are approaches of how to handle missing data in quantitative time series, the case of categorical time series seems not to have been treated so far. Both for the case of ordinal and nominal time series, solutions are developed that allow to analyze their marginal and serial properties in the presence of missing observations. This is achieved by adapting the concept of amplitude modulation, which allows to obtain closed-form asymptotic expressions for the derived statistics' distribution (assuming that missingness happens independently of the actual process). The proposed methods are investigated with simulations, and they are applied in a project on migraine patients, where the monitored qualitative time series are often incomplete.

Perceptual history propagates down to early levels of sensory analysis

One function of perceptual systems is to construct and maintain a reliable representation of the environment. A useful strategy intrinsic to modern “Bayesian” theories of perception1, 2, 3, 4, 5, 6 is to take advantage of the relative stability of the input and use perceptual history (priors) to predict current perception. This strategy is efficient1, 2, 3, 4, 5, 6, 7 but can lead to stimuli being biased toward perceptual history, clearly revealed in a phenomenon known as serial dependence.8, 9, 10, 11, 12, 13, 14 However, it is still unclear whether serial dependence biases sensory encoding or only perceptual decisions.^15,16 We leveraged on the “surround tilt illusion”—where tilted flanking stimuli strongly bias perceived orientation—to measure its influence on the pattern of serial dependence, which is typically maximal for similar orientations of past and present stimuli.^7,10 Maximal serial dependence for a neutral stimulus preceded by an illusory one occurred when the perceived, not the physical, orientations of the two stimuli matched, suggesting that the priors biasing current perception incorporate the effect of the illusion. However, maximal serial dependence of illusory stimuli induced by neutral stimuli occurred when their physical (not perceived) orientations were matched, suggesting that priors interact with incoming sensory signals before they are biased by flanking stimuli. The evidence suggests that priors are high-level constructs incorporating contextual information, which interact directly with early sensory signals, not with highly processed perceptual representations.

•

Perception is heavily biased by perceptual history and expectations

•

Perceptual history includes illusory effects driven by spatial context

•

This representation propagates back to sensory areas preceding context effects

•

The results point to a neural architecture consistent with predictive coding

A Bayesian and efficient observer model explains concurrent attractive and repulsive history biases in visual perception

Human perceptual decisions can be repelled away from (repulsive adaptation) or attracted towards recent visual experience (attractive serial dependence). It is currently unclear whether and how these repulsive and attractive biases interact during visual processing and what computational principles underlie these history dependencies. Here we disentangle repulsive and attractive biases by exploring their respective timescales. We find that perceptual decisions are concurrently attracted towards the short-term perceptual history and repelled from stimuli experienced up to minutes into the past. The temporal pattern of short-term attraction and long-term repulsion cannot be captured by an ideal Bayesian observer model alone. Instead, it is well captured by an ideal observer model with efficient encoding and Bayesian decoding of visual information in a slowly changing environment. Concurrent attractive and repulsive history biases in perceptual decisions may thus be the consequence of the need for visual processing to simultaneously satisfy constraints of efficiency and stability.

Auditory Perceptual History Is Propagated through Alpha Oscillations

Perception is a proactive, “predictive” process, in which the brain relies, at least in part, on accumulated experience to make best guesses about the world to test against sensory data, updating the guesses as new experience is acquired. Using novel behavioral methods, the present study demonstrates the role of alpha rhythms in communicating past perceptual experience. Participants were required to discriminate the ear of origin of brief sinusoidal tones that were presented monaurally at random times within a burst of uncorrelated dichotic white noise masks. Performance was not constant but varied with delay after noise onset in an oscillatory manner at about 9 Hz (alpha rhythm). Importantly, oscillations occurred only for trials preceded by a target tone to the same ear, either on the previous trial or two trials back. These results suggest that communication of perceptual history generates neural oscillations within specific perceptual circuits, strongly implicating behavioral oscillations in predictive perception and with formation of working memory.

•

We demonstrate the role of alpha rhythms in the propagation of perceptual history

•

Auditory decisions were rhythmically biased by stimuli presented 1 or 2 trials back

•

Bias oscillated at ∼9 Hz only when successive stimuli occurred in the same ear

•

Alpha is strongly implicated in predictive perception and working memory formation

An Assessment of Computer-Generated Stimuli for Use in Studies of Body Size Estimation and Bias

Inaccurate body size judgments are associated with body image disturbances, a clinical feature of many eating disorders. Accordingly, body-related stimuli have become increasingly important in the study of estimation inaccuracies and body image disturbances. Technological advancements in the last decade have led to an increased use of computer-generated (CG) body stimuli in body image research. However, recent face perception research has suggested that CG face stimuli are not recognized as readily and may not fully tap facial processing mechanisms. The current study assessed the effectiveness of using CG stimuli in an established body size estimation task (the “bodyline” task). Specifically, we examined whether employing CG body stimuli alters body size judgments and associated estimation biases. One hundred and six 17- to 25-year-old females completed the CG bodyline task, which involved estimating the size of full-length CG body stimuli along a visual analogue scale. Our results show that perception of body size for CG stimuli was non-linear. Participants struggled to discriminate between extreme bodies sizes and overestimated the size change between near to average bodies. Furthermore, one of our measured size estimation biases was larger for CG stimuli. Our collective findings suggest using caution when employing CG stimuli in experimental research on body perception.

Probabilistic Representation in Human Visual Cortex Reflects Uncertainty in Serial Decisions

How does the brain represent the reliability of its sensory evidence? Here, we test whether sensory uncertainty is encoded in cortical population activity as the width of a probability distribution, a hypothesis that lies at the heart of Bayesian models of neural coding. We probe the neural representation of uncertainty by capitalizing on a well-known behavioral bias called serial dependence. Human observers of either sex reported the orientation of stimuli presented in sequence, while activity in visual cortex was measured with fMRI. We decoded probability distributions from population-level activity and found that serial dependence effects in behavior are consistent with a statistically advantageous sensory integration strategy, in which uncertain sensory information is given less weight. More fundamentally, our results suggest that probability distributions decoded from human visual cortex reflect the sensory uncertainty that observers rely on in their decisions, providing critical evidence for Bayesian theories of perception.SIGNIFICANCE STATEMENT Virtually any decision that people make is based on uncertain and incomplete information. Although uncertainty plays a major role in decision-making, we have but a nascent understanding of its neural basis. Here, we probe the neural code of uncertainty by capitalizing on a well-known perceptual illusion. We developed a computational model to explain the illusion, and tested it in behavioral and neuroimaging experiments. This revealed that the illusion is not a mistake of perception, but rather reflects a rational decision under uncertainty. No less important, we discovered that the uncertainty that people use in this decision is represented in brain activity as the width of a probability distribution, providing critical evidence for current Bayesian theories of decision-making.

Working Memory Maintenance Modulates Serial Dependence Effects of Perceived Emotional Expression

The stability of face perception is vital in interpersonal interactions. Recent studies have revealed the mechanism of the stability in the perception of stable attributes of faces (such as facial identity) by serial dependence, a phenomenon in which perception of current stimuli is pulled toward recently viewed stimuli. However, whether serial dependence of perceived emotional expression (a changeable attribute of faces) exists remains controversial, and its exact nature has not been examined yet. To address these issues, we used the methods of constant stimuli and two-interval forced choice tasks in three psychophysical experiments. Participants compared two successive facial expressions selected from a continuum with 50 morphed faces ranging from sad to happy. Experiment 1a and 1b showed that a perceived facial expression pulled toward previously seen facial expressions (i.e., a significant serial dependence effect), independent of response instructions. Furthermore, a stronger serial dependence effect was found when the first facial expression was retained in working memory for a longer delay duration (Experiment 2), and yet a weaker serial dependence effect was observed when a longer delay between decision and response was performed (Experiment 3). These findings indicate that serial dependence facilitates the stability of facial expression perception and is modulated by working memory representations.

Reactivation of Previous Experiences in a Working Memory Task

Recent experiences influence the processing of new information even when those experiences are irrelevant to the current task. Does this reflect the indirect effects of a passively maintained representation of the previous experience, or is this representation reactivated when a new event occurs? To answer this question, we attempted to decode the orientation of the stimulus on the previous trial from the electroencephalogram on the current trial in a working memory task. Behavioral data confirmed that the previous-trial stimulus orientation influenced the reported orientation on the current trial, even though the previous-trial orientation was now task irrelevant. In two independent experiments, we found that the previous-trial orientation could be decoded from the current-trial electroencephalogram, indicating that the current-trial stimulus reactivated or boosted the representation of the previous-trial orientation. These results suggest that the effects of recent experiences on behavior are driven, in part, by a reactivation of those experiences and not solely by the indirect effects of passive memory traces.

Multilevel model with random effects for clustered survival data with multiple failure outcomes

We present a multilevel frailty model for handling serial dependence and simultaneous heterogeneity in survival data with a multilevel structure attributed to clustering of subjects and the presence of multiple failure outcomes. One commonly observes such data, for example, in multi-institutional, randomized placebo-controlled trials in which patients suffer repeated episodes (eg, recurrent migraines) of the disease outcome being measured. The model extends the proportional hazards model by incorporating a random covariate and unobservable random institution effect to respectively account for treatment-by-institution interaction and institutional variation in the baseline risk. Moreover, a random effect term with correlation structure driven by a first-order autoregressive process is attached to the model to facilitate estimation of between patient heterogeneity and serial dependence. By means of the generalized linear mixed model methodology, the random effects distribution is assumed normal and the residual maximum likelihood and the maximum likelihood methods are extended for estimation of model parameters. Simulation studies are carried out to evaluate the performance of the residual maximum likelihood and the maximum likelihood estimators and to assess the impact of misspecifying random effects distribution on the proposed inference. We demonstrate the practical feasibility of the modeling methodology by analyzing real data from a double-blind randomized multi-institutional clinical trial, designed to examine the effect of rhDNase on the occurrence of respiratory exacerbations among patients with cystic fibrosis.

The functional role of serial dependence

The world tends to be stable from moment to moment, leading to strong serial correlations in natural scenes. As similar stimuli usually require similar behavioural responses, it is highly likely that the brain has developed strategies to leverage these regularities. A good deal of recent psychophysical evidence is beginning to show that the brain is sensitive to serial correlations, causing strong drifts in observer responses towards previously seen stimuli. However, it is still not clear that this tendency leads to a functional advantage. Here, we test a formal model of optimal serial dependence and show that as predicted, serial dependence in an orientation reproduction task is dependent on current stimulus reliability, with less precise stimuli, such as low spatial frequency oblique Gabors, exhibiting the strongest effects. We also show that serial dependence depends on the similarity between two successive stimuli, again consistent with the behaviour of an ideal observer aiming at minimizing reproduction errors. Lastly, we show that serial dependence leads to faster response times, indicating that the benefits of serial integration go beyond reproduction error. Overall our data show that serial dependence has a beneficial role at various levels of perception, consistent with the idea that the brain exploits the temporal redundancy of the visual scene as an optimization strategy.

Two types of serial dependence in visual working memory

Stimulus representations in working memory depend on memory traces of past stimuli both from previous trials and from the current trial. However, it is unclear whether the same or different mechanisms underlie this serial dependence across and within trials. We directly contrasted estimates of bias for pairs of immediately successive stimuli across and within trials. In each trial, participants memorized two consecutive motion direction stimuli (S1 and S2) and after a short delay were cued to report one of them. We found serial dependence across trials: The current S1 was attracted towards the preceding S2 when the latter had been cued for report. In contrast, within the same trial S2 was repulsed from S1. In addition, repulsion within a trial occurred for a broader range of motion direction differences between stimuli than attraction across trials. A second experiment in which 25% of trials did not require a response demonstrated that across-trial attraction did not depend on whether the previous S2 actually had to be reported. Our findings provide evidence for two types of serial dependence operating across and within trials. They support the notion of different mechanisms integrating or segregating current from similar past memory contents depending on their task relevance.

Pairwise residuals and diagnostic tests for misspecified dependence structures in models for binary longitudinal data

Maximum likelihood estimation of models for binary longitudinal data is typically inconsistent if the dependence structure is misspecified. Unfortunately, diagnostics specifically designed for detecting such misspecifications are scant. We develop residuals and diagnostic tests based on comparing observed and expected frequencies of response patterns over time in the presence of arbitrary time-varying and time-invariant covariates. To overcome the sparseness problem, we use lower-order marginal tables, such as two-way tables for pairs of time-points, aggregated over covariate patterns. Our proposed pairwise concordance residuals are valuable for exploratory diagnostics and for constructing both generic tests for misspecified dependence structure as well as targeted adjacent pair concordance tests for excess serial dependence. The proposed methods are straightforward to implement and work well for general situations, regardless of the number of time-points and the number and types of covariates.

Serial Dependence across Perception, Attention, and Memory

Information that has been recently perceived or remembered can bias current processing. This has been viewed as both a corrupting (e.g., proactive interference in short-term memory) and stabilizing (e.g., serial dependence in perception) phenomenon. We hypothesize that this bias is a generally adaptive aspect of brain function that leads to occasionally maladaptive outcomes.

An autoregressive growth model for longitudinal item analysis

A first-order autoregressive growth model is proposed for longitudinal binary item analysis where responses to the same items are conditionally dependent across time given the latent traits. Specifically, the item response probability for a given item at a given time depends on the latent trait as well as the response to the same item at the previous time, or the lagged response. An initial conditions problem arises because there is no lagged response at the initial time period. We handle this problem by adapting solutions proposed for dynamic models in panel data econometrics. Asymptotic and finite sample power for the autoregressive parameters are investigated. The consequences of ignoring local dependence and the initial conditions problem are also examined for data simulated from a first-order autoregressive growth model. The proposed methods are applied to longitudinal data on Korean students' self-esteem.

Quantifying the effect of intertrial dependence on perceptual decisions

In the perceptual sciences, experimenters study the causal mechanisms of perceptual systems by probing observers with carefully constructed stimuli. It has long been known, however, that perceptual decisions are not only determined by the stimulus, but also by internal factors. Internal factors could lead to a statistical influence of previous stimuli and responses on the current trial, resulting in serial dependencies, which complicate the causal inference between stimulus and response. However, the majority of studies do not take serial dependencies into account, and it has been unclear how strongly they influence perceptual decisions. We hypothesize that one reason for this neglect is that there has been no reliable tool to quantify them and to correct for their effects. Here we develop a statistical method to detect, estimate, and correct for serial dependencies in behavioral data. We show that even trained psychophysical observers suffer from strong history dependence. A substantial fraction of the decision variance on difficult stimuli was independent of the stimulus but dependent on experimental history.We discuss the strong dependence of perceptual decisions on internal factors and its implications for correct data interpretation.

Time-series models for border inspection data

We propose a new modeling approach for inspection data that provides a more useful interpretation of the patterns of detections of invasive pests, using cargo inspection as a motivating example. Methods that are currently in use generally classify shipments according to their likelihood of carrying biosecurity risk material, given available historical and contextual data. Ideally, decisions regarding which cargo containers to inspect should be made in real time, and the models used should be able to focus efforts when the risk is higher. In this study, we propose a dynamic approach that treats the data as a time series in order to detect periods of high risk. A regulatory organization will respond differently to evidence of systematic problems than evidence of random problems, so testing for serial correlation is of major interest. We compare three models that account for various degrees of serial dependence within the data. First is the independence model where the prediction of the arrival of a risky shipment is made solely on the basis of contextual information. We also consider a Markov chain that allows dependence between successive observations, and a hidden Markov model that allows further dependence on past data. The predictive performance of the models is then evaluated using ROC and leakage curves. We illustrate this methodology on two sets of real inspection data.