Face familiarity promotes stable identity recognition: exploring face perception using serial dependence

Opposing serial dependencies revealed for sequences of auditory emotional stimuli

Our percept of the world is not solely determined by what we perceive and process at a given moment in time, but also depends on what we processed recently. In the present study, we investigate whether the perceived emotion of a spoken sentence is contingent upon the emotion of an auditory stimulus on the preceding trial (i.e., serial dependence). Thereto, participants were exposed to spoken sentences that varied in emotional affect by changing the prosody that ranged from 'happy' to 'fearful'. Participants were instructed to rate the emotion. We found a positive serial dependence for emotion processing whereby the perceived emotion was biased towards the emotion on the preceding trial. When we introduced 'no-go' trials (i.e., no rating was required), we found a negative serial dependence when participants knew in advance to withhold their response on a given trial (Experiment 2) and a positive serial dependence when participants received the information to withhold their response after the stimulus presentation (Experiment 3). We therefore established a robust serial dependence for emotion processing in speech and introduce a methodology to disentangle perceptual from post-perceptual processes. This approach can be applied to the vast majority of studies investigating sequential dependencies to separate positive from negative serial dependence.

Hysteresis reveals a happiness bias effect in dynamic emotion recognition from ambiguous biological motion

Considering the nonlinear dynamic nature of emotion recognition, it is believed to be strongly dependent on temporal context. This can be investigated by resorting to the phenomenon of hysteresis, which features a form of serial dependence, entailed by continuous temporal stimulus trajectories. Under positive hysteresis, the percept remains stable in visual memory (persistence) while in negative hysteresis, it shifts earlier (adaptation) to the opposite interpretation. Here, we asked whether positive or negative hysteresis occurs in emotion recognition of inherently ambiguous biological motion, while testing for the controversial debate of a negative versus positive emotional bias. Participants (n = 22) performed a psychophysical experiment in which they were asked to judge stimulus transitions between two emotions, happiness and sadness, from an actor database, and report perceived emotion across time, from one emotion to the opposite as physical cues were continuously changing. Our results reveal perceptual hysteresis in ambiguous emotion recognition, with positive hysteresis (visual persistence) predominating. However, negative hysteresis (adaptation/fatigue) was also observed in particular in the direction from sadness to happiness. This demonstrates a positive (happiness) bias in emotion recognition in ambiguous biological motion recognition. Finally, the interplay between positive and negative hysteresis suggests an underlying competition between visual persistence and adaptation mechanisms during ambiguous emotion recognition.

Stable perceptual phenotype of the magnitude of history biases even in the face of global task complexity

According to a Bayesian framework, visual perception requires active interpretation of noisy sensory signals in light of prior information. One such mechanism, serial dependence, is thought to promote perceptual stability by assimilating current percepts with recent stimulus history. Combining a delayed orientation-adjustment paradigm with predictable (study 1) or unpredictable (study 2) task structure, we test two key predictions of this account in a novel context: first, that serial dependence should persist even in variable environments, and, second, that, within a given observer and context, this behavioral bias should be stable from one occasion to the next. Relying on data of 41 human volunteers and two separate experimental sessions, we confirm both hypotheses. Group-level, attractive serial dependence remained strong even in the face of volatile settings with multiple, unpredictable types of tasks, and, despite considerable interindividual variability, within-subject patterns of attractive and repulsive stimulus-history biases were highly stable from one experimental session to the next. In line with the hypothesized functional role of serial dependence, we propose that, together with previous work, our findings suggest the existence of a more general individual-specific fingerprint with which the past shapes current perception. Congruent with the Bayesian account, interindividual differences may then result from differential weighting of sensory evidence and prior information.

Serial dependence in visual perception: A meta-analysis and review

Positive sequential dependencies are phenomena in which actions, perception, decisions, and memory of features or objects are systematically biased toward visual experiences from the recent past. Among many labels, serial dependencies have been referred to as priming, sequential dependencies, sequential effects, or serial effects. Despite extensive research on the topic, the field still lacks an operational definition of what counts as serial dependence. In this meta-analysis, we review the vast literature on serial dependence and quantitatively assess its key diagnostic characteristics across several different domains of visual perception. The meta-analyses fully characterize serial dependence in orientation, face, and numerosity perception. They show that serial dependence is defined by four main kinds of tuning: serial dependence decays with time (temporal-tuning), it depends on relative spatial location (spatial-tuning), it occurs only between similar features and objects (feature-tuning), and it is modulated by attention (attentional-tuning). We also review studies of serial dependence that report single observer data, highlighting the importance of individual differences in serial dependence. Finally, we discuss a range of outstanding questions and novel research avenues that are prompted by the meta-analyses. Together, the meta-analyses provide a full characterization of serial dependence as an operationally defined family of visual phenomena, and they outline several of the key diagnostic criteria for serial dependence that should serve as guideposts for future research.

Cross-Modality Evidence for Reduced Choice History Biases in Psychosis-Prone Individuals

OBJECTIVES

Predictive processing posits that perception emerges from inferential processes within a hierarchical cortical system. Alterations of these processes may result in psychotic experiences, such as hallucinations and delusions. Central to the predictive processing account of psychosis is the notion of aberrant weights attributed to prior information and sensory input. Based on the notion that previous perceptual choices represent a relevant source of prior information, we here asked whether the propensity towards psychotic experiences may be related to altered choice history biases in perceptual decision-making.

METHODS

We investigated the relationship between choice history biases in perceptual decision-making and psychosis proneness in the general population. Choice history biases and their adaptation to experimentally induced changes in stimulus serial dependencies were investigated in decision-making tasks with auditory (experiment 1) and visual (experiment 2) stimuli. We further explored a potential compensatory mechanism for reduced choice history biases by reliance on predictive cross-modal cues.

RESULTS

In line with our preregistered hypothesis, psychosis proneness was associated with decreased choice history biases in both experiments. This association is generalized across conditions with and without stimulus serial dependencies. We did not find consistent evidence for a compensatory reliance on cue information in psychosis-prone individuals across experiments.

CONCLUSIONS

Our results show reduced choice history biases in psychosis proneness. A compensatory mechanism between implicit choice history effects and explicit cue information is not supported unequivocally by our data.

A general serial dependence among various facial traits: Evidence from Markov Chain and derivative of Gaussian

The human vision system can extract a stable representation of the always-changing visual world. However, the mechanism underlying such perceptual continuity remains unclear. A possible candidate is the serial dependence: visual perception of an object is positively biased toward the visual input from the recent past. Does the visual system use one pattern of serial dependence for general purposes? Or different patterns of serial dependence for different visual tasks? Because different social facial traits (e.g., trustworthiness and dominance) are dissociable, it is reasonable to assume that the perception of different facial characteristics would require different patterns of serial dependences. In this study, we examine the existence and the similarities of the serial dependence(s) in the evaluation of seven facial characteristics (i.e., attractiveness, trustworthiness, confidence, dominance, intelligence, age, and aggressiveness). The convergent evidence from conventional Derivative of Gaussian fitting and Markov Chain modeling demonstrated that (1) serial dependence exists in judgments of all seven social facial characteristic, (2) the serial dependences of them are highly similar, and (3) the serial dependence follows the efficient coding. Thus it is highly possible that there exists a general serial dependence mechanism for (at least high-level) vision processing. Moreover, we used the Markov Chain modeling to better describe the transitional pattern of serial dependence, which is a kind of Markov process. These findings may shed light on future works regarding serial dependence, as well as face perception.

Individual differences in serial dependence manifest when sensory uncertainty is high

Perceptual experience in the recent past has been shown to alter subsequent perception. Recently, it has been suggested that this "serial dependence" effect is modulated by sensory uncertainty. In the current study, by overlaying three different levels of visual noise (i.e., no-, low-, or high-noise) on face stimuli, we investigated how serial dependence in face identity perception varies with sensory uncertainty. After learning two facial identities, the faces were combined at various morph levels and participants reported which identity was perceived while noise and noise-free presentations alternated over trials. Results showed that identity perception of noise-free faces was positively biased toward the past when the previous face was noise-free or highly noisy, but not when a low-noise was added. There were considerable individual differences in bias magnitude for trials preceded by high-noise stimuli which reflected individuals' general bias tendencies. When correlated with the other two conditions, a general bias tendency showed a significant relationship with low-noise trials, but not with no-noise trials. This indicates that the bias tendency of individuals manifests more strongly when the sensory information was uncertain. Therefore, the current findings suggest 1) that sensory uncertainty modulates serial dependence in face identity perception and 2) that an individual's general bias tendency interacts with sensory uncertainty to alter perception.

Serial dependence revealed in history-dependent perceptual templates

In any given perceptual task, the visual system selectively weighs or filters incoming information. The particular set of weights or filters form a kind of template, which reveals the regions or types of information that are particularly useful for a given perceptual decision.1,2 Unfortunately, sensory input is noisy and ever changing. To compensate for these fluctuations, the visual system could adopt a strategy of biasing the templates such that they reflect a temporal smoothing of input, which would be a form of serial dependence.3-5 Here, we demonstrate that perceptual templates are, in fact, altered by serial dependence. Using a simple orientation detection task and classification-image technique, we found that perceptual templates are systematically biased toward previously seen, task-irrelevant orientations. The results of an orientation discrimination task suggest that this shift in perceptual template derives from a change in the perceptual appearance of orientation. Our study reveals how serial dependence biases internal templates of orientation and suggests that the sensitivity of classification-image techniques in general could be improved by taking into account history-dependent fluctuations in templates.

A shared mechanism for facial expression in human faces and face pareidolia

Facial expressions are vital for social communication, yet the underlying mechanisms are still being discovered. Illusory faces perceived in objects (face pareidolia) are errors of face detection that share some neural mechanisms with human face processing. However, it is unknown whether expression in illusory faces engages the same mechanisms as human faces. Here, using a serial dependence paradigm, we investigated whether illusory and human faces share a common expression mechanism. First, we found that images of face pareidolia are reliably rated for expression, within and between observers, despite varying greatly in visual features. Second, they exhibit positive serial dependence for perceived facial expression, meaning an illusory face (happy or angry) is perceived as more similar in expression to the preceding one, just as seen for human faces. This suggests illusory and human faces engage similar mechanisms of temporal continuity. Third, we found robust cross-domain serial dependence of perceived expression between illusory and human faces when they were interleaved, with serial effects larger when illusory faces preceded human faces than the reverse. Together, the results support a shared mechanism for facial expression between human faces and illusory faces and suggest that expression processing is not tightly bound to human facial features.

Serial dependence of facial identity reflects high-level face coding

Serial dependence of facial identity is a type of bias where the perceived identity of a face is biased towards a previously presented face. There are individual differences in serial dependence strength and tuning (how the strength varies depending on stimuli similarity), and previous research has shown that both stronger and more narrowly tuned serial dependence of facial identity is associated with better face recognition abilities. These results are consistent with the idea that this bias plays a functional role in face perception. It is important, therefore, to determine whether serial dependence of facial identity reflects a high-level face-coding mechanism acting on the identity of a face or instead predominantly reflects a bias in low-level features, which are also subject to serial dependence. We first sought evidence that serial dependence of facial identity survived changes in low-level visual features, by varying face viewpoint between successive stimuli. We found that serial dependence persisted across changes in viewpoint, arguing against an entirely low-level locus for this bias. We next tested whether the bias was affected by inversion, as sensitivity to inversion is argued to be a characteristic of high-level face-selective processing. Serial dependence was stronger and more narrowly tuned for upright than inverted faces. Taken together, our results are consistent with the view that serial dependence of facial identity affects high-level visual representations and may reflect a face-coding mechanism that is operating at the level of facial identity.

Auditory Rate Perception Displays a Positive Serial Dependence

We investigated perceived timing in auditory rate perception using a reproduction task. The study aimed to test (a) whether central tendency occurs in rate perception, as shown for interval timing, and (b) whether rate is perceived independently on each trial or shows a serial dependence, as shown for other perceptual attributes. Participants were well able to indicate perceived rate as reproduced and presented rates were linearly related with a slope that approached unity, although tapping significantly overestimated presented rates. While the slopes approached unity, they were significantly less than 1, indicating a central tendency in which reproduced rates tended towards the mean of the presented range. We tested for serial dependency by seeing if current trial rate reproductions depended on the preceding rate. In two conditions, a positive dependence was observed. A third condition in which participants withheld responses on every second trial produced a negative dependency. These results suggest separate components of serial dependence linked to stimulus and response: Withholding responses reveals a negative perceptual effect, whereas making responses adds a stronger positive effect that is postperceptual and makes the combined effect positive. Together, these data show that auditory rate perception exhibits both central tendency and serial dependence effects.

Positive sequential dependency for face attractiveness perception

Recent findings from several groups have demonstrated that visual perception at a given moment can be biased toward what was recently seen. This is true both for basic visual attributes and for more complex representations, such as face identity, gender, or expression. This assimilation to the recent past is a positive serial dependency, similar to a temporal averaging process that capitalizes on short-term correlations in visual input to reduce noise and boost perceptual continuity. Here we examine serial dependencies in face perception using a simple attractiveness rating task and a rapid series of briefly presented face stimuli. In a series of three experiments, our results confirm a previous report that face attractiveness exhibits a positive serial dependency. This intertrial effect is not only determined by face attractiveness on the previous trial, but also depends on the faces shown up to five trials back. We examine the effect of stimulus presentation duration and find that stimuli as brief as 56 ms produce a significant positive dependency similar in magnitude to that produced by stimuli presented for 1,000 ms. We observed stronger positive dependencies between same-gender faces, and found a task dependency: Alternating gender discrimination trials with attractiveness rating trials produced no serial dependency. In sum, these findings show that a perception-stabilizing assimilation effect operates in face attractiveness perception that is task dependent and is acquired surprisingly quickly.

Serial dependence in a simulated clinical visual search task

In everyday life, we continuously search for and classify objects in the environment around us. This kind of visual search is extremely important when performed by radiologists in cancer image interpretation and officers in airport security screening. During these tasks, observers often examine large numbers of uncorrelated images (tumor x-rays, checkpoint x-rays, etc.) one after another. An underlying assumption of such tasks is that search and recognition are independent of our past experience. Here, we simulated a visual search task reminiscent of medical image search and found that shape classification performance was strongly impaired by recent visual experience, biasing classification errors 7% more towards the previous image content. This perceptual attraction exhibited the three main tuning characteristics of Continuity Fields: serial dependence extended over 12 seconds back in time (temporal tuning), it occurred only between similar tumor-like shapes (feature tuning), and only within a limited spatial region (spatial tuning). Taken together, these results demonstrate that serial dependence influences shape perception and occurs in visual search tasks. They also raise the possibility of a detrimental impact of serial dependence in clinical and practically relevant settings, such as medical image perception.

Individual Differences in Serial Dependence of Facial Identity are Associated with Face Recognition Abilities

Serial dependence is a perceptual bias where current perception is biased towards prior visual input. This bias occurs when perceiving visual attributes, such as facial identity, and has been argued to play an important functional role in vision, stabilising the perception of objects through integration. In face identity recognition, this bias could assist in building stable representations of facial identity. If so, then individual variation in serial dependence could contribute to face recognition ability. To investigate this possibility, we measured both the strength of serial dependence and the range over which individuals showed this bias (the tuning) in 219 adults, using a new measure of serial dependence of facial identity. We found that better face recognition was associated with stronger serial dependence and narrower tuning, that is, showing serial dependence primarily when sequential faces were highly similar. Serial dependence tuning was further found to be a significant predictor of face recognition abilities independently of both object recognition and face identity aftereffects. These findings suggest that the extent to which serial dependence is used selectively for similar faces is important to face recognition. Our results are consistent with the view that serial dependence plays a functional role in face recognition.

Working Memory Maintenance Modulates Serial Dependence Effects of Perceived Emotional Expression

The stability of face perception is vital in interpersonal interactions. Recent studies have revealed the mechanism of the stability in the perception of stable attributes of faces (such as facial identity) by serial dependence, a phenomenon in which perception of current stimuli is pulled toward recently viewed stimuli. However, whether serial dependence of perceived emotional expression (a changeable attribute of faces) exists remains controversial, and its exact nature has not been examined yet. To address these issues, we used the methods of constant stimuli and two-interval forced choice tasks in three psychophysical experiments. Participants compared two successive facial expressions selected from a continuum with 50 morphed faces ranging from sad to happy. Experiment 1a and 1b showed that a perceived facial expression pulled toward previously seen facial expressions (i.e., a significant serial dependence effect), independent of response instructions. Furthermore, a stronger serial dependence effect was found when the first facial expression was retained in working memory for a longer delay duration (Experiment 2), and yet a weaker serial dependence effect was observed when a longer delay between decision and response was performed (Experiment 3). These findings indicate that serial dependence facilitates the stability of facial expression perception and is modulated by working memory representations.