Joint Modulation of Facial Expression Processing by Contextual Congruency and Task Demands

Emotion and attention in face processing: Complementary evidence from surface event-related potentials and intracranial amygdala recordings

Face processing is biased by emotional and voluntarily directed attention, both of which modulate processing in distributed cortical areas. The amygdala is assumed to contribute to an attentional bias for emotional faces, although its interaction with directed attention awaits further clarification. Here, we studied the interaction of emotion and attention during face processing via scalp EEG potentials of healthy participants and intracranial EEG (iEEG) recordings of the right amygdala in one patient. Three randomized blocks consisting of angry, neutral, and happy facial expressions were presented, and one expression was denoted as the target category in each block. Happy targets were detected fastest and most accurately both in the group study and by the iEEG patient. Occipital scalp potentials revealed emotion differentiation for happy faces in the early posterior negativity (EPN) around 300 ms after stimulus onset regardless of the target condition. A similar response to happy faces occurred in the amygdala only for happy targets. On the scalp, a late positive potential (LPP, around 600 ms) enhancement for targets occurred for all target conditions alike. A simultaneous late signal in the amygdala was largest for emotional targets. No late signal enhancements were found for neutral targets in the amygdala. Cortical modulations, by contrast, showed both attention-independent effects of emotion and emotion-independent effects of attention. These results demonstrate an attention-dependence of amygdala activity during the processing of facial expressions and partly independent cortical mechanisms.

Shared Attention Amplifies the Neural Processing of Emotional Faces

Sharing an experience, without communicating, affects people's subjective perception of the experience, often by intensifying it. We investigated the neural mechanisms underlying shared attention by implementing an EEG study where participants attended to and rated the intensity of emotional faces, simultaneously or independently. Participants performed the task in three experimental conditions: (a) alone; (b) simultaneously next to each other in pairs, without receiving feedback of the other's responses (shared without feedback); and (c) simultaneously while receiving the feedback (shared with feedback). We focused on two face-sensitive ERP components: The amplitude of the N170 was greater in the "shared with feedback" condition compared to the alone condition, reflecting a top-down effect of shared attention on the structural encoding of faces, whereas the EPN was greater in both shared context conditions compared to the alone condition, reflecting an enhanced attention allocation in the processing of emotional content of faces, modulated by the social context. Taken together, these results suggest that shared attention amplifies the neural processing of faces, regardless of the valence of facial expressions.

Emotion, Gender and Gaze Discrimination Tasks do not Differentially Impact the Neural Processing of Angry or Happy Facial Expressions-a Mass Univariate ERP Analysis

Facial expression processing is a critical component of social cognition yet, whether it is influenced by task demands at the neural level remains controversial. Past ERP studies have found mixed results with classic statistical analyses, known to increase both Type I and Type II errors, which Mass Univariate statistics (MUS) control better. However, MUS open-access toolboxes can use different fundamental statistics, which may lead to inconsistent results. Here, we compared the output of two MUS toolboxes, LIMO and FMUT, on the same data recorded during the processing of angry and happy facial expressions investigated under three tasks in a within-subjects design. Both toolboxes revealed main effects of emotion during the N170 timing and main effects of task during later time points typically associated with the LPP component. Neither toolbox yielded an interaction between the two factors at the group level, nor at the individual level in LIMO, confirming that the neural processing of these two face expressions is largely independent from task demands. Behavioural data revealed main effects of task on reaction time and accuracy, but no influence of expression or an interaction between the two. Expression processing and task demands are discussed in the context of the consistencies and discrepancies between the two toolboxes and existing literature.

Does gaze direction of fearful faces facilitate the processing of threat? An ERP study of spatial precuing effects

Eye gaze is very important for attentional orienting in social life. By adopting the event-related potential (ERP) technique, we explored whether attentional orienting of eye gaze is modulated by emotional congruency between facial expressions and the targets in a spatial cuing task. Faces with different emotional expressions (fearful/angry/happy/neutral) directing their eye gaze to the left or right were used as cues, indicating the possible location of subsequent targets. Targets were line drawings of animals, which could be either threatening or neutral. Participants indicated by choice responses whether the animal would fit inside a shoebox in real life or not. Reaction times to targets were faster after valid compared with invalid cues, showing the typical eye gaze cuing effect. Analyses of the late positive potential (LPP) elicited by targets revealed a significant modulation of the gaze cuing effect by emotional congruency. Threatening targets elicited larger LPPs when validly cued by gaze in faces with negative (fearful and angry) expressions. Similarly, neutral targets showed larger LPPs when validly cued by faces with neutral expressions. Such effects were not present after happy face cues. Source localization in the LPP time window revealed that for threatening targets, the activity of right medial frontal gyrus could be related to a larger gaze-orienting effect for the fearful than the angry condition. Our findings provide electrophysiological evidence for the modulation of gaze cuing effects by emotional congruency.

Negative Mood States Are Related to the Characteristics of Facial Expression Drawing: A Cross-Sectional Study

An assessment of mood or emotion is important in developing mental health measures, and facial expressions are strongly related to mood or emotion. This study thus aimed to examine the relationship between levels of negative mood and characteristics of mouth parts when moods are drawn as facial expressions on a common platform. A cross-sectional study of Japanese college freshmen was conducted, and 1,068 valid responses were analyzed. The questionnaire survey consisted of participants’ characteristics, the Profile of Mood States (POMS), and a sheet of facial expression drawing (FACED), and the sheet was digitized and analyzed using an image-analysis software. Based on the total POMS score as an index of negative mood, the participants were divided into four groups: low (L), normal (N), high (H), and very high (VH). Lengths of drawn lines and between both mouth corners were significantly longer, and circularity and roundness were significantly higher in the L group. With increasing levels of negative mood, significant decreasing trends were observed in these lengths. Convex downward and enclosed figures were significantly predominant in the L group, while convex upward figures were significantly predominant and a tendency toward predominance of no drawn mouths or line figures was found in the H and VH groups. Our results suggest that mood states can be significantly related to the size and figure characteristics of drawn mouths of FACED on a non-verbal common platform. That is, these findings mean that subjects with low negative mood may draw a greater and rounder mouth and figures that may be enclosed and downward convex, while subjects with a high negative mood may not draw the line, or if any, may draw the line shorter and upward convex.

Feeling through another's eyes: Perceived gaze direction impacts ERP and behavioural measures of positive and negative affective empathy

Looking at the eyes informs us about the thoughts and emotions of those around us, and impacts our own emotional state. However, it is unknown how perceiving direct and averted gaze impacts our ability to share the gazer's positive and negative emotions, abilities referred to as positive and negative affective empathy. We presented 44 participants with contextual sentences describing positive, negative and neutral events happening to other people (e.g. "Her newborn was saved/killed/fed yesterday afternoon."). These were designed to elicit positive, negative, or little to no empathy, and were followed by direct or averted gaze images of the individuals described. Participants rated their affective empathy for the individual and their own emotional valence on each trial. Event-related potentials time-locked to face-onset and associated with empathy and emotional processing were recorded to investigate whether they were modulated by gaze direction. Relative to averted gaze, direct gaze was associated with increased positive valence in the positive and neutral conditions and with increased positive empathy ratings. A similar pattern was found at the neural level, using robust mass-univariate statistics. The N100, thought to reflect an automatic activation of emotion areas, was modulated by gaze in the affective empathy conditions, with opposite effect directions in positive and negative conditions.. The P200, an ERP component sensitive to positive stimuli, was modulated by gaze direction only in the positive empathy condition. Positive and negative trials were processed similarly at the early N200 processing stage, but later diverged, with only negative trials modulating the EPN, P300 and LPP components. These results suggest that positive and negative affective empathy are associated with distinct time-courses, and that perceived gaze direction uniquely modulates positive empathy, highlighting the importance of studying empathy with face stimuli.

Facing stereotypes: ERP responses to male and female faces after gender-stereotyped statements

Despite gender is a salient feature in face recognition, the question of whether stereotyping modulates face processing remains unexplored. Event-related potentials from 40 participants (20 female) was recorded as male and female faces matched or mismatched previous gender-stereotyped statements and were compared with those elicited by faces preceded by gender-unbiased statements. We conducted linear mixed-effects models to account for possible random effects from both participants and the strength of the gender bias. The amplitude of the N170 to faces was larger following stereotyped relative to gender-unbiased statements in both male and female participants, although the effect was larger for males. This result reveals that stereotyping exerts an early effect in face processing and that the impact is higher in men. In later time windows, male faces after female-stereotyped statements elicited large late positivity potential (LPP) responses in both men and women, indicating that the violation of male stereotypes induces a post-perceptual reevaluation of a salient or conflicting event. Besides, the largest LPP amplitude in women was elicited when they encountered a female face after a female-stereotyped statement. The later result is discussed from the perspective of recent claims on the evolution of women self-identification with traditionally held female roles.

Attentional conditions differentially affect early, intermediate and late neural responses to fearful and neutral faces

The processing of fearful facial expressions is prioritized by the human brain. This priority is maintained across various information processing stages as evident in early, intermediate and late components of event-related potentials (ERPs). However, emotional modulations are inconsistently reported for these different processing stages. In this pre-registered study, we investigated how feature-based attention differentially affects ERPs to fearful and neutral faces in 40 participants. The tasks required the participants to discriminate either the orientation of lines overlaid onto the face, the sex of the face or the face's emotional expression, increasing attention to emotion-related features. We found main effects of emotion for the N170, early posterior negativity (EPN) and late positive potential (LPP). While N170 emotional modulations were task-independent, interactions of emotion and task were observed for the EPN and LPP. While EPN emotion effects were found in the sex and emotion tasks, the LPP emotion effect was mainly driven by the emotion task. This study shows that early responses to fearful faces are task-independent (N170) and likely based on low-level and configural information while during later processing stages, attention to the face (EPN) or-more specifically-to the face's emotional expression (LPP) is crucial for reliable amplified processing of emotional faces.

The effect of empathy and context on face-processing ERPs

The investigation of emotional face processing has largely used faces devoid of context, and does not account for within-perceiver differences in empathy. The importance of context in face perception has become apparent in recent years. This study examined the interaction of the contextual factors of facial expression, knowledge of a person's character, and within-perceiver empathy levels on face processing event-related potentials (ERPs). Forty-two adult participants learned background information about six individuals' character. Three types of character were described, in which the character was depicted as deliberately causing harm to others, accidently causing harm to others, or undertaking neutral actions. Subsequently, EEG was recorded while participants viewed the characters' faces displaying neutral or emotional expressions. Participants' empathy was assessed using the Empathy Quotient survey. Results showed a significant interaction of character type and empathy on the early posterior negativity (EPN) ERP component. These results suggested that for those with either low or high empathy, more attention was paid to the face stimuli, with more distinction between the different characters. In contrast, those in the middle range of empathy tended to produce smaller EPN with less distinction between character types. Findings highlight the importance of trait empathy in accounting for how faces in context are perceived.

In context: emotional intent and temporal immediacy of contextual descriptions modulate affective ERP components to facial expressions

In this study, we explored how contextual information about threat dynamics affected the electrophysiological correlates of face perception. Forty-six healthy native Swedish speakers read verbal descriptions signaling an immediate vs delayed intent to escalate or deescalate an interpersonal conflict. Each verbal description was followed by a face with an angry or neutral expression, for which participants rated valence and arousal. Affective ratings confirmed that the emotional intent expressed in the descriptions modulated emotional reactivity to the facial stimuli in the expected direction. The electrophysiological data showed that compared to neutral faces, angry faces resulted in enhanced early and late event-related potentials (VPP, P300 and LPP). Additionally, emotional intent and temporal immediacy modulated the VPP and P300 similarly across angry and neutral faces, suggesting that they influence early face perception independently of facial affect. By contrast, the LPP amplitude to faces revealed an interaction between facial expression and emotional intent. Deescalating descriptions eliminated the LPP differences between angry and neutral faces. Together, our results suggest that information about a person’s intentions modulates the processing of facial expressions.

Attending to Eliza: rapid brain responses reflect competence attribution in virtual social feedback processing

In the age of virtual communication, the source of a message is often inferred rather than perceived, raising the question of how sender attributions affect content processing. We investigated this issue in an evaluative feedback scenario. Participants were told that an expert psychotherapist, a layperson or a randomly acting computer was going to give them online positive, neutral or negative personality feedback while high-density EEG was recorded. Sender attribution affected processing rapidly, even though the feedback was on average identical. Event-related potentials revealed a linear increase with attributed expertise beginning 150 ms after disclosure and most pronounced for N1, P2 and early posterior negativity components. P3 and late positive potential amplitudes were increased for both human senders and for emotionally significant (positive or negative) feedback. Strikingly, feedback from a putative expert prompted large P3 responses, even for inherently neutral content. Source analysis localized early enhancements due to attributed sender expertise in frontal and somatosensory regions and later responses in the posterior cingulate and extended visual and parietal areas, supporting involvement of mentalizing, embodied processing and socially motivated attention. These findings reveal how attributed sender expertise rapidly alters feedback processing in virtual interaction and have implications for virtual therapy and online communication.

Contextual information resolves uncertainty about ambiguous facial emotions: Behavioral and magnetoencephalographic correlates

Environmental conditions bias our perception of other peoples' facial emotions. This becomes quite relevant in potentially threatening situations, when a fellow's facial expression might indicate potential danger. The present study tested the prediction that a threatening environment biases the recognition of facial emotions. To this end, low- and medium-expressive happy and fearful faces (morphed to 10%, 20%, 30%, or 40% emotional) were presented within a context of instructed threat-of-shock or safety. Self-reported data revealed that instructed threat led to a biased recognition of fearful, but not happy facial expressions. Magnetoencephalographic correlates revealed spatio-temporal clusters of neural network activity associated with emotion recognition and contextual threat/safety in early to mid-latency time intervals in the left parietal cortex, bilateral prefrontal cortex, and the left temporal pole regions. Early parietal activity revealed a double dissociation of face-context information as a function of the expressive level of facial emotions: When facial expressions were difficult to recognize (low-expressive), contextual threat enhanced fear processing and contextual safety enhanced processing of subtle happy faces. However, for rather easily recognizable faces (medium-expressive) the left hemisphere (parietal cortex, PFC, and temporal pole) showed enhanced activity to happy faces during contextual threat and fearful faces during safety. Thus, contextual settings reduce the salience threshold and boost early face processing of low-expressive congruent facial emotions, whereas face-context incongruity or mismatch effects drive neural activity of easier recognizable facial emotions. These results elucidate how environmental settings help recognize facial emotions, and the brain mechanisms underlying the recognition of subtle nuances of fear.

Double-checking emotions: Valence and emotion category in contextual integration of facial expressions of emotion

Faces showing happy, angry or fearful expressions were presented in emotionally congruent or incongruent situational contexts (short sentences describing events that would usually provoke happiness, anger or fear). The participants were assigned the task of judging whether the expression was appropriate or not to the context (congruency judgment task). Effects of emotional congruency were observed at both the behavioral and electrophysiological levels. Behavioral results showed evidence of congruency effects based on specific emotion content (e.g., less accurate and slower responses to fear faces in angry contexts). Event-related-potentials (ERP) results also showed emotional congruency effects at different post-stimulus onset latencies, beginning with the face-sensitive N170 component. An effect of emotional congruency was also shown on the N400 component that is typically sensitive to semantic congruency. Finally, a late positive potential (LPP), appearing at 450-650 ms post-stimulus onset showed a complex pattern of effects with modulations driven by the different combinations of contexts and target expressions. These results are interpreted in terms of a double process of valence and emotion checking that is supposed to underlie affective processing and contextual integration of facial expressions of emotions.