Effects of emotional intensity under perceptual load: An event-related potentials (ERPs) study

Is processing superiority a universal trait for all threats? Divergent impacts of fearful, angry, and disgusted faces on attentional capture

Fearful, angry, and disgusted facial expressions are evolutionarily salient and convey different types of threat signals. However, it remains unclear whether these three expressions impact sensory perception and attention in the same way. The present ERP study investigated the temporal dynamics underlying the processing of different types of threatening faces and the impact of attentional resources employed during a perceptual load task. Participants were asked to judge the length of bars superimposed over faces presented in the center of the screen. A mass univariate statistical approach was used to analyze the EEG data. Behaviorally, task accuracy was significantly reduced following exposure to fearful faces relative to neutral distractors, independent of perceptual load. The ERP results revealed that the P1 amplitude over the right hemisphere was found to be enhanced for fearful relative to disgusted faces, reflecting the rapid and coarse detection of fearful cues. The N170 responses elicited by fearful, angry, and disgusted faces were larger than those elicited by neutral faces, suggesting the largely automatic and preferential processing of threats. Furthermore, the early posterior negativity (EPN) component yielded increased responses to fearful and angry faces, indicating prioritized attention to stimuli representing acute threats. Additionally, perceptual load exerted a pronounced influence on the EPN and late positive potential (LPP), with larger responses observed in the low perceptual load condition, indicating goal-directed cognitive processing. Overall, the early sensory processing of fearful, angry, and disgusted faces is characterized by differential sensitivity in capturing attention automatically, despite the importance of these facial signals for survival. Fearful faces produce a strong interference effect and are processed with higher priority than angry and disgusted ones.

Investigating the neural effects of typicality and predictability for face and object stimuli

The brain calibrates itself based on the past stimulus diet, which makes frequently observed stimuli appear as typical (as opposed to uncommon stimuli, which appear as distinctive). Based on predictive processing theory, the brain should be more "prepared" for typical exemplars, because these contain information that has been encountered frequently, allowing it to economically represent items of that category. Thus, one could ask whether predictability and typicality of visual stimuli interact, or rather act in an additive manner. We adapted the design by Egner and colleagues (2010), who used cues to induce expectations about stimulus category (face vs. chair) occurrence during an orthogonal inversion detection task. We measured BOLD responses with fMRI in 35 participants. First, distinctive stimuli always elicited stronger responses than typical ones in all ROIs, and our whole-brain directional contrasts for the effects of typicality and distinctiveness converge with previous findings. Second and importantly, we could not replicate the interaction between category and predictability reported by Egner et al. (2010), which casts doubt on whether cueing designs are ideal to elicit reliable predictability effects. Third, likely as a consequence of the lack of predictability effects, we found no interaction between predictability and typicality in any of the four tested regions (bilateral fusiform face areas, lateral occipital complexes) when considering both categories, nor in the whole brain. We discuss the issue of replicability in neuroscience and sketch an agenda for how future studies might address the same question.

Comparison with others influences encoding and recognition of their faces: Behavioural and ERP evidence

In daily life, faces are often memorized within contexts involving interpersonal interactions. However, little is known about whether interpersonal interaction-related contexts influence face memory. The present study aimed to understand this question by investigating how social comparison-related context affects face encoding and recognition. To address this issue, 40 participants were informed that they and another player each played a monetary game and were then presented with both of their outcomes (either monetary gain or loss). Subsequently, participants were shown the face of the player whom they were just paired with. After all the faces had been encoded, participants were asked to perform a sudden old/new recognition task involving these faces. The results showed that, during the encoding phase, another player's monetary gain, compared to loss, resulted in more negative responses in the N170 and early posterior negativity (EPN)/N250 to relevant players' faces when participants encountered monetary loss and a smaller late positive potential (LPP) response irrespective of self-related outcomes. In the subsequent recognition phase, preceding another player's monetary gain as compared to loss led to better recognition performance and stronger EPN/N250 and LPP responses to the faces of relevant players when participants had lost some amount of money. These findings suggest that the social comparison-related context, particularly self-disadvantageous outcomes in the context, influences the memory of comparators' faces.

Recognition and evaluation of mental workload in different stages of perceptual and cognitive information processing using a multimodal approach

This study explores the effects of different perceptual and cognitive information processing stages on mental workload by assessing multimodal indicators of mental workload such as the NASA-TLX, task performance, ERPs and eye movements. Repeated measures ANOVA of the data showed that among ERP indicators, P1, N1 and N2 amplitudes were sensitive to perceptual load (P-load), P3 amplitude was sensitive to P-load only in the prefrontal region during high cognitive load (C-load) states, and P3 amplitude in the occipital and parietal regions was sensitive to C-load. Among the eye movement indicators, blink frequency was sensitive to P-load in all C-load states, but to C-load in only low P-load states; pupil diameter and blink duration were sensitive to both P-load and C-load. Based on the above indicators, the k-nearest neighbours (KNN) algorithm was used to propose a classification method for the four different mental workload states with an accuracy of 97.89%.Practitioner summary: Based on the results of this study, it is possible to implement the monitoring of mental workload states and optimise brain task allocation in operations involving high mental workload, such as human-computer interaction.

Emotional interference and attentional control in schizophrenia-spectrum disorders: The special case of neutral faces

BACKGROUND AND OBJECTIVES

Schizophrenia-spectrum disorders (SSD) are characterized by impaired emotion processing and attention. SSD patients are more sensitive to the presence of emotional distractors. But despite growing interest on the emotion-attention interplay, emotional interference in SSD is far from fully understood. Moreover, research to date has not established the link between emotional interference and attentional control in SSD. This study thus aimed to investigate the effects of facial expression and attentional control in SSD, by manipulating perceptual load.

METHODS

Twenty-two SSD patients and 22 healthy controls performed a target-letter discrimination task with task-irrelevant angry, happy, and neutral faces. Target-letter was presented among homogenous (low load) or heterogenous (high load) distractor-letters. Accuracy and RT were analysed using (generalized) linear mixed-effect models.

RESULTS

Accuracy was significantly lower in SSD patients than controls, regardless of perceptual load and facial expression. Concerning RT, SSD patients were significantly slower than controls in the presence of neutral faces, but only at high load. No group differences were observed for angry and happy faces.

LIMITATIONS

Heterogeneity of SSD, small sample size, lack of clinical control group, medication.

CONCLUSIONS

One possible explanation is that neutral faces captured exogenous attention to a greater extent in SSD, thus challenging attentional control in perceptually demanding conditions. This may reflect abnormal processing of neutral faces in SSD. If replicated, these findings will help to understand the interplay between exogenous attention, attentional control, and emotion processing in SSD, which may unravel the mechanism underlying socioemotional dysfunction in SSD.

Gender congruence and emotion effects in cross-modal associative learning: Insights from ERPs and pupillary responses

Social and emotional cues from faces and voices are highly relevant and have been reliably demonstrated to attract attention involuntarily. However, there are mixed findings as to which degree associating emotional valence to faces occurs automatically. In the present study, we tested whether inherently neutral faces gain additional relevance by being conditioned with either positive, negative, or neutral vocal affect bursts. During learning, participants performed a gender-matching task on face-voice pairs without explicit emotion judgments of the voices. In the test session on a subsequent day, only the previously associated faces were presented and had to be categorized regarding gender. We analyzed event-related potentials (ERPs), pupil diameter, and response times (RTs) of N = 32 subjects. Emotion effects were found in auditory ERPs and RTs during the learning session, suggesting that task-irrelevant emotion was automatically processed. However, ERPs time-locked to the conditioned faces were mainly modulated by the task-relevant information, that is, the gender congruence of the face and voice, but not by emotion. Importantly, these ERP and RT effects of learned congruence were not limited to learning but extended to the test session, that is, after removing the auditory stimuli. These findings indicate successful associative learning in our paradigm, but it did not extend to the task-irrelevant dimension of emotional relevance. Therefore, cross-modal associations of emotional relevance may not be completely automatic, even though the emotion was processed in the voice.

Beyond facial expressions: A systematic review on effects of emotional relevance of faces on the N170

The N170 is the most prominent electrophysiological signature of face processing. While facial expressions reliably modulate the N170, there is considerable variance in N170 modulations by other sources of emotional relevance. Therefore, we systematically review and discuss this research area using different methods to manipulate the emotional relevance of inherently neutral faces. These methods were categorized into (1) existing pre-experimental affective person knowledge (e.g., negative attitudes towards outgroup faces), (2) experimentally instructed affective person knowledge (e.g., negative person information), (3) contingency-based affective learning (e.g., fear-conditioning), or (4) the immediate affective context (e.g., emotional information directly preceding the face presentation). For all categories except the immediate affective context category, the majority of studies reported significantly increased N170 amplitudes depending on the emotional relevance of faces. Furthermore, the potentiated N170 was observed across different attention conditions, supporting the role of the emotional relevance of faces on the early prioritized processing of configural facial information, regardless of low-level differences. However, we identified several open research questions and suggest venues for further research.

Reward History Modulates the Processing of Task-Irrelevant Emotional Faces in a Demanding Task

The aim of the current study was to examine how reward-associated emotional facial distractors could capture attentional resources in a demanding visual task using event-related potentials (ERPs). In the learning phase, a high- or low-reward probability was paired with angry, happy, or neutral faces. Then, in the test phase, participants performed a face-irrelevant task with no reward at stake, in which they needed to discriminate the length of two lines presented in the center of the screen while faces that were taken from the learning phase were used as distractors presented in the periphery. The behavioral results revealed no effect of distractor emotional valence since the emotional information was task-irrelevant. The ERP results in the test phase revealed a significant main effect of distractor emotional valence for the parieto-occipital P200 (170-230 ms); the mean amplitudes in both the angry- and happy-face conditions were more positive than the neutral-face condition. Moreover, we found that the high-reward association enhanced both the N170 (140-180 ms) and EPN (260-330 ms) relative to the low-reward association condition. Finally, the N2pc (270-320 ms) also exhibited enhanced neural activity in the high-reward condition compared to the low-reward condition. The absence of emotional effects indicated that task-irrelevant emotional facial stimuli did not impact behavioral or neural responses in this highly demanding task. However, reward-associated information was processed when attention was directed elsewhere, suggesting that the processing of reward-associated information worked more in an automatic way, irrespective of the top-down task demand.

The effects of visual perceptual load on detection performance and event-related potentials to auditory stimuli

Load Theory states that perceptual load prevents, or at least reduces, the processing of task-unrelated stimuli. This study systematically examined the detection and neural processing of auditory stimuli unrelated to a visual foreground task. The visual task was designed to create continuous perceptual load, alternated between low and high load, and contained performance feedback to motivate participants to focus on the visual task instead of the auditory stimuli presented in the background. The auditory stimuli varied in intensity, and participants signaled their subjective perception of these stimuli without receiving feedback. Depending on stimulus intensity, we observed load effects on detection performance and P3 amplitudes of the event-related potential (ERP). N1 amplitudes were unaffected by perceptual load, as tested by Bayesian statistics. Findings suggest that visual perceptual load affects the processing of auditory stimuli in a late time window, which is associated with a lower probability of reported awareness of these stimuli.

The effects of visual working memory load on detection and neural processing of task-unrelated auditory stimuli

While perceptual load has been proposed to reduce the processing of task-unrelated stimuli, theoretical arguments and empirical findings for other forms of task load are inconclusive. Here, we systematically investigated the detection and neural processing of auditory stimuli varying in stimulus intensity during a stimuli-unrelated visual working memory task alternating between low and high load. We found, depending on stimulus strength, decreased stimulus detection and reduced P3, but unaffected N1 amplitudes of the event-related potential to auditory stimuli under high as compared to low load. In contrast, load independent awareness effects were observed during both early (N1) and late (P3) time windows. Findings suggest a late neural effect of visual working memory load on auditory stimuli leading to lower probability of reported awareness of these stimuli.

The effect of sad mood on early sensory event-related potentials to task-irrelevant faces

It has been shown that the perceiver's mood affects the perception of emotional faces, but it is not known how mood affects preattentive brain responses to emotional facial expressions. To examine the question, we experimentally induced sad and neutral mood in healthy adults before presenting them with task-irrelevant pictures of faces while an electroencephalography was recorded. Sad, happy, and neutral faces were presented to the participants in an ignore oddball condition. Differential responses (emotional - neutral) for the P1, N170, and P2 amplitudes were extracted and compared between neutral and sad mood conditions. Emotional facial expressions modulated all the components, and an interaction effect of expression by mood was found for P1: an emotional modulation to happy faces, which was found in neutral mood condition, disappeared in sad mood condition. For N170 and P2, we found larger response amplitudes for both emotional faces, regardless of the mood. The results add to the previous behavioral findings showing that mood already affects low-level cortical feature encoding of task-irrelevant faces.

Testing stimulus exposure time as the critical factor of increased EPN and LPP amplitudes for fearful faces during perceptual distraction tasks

Fearful facial expressions are prioritized across different information processing stages, as evident in early, intermediate, and late components of event-related brain potentials (ERPs). Recent studies showed that, in contrast to early N170 modulations, mid-latency (Early Posterior Negativity, EPN) and late (Late Positive Potential, LPP) emotional modulations depend on the attended perceptual feature. Nevertheless, several studies reported significant differences between emotional and neutral faces for the EPN or LPP components during distraction tasks. One cause for these conflicting findings might be that when faces are presented sufficiently long, participants attend to task-irrelevant features of the faces. In this registered report, we tested whether the presentation duration of faces is the critical factor for differences between reported emotional modulations during perceptual distraction tasks. To this end, 48 participants were required to discriminate the orientation of lines overlaid onto fearful or neutral faces, while face presentation varied (100 msec, 300 msec, 1,000 msec, 2,000 msec). While participants did not need to pay attention to the faces, we observed main effects of emotion for the EPN and LPP, but no interaction between emotion and presentation duration. Of note, unregistered exploratory tests per presentation duration showed no significant EPN and LPP emotion differences during short durations (100 and 300 msec) but significant differences with longer durations. While the presentation duration seems not to be a critical factor for EPN and LPP emotion effects, future studies are needed to investigate the role of threshold effects and the applied analytic designs to explain conflicting findings in the literature.

Up close and emotional: Electrophysiological dynamics of approaching angry faces

Recent evidence suggests that looming emotional faces are processed rapidly by the neural system, and that this apparent approach further interacts with emotion, causing an enhanced neural response for angry expressions. However, previous research has not demonstrated unequivocally if these effects are due to low-level visual features, or if they are indeed due to the emotional content of the stimuli. To address this question, the current study presented upright and inverted angry and neutral faces, which either expanded or contracted in size on a constant depth-cued background, such that they appeared to approach or retreat from the viewer. EEG/ERP measures were used to identify the time course of brain activity for these stimuli. The results showed that when faces were upright, both the P1 and N170 were enhanced for angry expressions, with the P1 being further increased with looming angry faces. The inversion of the faces caused an increase in both the P1 and N170 amplitudes, but no modulation was found for emotions. These findings show an early modulation of brain activity for upright looming angry faces and rule out the influence of low-level visual features as a contributing factor.

Promise for Personalized Diagnosis? Assessing the Precision of Wireless Consumer-Grade Electroencephalography across Mental States

In the last decade there has been significant growth in the interest and application of using EEG (electroencephalography) outside of laboratory as well as in medical and clinical settings, for more ecological and mobile applications. However, for now such applications have mainly included military, educational, cognitive enhancement, and consumer-based games. Given the monetary and ecological advantages, consumer-grade EEG devices such as the Emotiv EPOC have emerged, however consumer-grade devices make certain compromises of data quality in order to become affordable and easy to use. The goal of this study was to investigate the reliability and accuracy of EPOC as compared to a research-grade device, Brainvision. To this end, we collected data from participants using both devices during three distinct cognitive tasks designed to elicit changes in arousal, valence, and cognitive load: namely, Affective Norms for English Words, International Affective Picture System, and the n-Back task. Our design and analytical strategies followed an ideographic person-level approach (electrode-wise analysis of vincentized repeated measures). We aimed to assess how well the Emotiv could differentiate between mental states using an Event-Related Band Power approach and EEG features such as amplitude and power, as compared to Brainvision. The Emotiv device was able to differentiate mental states during these tasks to some degree, however it was generally poorer than Brainvision, with smaller effect sizes. The Emotiv may be used with reasonable reliability and accuracy in ecological settings and in some clinical contexts (for example, for training professionals), however Brainvision or other, equivalent research-grade devices are still recommended for laboratory or medical based applications.

Neurocognitive effects of a training program for poor face recognizers using shape and texture caricatures: A pilot investigation

Recent research suggested disproportional usage of shape information by people with poor face recognition, although texture information appears to be more important for familiar face recognition. Here, we tested a training program with faces that were selectively caricatured in either shape or texture parameters. Forty-eight young adults with poor face recognition skills (1 SD below the mean in at least 2/3 face processing tests: CFMT, GFMT, BFFT) were pseudo-randomly assigned to either one of two training groups or a control group (n = 16 each). Training comprised six sessions over three weeks. Per session, participants studied ten unfamiliar facial identities whose shape or texture characteristics were caricatured. Before and after training (or waiting in the control group), all participants completed EEG experiments on face learning and famous face recognition, and behavioral face processing tests. Results showed small but specific training-induced improvements: Whereas shape training improved face matching (training tasks, and to some extent GFMT), texture training elicited marked improvements in face learning (CFMT). Moreover, for the texture training group the N170 ERP was enhanced for novel faces post-training, suggesting training-induced changes in early markers of face processing. Although further research is necessary, this suggests that parameter-specific caricature training is a promising way to improve performance in people with poor face recognition skills.

How Task Set and Task Switching Modulate Perceptual Processes: Is Recognition of Facial Emotion an Exception?

In Part 1 we review task-switching and other studies showing that, even with time for preparation, participants' ability to shift attention to a relevant attribute or object before the stimulus onset is limited: there is a 'residual cost'. In particular, several brain potential markers of perceptual encoding are delayed on task-switch trials, compared to task-repeat trials that require attention to the same attribute as before. Such effects have been documented even for a process often considered 'automatic' - visual word recognition: ERP markers of word frequency and word/nonword status are (1) delayed when the word recognition task follows a judgement of a perceptual property compared to repeating the lexical task, and (2) strongly attenuated during the perceptual judgements. Thus, even lexical access seems influenced by the task/attentional set. In Part 2, we report in detail a demonstration of what seems to be a special case, where task-set and a task switch have no such effect on perceptual encoding. Participants saw an outline letter superimposed on a face expressing neutral or negative emotion, and were auditorily cued to categorise the letter as vowel/consonant, or the face as emotional/neutral. ERPs exhibited a robust emotional-neutral difference (Emotional Expression Effect) no smaller or later when switching to the face task than when repeating it; in the first half of its time-course it did not vary with the task at all. The initial encoding of the valence of a fixated facial emotional expression appears to be involuntary and invariant, whatever the endogenous task/attentional set.

Turning to the negative: attention allocation to emotional faces in adolescents with dysregulation profile-an event-related potential study

Patients with irritability, temper outbursts, hyperactivity and mood swings often meet the dysregulation profile (DP) of the Child Behavior Checklist (CBCL) or the Strengths and Difficulties Questionnaire (SDQ), which have been investigated over the past few decades. While the DP has emerged as a transdiagnostic marker with a negative impact on therapeutic outcome and psychosocial functioning, little is known about its underlying mechanisms such as attention and emotion regulation processes. In this study, we tested whether adolescent psychiatric patients (n = 27) with the SDQ-DP show impaired emotional face processing for task-irrelevant stimuli compared to psychiatric patients without the SDQ-DP (n = 30) and non-clinical adolescents (n = 21). Facial processing was tested with event-related potential (ERP) measures known to be modulated by attention (i.e., P1, N1, N170, P2, and Nc) during a modified Attention Network Task, to which task-irrelevant emotional stimuli (sad, fearful, and neutral faces) were added prior to the actual trial. The results reveal group differences in the orienting and in the conflicting network. Patients with DP showed a less efficient orienting network and the clinical control group showed a less efficient conflicting network. Moreover, patients with the dysregulation profile had a shorter N1/N170 latency than did the two control groups, suggesting that dysregulation in adolescents is associated with a faster but less arousing encoding of (task-irrelevant) emotional information and less top-down control.

Face and emotional expression processing under continuous perceptual load tasks: An ERP study

High perceptual load is thought to impair already the early stages of visual processing of task-irrelevant visual stimuli. However, recent studies showed no effects of perceptual load on early ERPs in response to task-irrelevant emotional faces. In this preregistered EEG study (N = 40), we investigated the effects of continuous perceptual load on ERPs to fearful and neutral task-irrelevant faces and their phase-scrambled versions. Perceptual load did not modulate face or emotion effects for the P1 or N170. In contrast, larger face-scramble and fearful-neutral differentiation were found during low as compared to high load for the Early Posterior Negativity (EPN). Further, face-independent P1, but face-dependent N170 emotional modulations were observed. Taken together, our findings show that P1 and N170 face and emotional modulations are highly resistant to load manipulations, indicating a high degree of automaticity during this processing stage, whereas the EPN might represent a bottleneck in visual information processing.

Effects of low-level visual information and perceptual load on P1 and N170 responses to emotional expressions

Emotional facial expressions lead to modulations of early event-related potentials (ERPs). However, it has so far remained unclear how far these modulations represent face-specific effects rather than differences in low-level visual features, and to which extent they depend on available processing resources. To examine these questions, we conducted two preregistered independent experiments (N = 40 in each experiment) using different variants of a novel task that manipulates peripheral perceptual load across levels but keeps overall visual stimulation constant. At the display center, we presented task-irrelevant angry, neutral, and happy faces and their Fourier phase-scrambled versions, which preserved low-level visual features. The results of both studies showed load-independent P1 and N170 emotional expression effects. Importantly, by using Bayesian analyses we could confirm that these facial expression effects were face-independent for the P1 but not for the N170 component. We conclude that firstly, ERP modulations during the P1 interval strongly depend on low-level visual information, while the N170 modulation requires the processing of figural facial expression features. Secondly, both P1 and N170 modulations appear to be immune to a large range of variations in perceptual load.

Attentional conditions differentially affect early, intermediate and late neural responses to fearful and neutral faces

The processing of fearful facial expressions is prioritized by the human brain. This priority is maintained across various information processing stages as evident in early, intermediate and late components of event-related potentials (ERPs). However, emotional modulations are inconsistently reported for these different processing stages. In this pre-registered study, we investigated how feature-based attention differentially affects ERPs to fearful and neutral faces in 40 participants. The tasks required the participants to discriminate either the orientation of lines overlaid onto the face, the sex of the face or the face's emotional expression, increasing attention to emotion-related features. We found main effects of emotion for the N170, early posterior negativity (EPN) and late positive potential (LPP). While N170 emotional modulations were task-independent, interactions of emotion and task were observed for the EPN and LPP. While EPN emotion effects were found in the sex and emotion tasks, the LPP emotion effect was mainly driven by the emotion task. This study shows that early responses to fearful faces are task-independent (N170) and likely based on low-level and configural information while during later processing stages, attention to the face (EPN) or-more specifically-to the face's emotional expression (LPP) is crucial for reliable amplified processing of emotional faces.

The effect of empathy and context on face-processing ERPs

The investigation of emotional face processing has largely used faces devoid of context, and does not account for within-perceiver differences in empathy. The importance of context in face perception has become apparent in recent years. This study examined the interaction of the contextual factors of facial expression, knowledge of a person's character, and within-perceiver empathy levels on face processing event-related potentials (ERPs). Forty-two adult participants learned background information about six individuals' character. Three types of character were described, in which the character was depicted as deliberately causing harm to others, accidently causing harm to others, or undertaking neutral actions. Subsequently, EEG was recorded while participants viewed the characters' faces displaying neutral or emotional expressions. Participants' empathy was assessed using the Empathy Quotient survey. Results showed a significant interaction of character type and empathy on the early posterior negativity (EPN) ERP component. These results suggested that for those with either low or high empathy, more attention was paid to the face stimuli, with more distinction between the different characters. In contrast, those in the middle range of empathy tended to produce smaller EPN with less distinction between character types. Findings highlight the importance of trait empathy in accounting for how faces in context are perceived.

Recognition of Contextually Threat-Related Scenes is Enhanced by Preceding Emotionally Incongruent Facial Expression

Previous studies on intentional and incidental face memory have investigated the effects of emotional facial expression on facial recognition itself. However, it is still uncertain whether facial expression influences later recognition of other emotional stimuli, such as emotional scenes. To address this issue, participants during the encoding phase were presented with emotional scenes together with facial expressions. The emotion of the scenes was either congruent or incongruent with that of the facial expression. In order to increase the attention towards faces/facial expressions, the task was related to faces rather than scenes, for example, the participants were asked either to memorize the facial identities (intentional face learning, Experiment 1) or to identify the gender of the faces (incidental face learning, Experiment 2). Subsequently, the participants were asked to perform an unexpected old/new recognition task regarding the scene pictures. In general, the results showed that recognition of threat-related scenes was enhanced by a preceding emotionally incongruent facial expression irrespective of intentional or incidental face learning. The findings indicate that facial expression influences recognition of contextual threat-related scenes.

Stimulus arousal drives amygdalar responses to emotional expressions across sensory modalities

The factors that drive amygdalar responses to emotionally significant stimuli are still a matter of debate - particularly the proneness of the amygdala to respond to negatively-valenced stimuli has been discussed controversially. Furthermore, it is uncertain whether the amygdala responds in a modality-general fashion or whether modality-specific idiosyncrasies exist. Therefore, the present functional magnetic resonance imaging (fMRI) study systematically investigated amygdalar responding to stimulus valence and arousal of emotional expressions across visual and auditory modalities. During scanning, participants performed a gender judgment task while prosodic and facial emotional expressions were presented. The stimuli varied in stimulus valence and arousal by including neutral, happy and angry expressions of high and low emotional intensity. Results demonstrate amygdalar activation as a function of stimulus arousal and accordingly associated emotional intensity regardless of stimulus valence. Furthermore, arousal-driven amygdalar responding did not depend on the visual and auditory modalities of emotional expressions. Thus, the current results are consistent with the notion that the amygdala codes general stimulus relevance across visual and auditory modalities irrespective of valence. In addition, whole brain analyses revealed that effects in visual and auditory areas were driven mainly by high intense emotional facial and vocal stimuli, respectively, suggesting modality-specific representations of emotional expressions in auditory and visual cortices.

Dual-Task Interference on Early and Late Stages of Facial Emotion Detection Is Revealed by Human Electrophysiology

Rapid and accurate processing of potential social threats is paramount to social thriving, and provides a clear evolutionary advantage. Though automatic processing of facial expressions has been assumed for some time, some researchers now question the extent to which this is the case. Here, we provide electrophysiological data from a psychological refractory period (PRP) dual-task paradigm in which participants had to decide whether a target face exhibited a neutral or fearful expression, as overlap with a concurrent auditory tone categorization task was experimentally manipulated. Specifically, we focused on four event-related potentials (ERP) linked to emotional face processing, covering distinct processing stages and topography: the early posterior negativity (EPN), early frontal positivity (EFP), late positive potential (LPP), and also the face-sensitive N170. As expected, there was an emotion modulation of each ERP. Most importantly, there was a significant attenuation of this emotional response proportional to the degree of task overlap for each component, except the N170. In fact, when the central overlap was greatest, this emotion-specific amplitude was statistically null for the EFP and LPP, and only marginally different from zero for the EPN. N170 emotion modulation was, on the other hand, unaffected by central overlap. Thus, our results show that emotion-specific ERPs for three out of four processing stages—i.e., perceptual encoding (EPN), emotion detection (EFP), or content evaluation (LPP)—are attenuated and even eliminated by central resource scarcity. Models assuming automatic processing should be revised to account for these results.

Contextual effects of angry vocal expressions on the encoding and recognition of emotional faces: An event-related potential (ERP) study

It has been shown that stimulus memory (e.g., encoding and recognition) is influenced by emotion. In terms of face memory, event-related potential (ERP) studies have shown that the encoding of emotional faces is influenced by the emotion of concomitant context, when contextual stimuli were input from a visual modality. Behavioral studies also investigated the effect of contextual emotion on subsequent recognition of neutral faces. However, there might be no studies ever investigating the context effect on face encoding and recognition, when contextual stimuli were input from other sensory modalities (e.g., an auditory modality). Additionally, it may be unknown about the neural mechanisms underlying context effects on recognition of emotional faces. Therefore, the present study aimed to use vocal expressions as contexts to investigate whether contextual emotion influences ERP responses during face encoding and recognition. To this end, participants in the present study were asked to memorize angry and neutral faces. The faces were presented concomitant with either angry or neutral vocal expressions. Subsequently, participants were asked to perform an old/new recognition task, in which only faces were presented. In the encoding phase, ERP results showed that compared to neutral vocal expression, angry vocal expressions led to smaller P1 and N170 responses to both angry and neutral faces. For angry faces, however, late positive potential (LPP) responses were increased in the angry voice condition. In the later recognition phase, N170 responses were larger for neutral-encoded faces that had been presented with angry compared to neutral vocal expressions. Preceding angry vocal expression increased FN400 and LPP responses to both neutral-encoded and angry-encoded faces, when the faces showed the encoded expression. Therefore, the present study indicates that contextual emotion with regard to vocal expression influences neural responses during face encoding and subsequent recognition.

TMS-EEG approach unveils brain mechanisms underlying conscious and unconscious face perception

BACKGROUND

Conscious perception of external stimuli has been related to recurrent activity in distributed cortical networks, although brain mechanisms controlling unconscious processing and stimuli access to conscious report need to be clarified.

OBJECTIVE

This study aims at investigating modulations in cortical excitability related to conscious perception and unconscious processing of face stimuli with different visibility levels.

METHODS

We used TMS-EEG over the right occipital face area (rOFA), or the right premotor cortex (rPMC) as control site, to measure cortical excitability during a backward masking paradigm with individually defined stimuli visibility.

RESULTS

Event related potentials showed significant differences for faces compared to houses, and detected faces compared to missed ones, 200 ms post target onset. TMS over rOFA, but not over rPMC, triggered a relative positivity starting 150 ms post target when faces with high visibility were consciously reported. Moreover, rOFA TMS evoked differential responses for high versus low visible faces in conscious and unconscious processing at 290-390 and 180-240 ms, respectively.

CONCLUSION

Results unveiled a causal link between rOFA excitability and late responses related to access to conscious perception, suggesting a critical role of recurrent activity, but distinct components, for consciously perceived stimuli and unconscious face processing.

N170 Reveals the Categorical Perception Effect of Emotional Valence

As an important attribute of facial expression, emotional valence has been well explored, but its processing mechanisms remain ambiguous. Investigating the categorical perception (CP) of emotional valence might help uncover the objective basis of the subjective dichotomy of emotional valence and identify the stage at which this processing of valence information might occur. A judgment task was used in the current study with stimuli from the within- or between-category condition, in which participants were required to decide whether two presented faces showed the same emotion. The results of the behavioral experiment revealed a significant CP effect of emotional valence, with faster RTs and greater accuracy for the between- than for the within-category stimuli. In the ERP experiment, the N170 (peaking at approximately 150-170 ms) was found to reflect the CP effect of emotional valence, with a larger amplitude for the within- than for the between-category condition. In contrast, the P1 component (peaking at approximately 100-130 ms) was insensitive to the CP effect of emotional valence. These results reveal the existence of the CP of emotional valence and indicate that the N170 is its earliest electrophysiological index. Therefore, the categorization of emotional valence not only has an objective neural basis but occurs at a relatively early stage of processing.

Contextual effects of surprised expressions on the encoding and recognition of emotional target faces: An event-related potential (ERP) study

Several studies reported that the encoding and recognition of emotional target faces are modulated by negative contextual expressions. However, it is unknown whether other contextual expressions, such as emotionally ambiguous expressions, affect the encoding and recognition of target faces. To this end, electroencephalography (EEG) was recorded during the presentation of angry or happy target faces after a random sequence of surprised or neutral contextual faces that did not differ in normative valence ratings. Subsequently, participants were asked to perform an unexpected old/new recognition task in which target faces were shown in either the encoded or a non-encoded expression. During the encoding phase, event-related potential (ERP) results showed that surprised as compared to neutral contextual faces led to smaller late positive potentials (LPP) for happy but not for angry target faces. Similar effects were also observed in the N170, even though the interaction of context and target expression failed to reach statistical significance. In the later recognition phase, recognition rates were lower for encoded happy faces when they had been encountered in surprised as compared to neutral context, regardless of whether the target face showed the encoded or a non-encoded expression. However, this context effect was not observed for angry-encoded faces. Taken together, the present study indicates that ambiguous contextual expressions, e.g., surprised faces, affect structural and cognitive encoding and later recognition of happy target faces to a larger extent than neutral contextual faces; whereas angry faces are more resistant to context effects.

Testing the effects of expression, intensity and age on emotional face processing in ASD

Individuals with autism spectrum disorder (ASD) commonly show global deficits in the processing of facial emotion, including impairments in emotion recognition and slowed processing of emotional faces. Growing evidence has suggested that these challenges may increase with age, perhaps due to minimal improvement with age in individuals with ASD. In the present study, we explored the role of age, emotion type and emotion intensity in face processing for individuals with and without ASD. Twelve- and 18-22- year-old children with and without ASD participated. No significant diagnostic group differences were observed on behavioral measures of emotion processing for younger versus older individuals with and without ASD. However, there were significant group differences in neural responses to emotional faces. Relative to TD, at 12 years of age and during adulthood, individuals with ASD showed slower N170 to emotional faces. While the TD groups' P1 latency was significantly shorter in adults when compared to 12 year olds, there was no significant age-related difference in P1 latency among individuals with ASD. Findings point to potential differences in the maturation of cortical networks that support visual processing (whether of faces or stimuli more broadly), among individuals with and without ASD between late childhood and adulthood. Finally, associations between ERP amplitudes and behavioral responses on emotion processing tasks suggest possible neural markers for emotional and behavioral deficits among individuals with ASD.

Effects of Intensity of Facial Expressions on Amygdalar Activation Independently of Valence

For several stimulus categories (e.g., pictures, odors, and words), the arousal of both negative and positive stimuli has been shown to modulate amygdalar activation. In contrast, previous studies did not observe similar amygdalar effects in response to negative and positive facial expressions with varying intensity of facial expressions. Reasons for this discrepancy may be related to analytical strategies, experimental design and stimuli. Therefore, the present study aimed at re-investigating whether the intensity of facial expressions modulates amygdalar activation by circumventing limitations of previous research. Event-related functional magnetic resonance imaging was used to assess brain activation while participants observed a static neutral expression and positive (happy) and negative (angry) expressions of either high or low intensity from an ecologically valid, novel stimulus set. The ratings of arousal and intensity were highly correlated. We found that amygdalar activation followed a u-shaped activation pattern with highest activation to high intense facial expressions as compared to low intensity facial expressions and to the neutral expression irrespective of valence, suggesting a critical role of the amygdala in valence-independent arousal processing of facial expressions. Additionally, consistent with previous studies, intensity effects were also found in visual areas and generally increased activation to angry versus happy faces were found in visual cortex and insula, indicating enhanced visual representations of high arousing facial expressions and increased visual and somatosensory representations of threat.