Shedding light on emotional perception: Interaction of brightness and semantic content in extrastriate visual cortex

A beautiful face is good when we're judged by others, a moral character is better

Moral beauty, reflected in one's actions, and facial beauty both affect how we are judged. Here, we investigated how moral and facial beauty interact to affect social judgments and emotional responses, employing event-related brain potentials (ERPs). All participants (female) associated positive, neutral, or negative verbal information with faces scoring high or low on attractiveness and performed ratings of the faces as manipulation checks. In a separate test phase, the faces were presented again, and participants made valenced social judgments of the persons. Results show a dominance of moral beauty in valenced social judgments as well as ERPs related to reflexive and evaluative emotional responses (early posterior negativity and late positive potential), whereas facial attractiveness mattered little. In contrast, facial attractiveness affected visual processing (N170). Similarly, relatively shallow impressions of attractiveness and likability that require no knowledge about the person were influenced by both facial attractiveness and social-emotional information. This pattern of dominant effects of social-emotional information regardless of attractiveness shows that when it comes to our emotional responses and social judgments, moral beauty is what matters most, even in the face of physical beauty.

Acupuncture effects on emotional reactivity of youth with self-reported depressive symptoms

Background

Emotional symptomatology is a hallmark of depression. Antidepressant often fail to effectively target emotional blunting, while acupuncture, by contrast, has emerged as a promising alternative. However, the exact electrophysiologic mechanisms remain unclear. This study aimed to investigate how acupuncture influences emotional reactivity in youth with self-reported depressive symptoms.

Methods

A modified oddball paradigm incorporating a negative emotional valence deviant, combined with event-related potential analysis, was used to measure emotional reactivity before and after intervention. Seventy individuals exhibiting depressive symptoms in the previous 2 weeks, were randomly assigned to either a verum or sham acupuncture group. Electroencephalogram data from 59 participants were analyzed following preprocessing and quality assessment. Occipital P1, N170, frontal N1, N2, and parietal P3 components were extracted. The Positive and Negative Affect Schedule (PANAS) was completed after each oddball session. The Massachusetts General Hospital Acupuncture Sensation Scale (MASS) was completed after each intervention session.

Results

The MASS Index was significantly higher in the verum group. However, significant increases in occipital P1, N170, frontal N1, N2, and parietal P3 amplitudes for high-negative, mild-negative, and neutral pictures were observed after the intervention in both the verum and sham groups, with no significant difference between the groups. Additionally, both groups induced PANAS changes, and positive effect changes were significantly correlated with N170 and P1 (in response to high-negative pictures) changes in the sham group.

Conclusion

Acupuncture altered emotional reactivity in youth with depressive symptoms, highlighting its potential role, albeit possibly non-specific, in depression prevention and treatment.

Neural dynamics of attentional resource allocation in early visual cortex: emotional scenes produce competitive interactions

Salient emotional visual cues receive prioritized processing in human visual cortex. To what extent emotional facilitation relies on preattentional stimulus processing preceding semantic analysis remains controversial. Making use of steady-state visual evoke potentials frequency-tagged to meaningful complex emotional scenes and their scrambled versions, presented in a 4-Hz rapid serial visual presentation fashion, the current study tested temporal dynamics of semantic and emotional cue processing. The neural dynamics of bottom-up capture of attention driven by concrete images were analyzed under a passive-viewing-like scenario and in a competitive context, where a concurrent foreground task realized with a random dot kinematogram flickering at 15 Hz enabled the concurrent monitoring of top-down selective attention. Aligned with the semantic primacy hypothesis, the steady-state visual evoke potentials' results provide evidence of an initial rapid capture of attention driven by objecthood, followed by heightened deployment of attentional resources to emotional scenes that remained stable for the entire stimulation period. We replicated previous findings in which emotional distractors first prompt visuocortical facilitation, followed by suppression of a concurrent foreground task. Modeling this time-delayed competition process fit the data better than a time-invariant trade-off between concurrent cues as assumed by most models of selective attention.

Empathy enhances decoding accuracy of human neurophysiological responses to emotional facial expressions of humans and dogs

Despite the growing interest in the nonhuman animal emotionality, we currently know little about the human brain processing of nonconspecific emotional expressions. Here, we characterized the millisecond-scale temporal dynamics of human brain responses to conspecific human and nonconspecific canine emotional facial expressions. Our results revealed generally similar cortical responses to human and dog facial expressions in the occipital cortex during the first 500 ms, temporal cortex at 100-500 ms and parietal cortex at 150-350 ms from the stimulus onset. Responses to dog faces were pronounced at the latencies in temporal cortices corresponding to the time windows of early posterior negativity and late posterior positivity, suggesting attentional engagement to emotionally salient stimuli. We also utilized support vector machine-based classifiers to discriminate between the brain responses to different images. The subject trait-level empathy correlated with the accuracy of classifying the brain responses of aggressive from happy dog faces and happy from neutral human faces. This result likely reflects the attentional enhancement provoked by the subjective ecological salience of the stimuli.

Beyond facial expressions: A systematic review on effects of emotional relevance of faces on the N170

The N170 is the most prominent electrophysiological signature of face processing. While facial expressions reliably modulate the N170, there is considerable variance in N170 modulations by other sources of emotional relevance. Therefore, we systematically review and discuss this research area using different methods to manipulate the emotional relevance of inherently neutral faces. These methods were categorized into (1) existing pre-experimental affective person knowledge (e.g., negative attitudes towards outgroup faces), (2) experimentally instructed affective person knowledge (e.g., negative person information), (3) contingency-based affective learning (e.g., fear-conditioning), or (4) the immediate affective context (e.g., emotional information directly preceding the face presentation). For all categories except the immediate affective context category, the majority of studies reported significantly increased N170 amplitudes depending on the emotional relevance of faces. Furthermore, the potentiated N170 was observed across different attention conditions, supporting the role of the emotional relevance of faces on the early prioritized processing of configural facial information, regardless of low-level differences. However, we identified several open research questions and suggest venues for further research.

FamFac - A Database of Famous Faces for Psychology Experiments

Introduction

High variation in the low-level proprieties of visual stimuli and varying degrees of familiarity with famous faces may have caused a bias in the results of investigations that tried to disentangle the processes involved in familiar and unfamiliar face processing (e.g., temporal differences in the detection of the first event-related potentials specialized in face processing may have been caused by different methods of controlling variance in the low-level proprieties of visual stimuli).

Objective

To address these problems, we developed a freely available database of 183 famous faces whose low-level proprieties (brightness, size, resolution) have been homogenized and the level of familiarity established.

Method

The brightness of the stimuli was standardized by a custom-developed algorithm. The size and the resolution of the pictures were homogenized in Gimp. The familiarity level of the famous faces was established by a group of 48 Portuguese college students.

Results

Our results suggest that the brightness of each image did not differ significantly from the mean brightness value of the stimuli set, confirming the standardizing ability of the algorithm. Forty-one famous faces were classified as highly familiar.

Main findings and implications

This study provides two important resources, as both the algorithm and the database are freely available for research purposes. The homogenization of the low-level features and the control of the level of familiarity of the famous faces included in our database should ensure that they do not elicit confounding effects such as the ones verified in past studies.

A comparative experimental study of visual brain event-related potentials to a working memory task: virtual reality head-mounted display versus a desktop computer screen

Virtual reality head mounted display (VR HMD) systems are increasingly utilised in combination with electroencephalography (EEG) in the experimental study of cognitive tasks. The aim of our investigation was to determine the similarities/differences between VR HMD and the computer screen (CS) in response to an n-back working memory task by comparing visual electrophysiological event-related potential (ERP) waveforms (N1/P1/P3 components). The same protocol was undertaken for VR HMD and CS with participants wearing the same EEG headcap. ERP waveforms obtained with the VR HMD environment followed a similar time course to those acquired in CS. The P3 mean and peak amplitudes obtained in VR HMD were not significantly different to those obtained in CS. In contrast, the N1 component was significantly higher in mean and peak amplitudes for the VR HMD environment compared to CS at the frontal electrodes. Significantly higher P1 mean and peak amplitudes were found at the occipital region compared to the temporal for VR HMD. Our results show that successful acquisition of ERP components to a working memory task is achievable by combining VR HMD with EEG. In addition, the higher amplitude N1/P1 components seen in VR HMD indicates the potential utility of this VR modality in the investigation of early ERPs. In conclusion, the combination of VR HMD with EEG/ERP would be a useful approach to advance the study of cognitive function in experimental brain research.

Impact of Visual Design Elements and Principles in Human Electroencephalogram Brain Activity Assessed with Spectral Methods and Convolutional Neural Networks

The visual design elements and principles (VDEPs) can trigger behavioural changes and emotions in the viewer, but their effects on brain activity are not clearly understood. In this paper, we explore the relationships between brain activity and colour (cold/warm), light (dark/bright), movement (fast/slow), and balance (symmetrical/asymmetrical) VDEPs. We used the public DEAP dataset with the electroencephalogram signals of 32 participants recorded while watching music videos. The characteristic VDEPs for each second of the videos were manually tagged for by a team of two visual communication experts. Results show that variations in the light/value, rhythm/movement, and balance in the music video sequences produce a statistically significant effect over the mean absolute power of the Delta, Theta, Alpha, Beta, and Gamma EEG bands (p < 0.05). Furthermore, we trained a Convolutional Neural Network that successfully predicts the VDEP of a video fragment solely by the EEG signal of the viewer with an accuracy ranging from 0.7447 for Colour VDEP to 0.9685 for Movement VDEP. Our work shows evidence that VDEPs affect brain activity in a variety of distinguishable ways and that a deep learning classifier can infer visual VDEP properties of the videos from EEG activity.

Effects of affective content and motivational context on neural gain functions during naturalistic scene perception

Visual scene processing is modulated by semantic, motivational, and emotional factors, in addition to physical scene statistics. An open question is to what extent those factors affect low-level visual processing. One index of low-level visual processing is the contrast response function (CRF), representing the change in neural or psychophysical gain with increasing stimulus contrast. Here we aimed to (a) establish the use of an electrophysiological technique for assessing CRFs with complex emotional scenes and (b) examine the effects of motivational context and emotional content on CRFs elicited by naturalistic stimuli, including faces and complex scenes (humans, animals). Motivational context varied by expectancy of threat (a noxious noise) versus safety. CRFs were measured in 18 participants by means of sweep steady-state visual evoked potentials. Results showed a facilitation in visuocortical sensitivity (contrast gain) under threat, compared with safe conditions, across all stimulus categories. Facial stimuli prompted heightened neural response gain, compared with scenes. Within the scenes, response gain was smaller for scenes high in emotional arousal, compared with low-arousing scenes, consistent with interference effects of emotional content. These findings support the notion that motivational context alters the contrast sensitivity of cortical tissue, differing from changes in response gain (activation) when visual cues themselves carry motivational/affective relevance.

Time-resolved classification of dog brain signals reveals early processing of faces, species and emotion

Dogs process faces and emotional expressions much like humans, but the time windows important for face processing in dogs are largely unknown. By combining our non-invasive electroencephalography (EEG) protocol on dogs with machine-learning algorithms, we show category-specific dog brain responses to pictures of human and dog facial expressions, objects, and phase-scrambled faces. We trained a support vector machine classifier with spatiotemporal EEG data to discriminate between responses to pairs of images. The classification accuracy was highest for humans or dogs vs. scrambled images, with most informative time intervals of 100–140 ms and 240–280 ms. We also detected a response sensitive to threatening dog faces at 30–40 ms; generally, responses differentiating emotional expressions were found at 130–170 ms, and differentiation of faces from objects occurred at 120–130 ms. The cortical sources underlying the highest-amplitude EEG signals were localized to the dog visual cortex.

Comparison of Outlier-Tolerant Models for Measuring Visual Complexity

Providing the visual complexity of an image in terms of impact or aesthetic preference can be of great applicability in areas such as psychology or marketing. To this end, certain areas such as Computer Vision have focused on identifying features and computational models that allow for satisfactory results. This paper studies the application of recent ML models using input images evaluated by humans and characterized by features related to visual complexity. According to the experiments carried out, it was confirmed that one of these methods, Correlation by Genetic Search (CGS), based on the search for minimum sets of features that maximize the correlation of the model with respect to the input data, predicted human ratings of image visual complexity better than any other model referenced to date in terms of correlation, RMSE or minimum number of features required by the model. In addition, the variability of these terms were studied eliminating images considered as outliers in previous studies, observing the robustness of the method when selecting the most important variables to make the prediction.

Effect of brightness of visual stimuli on EEG signals

The aim of this study was to examine brightness effect, which is the perceptual property of visual stimuli, on brain responses obtained during visual processing of these stimuli. For this purpose, brain responses of the brain to changes in brightness were explored comparatively using different emotional images (pleasant, unpleasant and neutral) with different luminance levels. In the study, electroencephalography recordings from 12 different electrode sites of 31 healthy participants were used. The power spectra obtained from the analysis of the recordings using short time Fourier transform were analyzed, and a statistical analysis was performed on features extracted from these power spectra. Statistical findings were compared with those obtained from behavioral data. The results showed that the brightness of visual stimuli affected the power of brain responses depending on frequency, time and location. According to the statistically verified findings, the increase in the brightness of pleasant and neutral images increased the average power of responses in the parietal and occipital regions whereas the increase in the brightness of unpleasant images decreased the average power of responses in these regions. Moreover, the statistical results obtained for unpleasant images were found to be in accordance with the behavioral data. The results revealed that the brightness of visual stimuli could be represented by changing the activity power of the brain cortex. The findings emphasized that the brightness of visual stimuli should be viewed as an important parameter in studies using emotional image techniques such as image classification, emotion evaluation and neuro-marketing.

Visual complexity modelling based on image features fusion of multiple kernels

Humans' perception of visual complexity is often regarded as one of the key principles of aesthetic order, and is intimately related to the physiological, neurological and, possibly, psychological characteristics of the human mind. For these reasons, creating accurate computational models of visual complexity is a demanding task. Building upon on previous work in the field (Forsythe et al., 2011; Machado et al., 2015) we explore the use of Machine Learning techniques to create computational models of visual complexity. For that purpose, we use a dataset composed of 800 visual stimuli divided into five categories, describing each stimulus by 329 features based on edge detection, compression error and Zipf's law. In an initial stage, a comparative analysis of representative state-of-the-art Machine Learning approaches is performed. Subsequently, we conduct an exhaustive outlier analysis. We analyze the impact of removing the extreme outliers, concluding that Feature Selection Multiple Kernel Learning obtains the best results, yielding an average correlation to humans' perception of complexity of 0.71 with only twenty-two features. These results outperform the current state-of-the-art, showing the potential of this technique for regression.

Rapid Extraction of Emotion Regularities from Complex Scenes in the Human Brain

Adaptive behavior requires the rapid extraction of behaviorally relevant information in the environment, with particular emphasis on emotional cues. However, the speed of emotional feature extraction from complex visual environments is largely undetermined. Here we use objective electrophysiological recordings in combination with frequency tagging to demonstrate that the extraction of emotional information from neutral, pleasant, or unpleasant naturalistic scenes can be completed at a presentation speed of 167 ms (i.e., 6 Hz) under high perceptual load. Emotional compared to neutral pictures evoked enhanced electrophysiological responses with distinct topographical activation patterns originating from different neural sources. Cortical facilitation in early visual cortex was also more pronounced for scenes with pleasant compared to unpleasant or neutral content, suggesting a positivity offset mechanism dominating under conditions of rapid scene processing. These results significantly advance our knowledge of complex scene processing in demonstrating rapid integrative content identification, particularly for emotional cues relevant for adaptive behavior in complex environments.

The role of low-level image features in the affective categorization of rapidly presented scenes

It remains unclear how the visual system is able to extract affective content from complex scenes even with extremely brief (< 100 millisecond) exposures. One possibility, suggested by findings in machine vision, is that low-level features such as unlocalized, two-dimensional (2-D) Fourier spectra can be diagnostic of scene content. To determine whether Fourier image amplitude carries any information about the affective quality of scenes, we first validated the existence of image category differences through a support vector machine (SVM) model that was able to discriminate our intact aversive and neutral images with ~ 70% accuracy using amplitude-only features as inputs. This model allowed us to confirm that scenes belonging to different affective categories could be mathematically distinguished on the basis of amplitude spectra alone. The next question is whether these same features are also exploited by the human visual system. Subsequently, we tested observers' rapid classification of affective and neutral naturalistic scenes, presented briefly (~33.3 ms) and backward masked with synthetic textures. We tested categorization accuracy across three distinct experimental conditions, using: (i) original images, (ii) images having their amplitude spectra swapped within a single affective image category (e.g., an aversive image whose amplitude spectrum has been swapped with another aversive image) or (iii) images having their amplitude spectra swapped between affective categories (e.g., an aversive image containing the amplitude spectrum of a neutral image). Despite its discriminative potential, the human visual system does not seem to use Fourier amplitude differences as the chief strategy for affectively categorizing scenes at a glance. The contribution of image amplitude to affective categorization is largely dependent on interactions with the phase spectrum, although it is impossible to completely rule out a residual role for unlocalized 2-D amplitude measures.

Rapid brain responses to affective pictures indicate dimensions of trauma-related psychopathology in adolescents

A variety of mental disorders are related to deviant brain activity, but these neural alterations do not validate psychiatric diagnostic categories. High symptom overlap and variable symptom patterns encourage a dimensional approach. Following the logic of the Research Domain Criteria (RDoC), we investigated trauma survivors for symptom clusters that might be associated with characteristics of ERPs, in particular with the early posterior negativity (EPN) elicited during affective picture processing. In rapid serial visual presentation, 90 adolescents (40 male/50 female, age M = 15.0 ± 2.5 years) who had been exposed to varying amounts of traumatic stress passively viewed a stream of high‐arousing positive and low‐arousing neutral pictures taken from the International Affective Picture System (IAPS). Using standardized interviews, symptoms of trauma‐related mental disorders were assessed (including those for PTSD, depression, borderline personality disorder, and behavioral problems). A principal component analysis was performed to derive potential dimensions of psychopathology. Multiple regression analysis confirmed a factor comprising problems concentrating, sleeping difficulties, and mistrust as a predictor of a larger EPN difference between high‐arousing positive and low‐arousing neutral IAPS pictures (β = 0.19, p < 0.05). Sex predicted the magnitude of the EPN (β = 0.45, p < 0.001). Male adolescents displayed a stronger EPN suppression than female adolescents. The result suggests that problems concentrating, sleeping difficulties, and mistrust seem to be trans‐diagnostic elements related to diminished early emotional discrimination represented by the EPN. Furthermore, our findings indicate that the EPN in response to emotional processing is modulated by sex.

Based on the RDoC heuristic, symptom clusters that may be associated with the early posterior negativity (EPN) elicited during affective picture processing were investigated in youth trauma survivors. Analysis confirmed a factor comprising problems concentrating, sleeping difficulties, and mistrust as a predictor of a larger EPN difference between high arousing positive and low arousing neutral IAPS pictures using rapid serial visual presentation (RSVP). The EPN component was also modulated by sex. The findings also extend those of prior research on emotional processing confirming that EPN suppression can also be found in adolescents.

Electrophysiological correlates of the interplay between low-level visual features and emotional content during word reading

Processing affectively charged visual stimuli typically results in increased amplitude of specific event-related potential (ERP) components. Low-level features similarly modulate electrophysiological responses, with amplitude changes proportional to variations in stimulus size and contrast. However, it remains unclear whether emotion-related amplifications during visual word processing are necessarily intertwined with changes in specific low-level features or, instead, may act independently. In this pre-registered electrophysiological study, we varied font size and contrast of neutral and negative words while participants were monitoring their semantic content. We examined ERP responses associated with early sensory and attentional processes as well as later stages of stimulus processing. Results showed amplitude modulations by low-level visual features early on following stimulus onset - i.e., P1 and N1 components -, while the LPP was independently modulated by these visual features. Independent effects of size and emotion were observed only at the level of the EPN. Here, larger EPN amplitudes for negative were observed only for small high contrast and large low contrast words. These results suggest that early increase in sensory processing at the EPN level for negative words is not automatic, but bound to specific combinations of low-level features, occurring presumably via attentional control processes.

Competition for attentional resources between low spatial frequency content of emotional images and a foreground task in early visual cortex

Low spatial frequency (LSF) image content has been proposed to play a superior functional role in emotional content extraction via the magnocellular pathway biasing attentional resources toward emotional content in visual cortex. We investigated whether emotionally unpleasant complex images that were presented either unfiltered or with LSF content only in the background while subjects performed a foreground task will withdraw more attentional resources from the task compared to unemotional, neutral images (distraction paradigm). We measured steady-state visual evoked potentials (SSVEPs) driven by flickering stimuli of a foreground task. Unfiltered unpleasant images resulted in a significant reduction of SSVEP amplitude compared to neutral images. No statistically significant differences were found with LSF background images. In a behavioral control experiment, we found no significant differences for complexity ratings between unfiltered and LSF pictures. Content identification was possible for unfiltered and LSF picture (correct responses > 74%). An additional EEG study examined typical emotion-related components for complex images presented either as unfiltered, LSF, or high spatial frequency (HSF, as an additional control) filtered, unpleasant, and neutral images. We found a significant main effect of emotional valence in the early posterior negativity. Late positive potential differences were only found for unfiltered and HSF images. Results suggest that, while LSF content is sufficient to allow for content and emotional cue extraction when images were presented alone, LSF content is not salient enough to serve as emotional distractor that withdraws attentional resources from a foreground task in early visual cortex.

Electrophysiological Correlates of Emotional Content and Volume Level in Spoken Word Processing

For visual stimuli of emotional content as pictures and written words, stimulus size has been shown to increase emotion effects in the early posterior negativity (EPN), a component of event-related potentials (ERPs) indexing attention allocation during visual sensory encoding. In the present study, we addressed the question whether this enhanced relevance of larger (visual) stimuli might generalize to the auditory domain and whether auditory emotion effects are modulated by volume. Therefore, subjects were listening to spoken words with emotional or neutral content, played at two different volume levels, while ERPs were recorded. Negative emotional content led to an increased frontal positivity and parieto-occipital negativity-a scalp distribution similar to the EPN-between ~370 and 530 ms. Importantly, this emotion-related ERP component was not modulated by differences in volume level, which impacted early auditory processing, as reflected in increased amplitudes of the N1 (80-130 ms) and P2 (130-265 ms) components as hypothesized. However, contrary to effects of stimulus size in the visual domain, volume level did not influence later ERP components. These findings indicate modality-specific and functionally independent processing triggered by emotional content of spoken words and volume level.