Automatic Processing of Changes in Facial Emotions in Dysphoria: A Magnetoencephalography Study

Intensity dependence of auditory evoked potentials distinguish participants with unmedicated depression from non-depressed controls

Depression is a heterogeneous syndrome that impacts an individual's emotional, social, cognitive and bodily functioning. Depression is associated with biases in emotional processing, but alterations in basic sensory processing have received less attention in depression research. Here, we measured event-related potentials (ERPs) in response to changes in the intensity of auditory stimuli and the location of somatosensory stimuli in participants with depression and in non-depressed control participants. We tested whether auditory mismatch negativity, P3a or N1 intensity dependence response or somatosensory mismatch response, P3a, P50 or N80 can dissociate depressed participants and non-depressed controls, and we also analysed the effects of depression medication and age in this sample. N1 intensity dependence response was increased in unmedicated depressed participants relative to non-depressed controls. When age was controlled for in the analysis, the effect of depression was only at a trend level. N1 intensity dependence response correlated with depression severity at the whole sample level. We did not observe any depression-related alterations in auditory mismatch negativity or P3a or somatosensory ERPs. Our results may reflect an association between the N1 intensity dependence response and altered neurotransmitter activity in depression, but this should be confirmed in future studies.

The differential impact of face distractors on visual working memory across encoding and delay stages

External distractions often occur when information must be retained in visual working memory (VWM)-a crucial element in cognitive processing and everyday activities. However, the distraction effects can differ if they occur during the encoding rather than the delay stages. Previous research on these effects used simple stimuli (e.g., color and orientation) rather than considering distractions caused by real-world stimuli on VWM. In the present study, participants performed a facial VWM task under different distraction conditions across the encoding and delay stages to elucidate the mechanisms of distraction resistance in the context of complex real-world stimuli. VWM performance was significantly impaired by delay-stage but not encoding-stage distractors (Experiment 1). In addition, the delay distraction effect arose primarily due to the absence of distractor process at the encoding stage rather than the presence of a distractor during the delay stage (Experiment 2). Finally, the impairment in the delay-distraction condition was not due to the abrupt appearance of distractors (Experiment 3). Taken together, these findings indicate that the processing mechanisms previously established for resisting distractions in VWM using simple stimuli can be extended to more complex real-world stimuli, such as faces.

Alterations in working memory maintenance of fearful face distractors in depressed participants: An ERP study

Task-irrelevant threatening faces (e.g., fearful) are difficult to filter from visual working memory (VWM), but the difficulty in filtering non-threatening negative faces (e.g., sad) is not known. Depressive symptoms could also potentially affect the ability to filter different emotional faces. We tested the filtering of task-irrelevant sad and fearful faces by depressed and control participants performing a color-change detection task. The VWM storage of distractors was indicated by contralateral delay activity, a specific event-related potential index for the number of objects stored in VWM during the maintenance phase. The control group did not store sad face distractors, but they automatically stored fearful face distractors, suggesting that threatening faces are specifically difficult to filter from VWM in non-depressed individuals. By contrast, depressed participants showed no additional consumption of VWM resources for either the distractor condition or the non-distractor condition, possibly suggesting that neither fearful nor sad face distractors were maintained in VWM. Our control group results confirm previous findings of a threat-related filtering difficulty in the normal population while also suggesting that task-irrelevant non-threatening negative faces do not automatically load into VWM. The novel finding of the lack of negative distractors within VWM storage in participants with depressive symptoms may reflect a decreased overall responsiveness to negative facial stimuli. Future studies should investigate the mechanisms underlying distractor filtering in depressed populations.

A Review of AI Cloud and Edge Sensors, Methods, and Applications for the Recognition of Emotional, Affective and Physiological States

Affective, emotional, and physiological states (AFFECT) detection and recognition by capturing human signals is a fast-growing area, which has been applied across numerous domains. The research aim is to review publications on how techniques that use brain and biometric sensors can be used for AFFECT recognition, consolidate the findings, provide a rationale for the current methods, compare the effectiveness of existing methods, and quantify how likely they are to address the issues/challenges in the field. In efforts to achieve the key goals of Society 5.0, Industry 5.0, and human-centered design better, the recognition of emotional, affective, and physiological states is progressively becoming an important matter and offers tremendous growth of knowledge and progress in these and other related fields. In this research, a review of AFFECT recognition brain and biometric sensors, methods, and applications was performed, based on Plutchik's wheel of emotions. Due to the immense variety of existing sensors and sensing systems, this study aimed to provide an analysis of the available sensors that can be used to define human AFFECT, and to classify them based on the type of sensing area and their efficiency in real implementations. Based on statistical and multiple criteria analysis across 169 nations, our outcomes introduce a connection between a nation's success, its number of Web of Science articles published, and its frequency of citation on AFFECT recognition. The principal conclusions present how this research contributes to the big picture in the field under analysis and explore forthcoming study trends.

Auditory event-related potentials in separating patients with depressive disorders and non-depressed controls: A narrative review

This narrative review brings together the findings regarding the differences in the auditory event-related potentials (ERPs) between patients with depressive disorder and non-depressed control subjects. These studies' results can inform us of the possible alterations in sensory-cognitive processing in depressive disorders and the potential of using these ERPs in clinical applications. Auditory P3, mismatch negativity (MMN) and loudness dependence of auditory evoked potentials (LDAEP) were the subjects of the investigation. A search in PubMed yielded 84 studies. The findings of the reviewed studies were not highly consistent, but some patterns could be identified. For auditory P3b, the common findings were attenuated amplitude and prolonged latency among depressed patients. Regarding auditory MMN, especially the amplitude of duration deviance MMN was commonly attenuated, and the amplitude of frequency deviance MMN was increased in depressed patients. In LDAEP studies, generally, no differences between depressed patients and non-depressed controls were reported, although some group differences concerning specific depression subtypes were found. This review posits that future research should investigate whether certain stimulus conditions are particularly efficient at separating depressed and non-depressed participant groups. Future studies should contrast responses in different subpopulations of depressed patients, as well as different clinical groups (e.g., depressive disorder and anxiety disorder patients), to investigate the specificity of the auditory ERP alterations for depressive disorders.

Spatiotemporal Dynamics of Covert vs. Overt Emotional Face Processing in Dysphoria

People at risk of developing clinical depression exhibit attentional biases for emotional faces. To clarify whether such effects occur at an early, automatic, or at a late, deliberate processing stage of emotional processing, the present study used high-density electroencephalography during both covert and overt processing of sad, fearful, happy, and neutral expressions in healthy participants with high dysphoria (n = 16) and with low dysphoria (n = 19). A state-of-the-art non-parametric permutation-based statistical approach was then used to explore the effects of emotion, attentional task demands, and group. Behaviorally, participants responded faster and more accurately when overtly categorizing happy faces and they were slower and less accurate when categorizing sad and fearful faces, independent of the dysphoria group. Electrophysiologically, in an early time-window (N170: 140-180 ms), there was a significant main effect for the dysphoria group, with greater negative voltage for the high vs. low dysphoria group over the left-sided temporo-occipital scalp. Furthermore, there was a significant group by emotional interaction, with the high dysphoria group displaying greater negative amplitude N170 for happy than fearful faces. Attentional task demands did not influence such early effects. In contrast, in an intermediate time-window (EPN: 200-400 ms) and in a late time-window (LPP: 500-750 ms) there were no significant main effects nor interactions involving the dysphoria Group. The LPP results paralleled the behavioral results, with greater LPP voltages for sad and fearful relative to happy faces only in the overt task, but similarly so in the two dysphoria groups. This study provides novel evidence that alterations in face processing in dysphoric individuals can be seen at the early stages of face perception, as indexed by the N170, although not in the form of a typical pattern of mood-congruent attentional bias. In contrast, intermediate (EPN) and late (LPP) stages of emotional face processing appear unaffected by dysphoria. Importantly, the early dysphoria effect appears to be independent of the top-down allocation of attention, further supporting the idea that dysphoria may influence a stage of automatic emotional appraisal. It is proposed that it may be a consequence of a shift from holistic to feature-based processing of facial expressions, or may be due to the influence of negative schemas acting as a negative context for emotional facial processing.

Decreased intersubject synchrony in dynamic valence ratings of sad movie contents in dysphoric individuals

Emotional reactions to movies are typically similar between people. However, depressive symptoms decrease synchrony in brain responses. Less is known about the effect of depressive symptoms on intersubject synchrony in conscious stimulus-related processing. In this study, we presented amusing, sad and fearful movie clips to dysphoric individuals (those with elevated depressive symptoms) and control participants to dynamically rate the clips' valences (positive vs. negative). We analysed both the valence ratings' mean values and intersubject correlation (ISC). We used electrodermal activity (EDA) to complement the measurement in a separate session. There were no group differences in either the EDA or mean valence rating values for each movie type. As expected, the valence ratings' ISC was lower in the dysphoric than the control group, specifically for the sad movie clips. In addition, there was a negative relationship between the valence ratings' ISC and depressive symptoms for sad movie clips in the full sample. The results are discussed in the context of the negative attentional bias in depression. The findings extend previous brain activity results of ISC by showing that depressive symptoms also increase variance in conscious ratings of valence of stimuli in a mood-congruent manner.

Negative and Positive Bias for Emotional Faces: Evidence from the Attention and Working Memory Paradigms

Visual attention and visual working memory (VWM) are two major cognitive functions in humans, and they have much in common. A growing body of research has investigated the effect of emotional information on visual attention and VWM. Interestingly, contradictory findings have supported both a negative bias and a positive bias toward emotional faces (e.g., angry faces or happy faces) in the attention and VWM fields. We found that the classical paradigms-that is, the visual search paradigm in attention and the change detection paradigm in VWM-are considerably similar. The settings of these paradigms could therefore be responsible for the contradictory results. In this paper, we compare previous controversial results from behavioral and neuroscience studies using these two paradigms. We suggest three possible contributing factors that have significant impacts on the contradictory conclusions regarding different emotional bias effects; these factors are stimulus choice, experimental setting, and cognitive process. We also propose new research directions and guidelines for future studies.

Subjectively different emotional schematic faces not automatically discriminated from the brain's bioelectrical responses

The present study investigates how the brain automatically discriminates emotional schematic faces, as indicated by the mismatch responses, and how reliable these brain responses are. Thirty-three healthy volunteers participated in the vMMN EEG experiment with four experimental sets differing from each other by the type of standard (object with scrambled face features) and the type of deviants (Angry, Happy and Neutral schematic faces) presented. Conscious subjective evaluations of valence, arousal and attention catching of the same stimuli showed clear differentiation of emotional expressions. Deviant faces elicited rather similar vMMN at frontal and occipital sites. Bayesian analyses suggest that vMMN does not differ between angry and happy faces. Neutral faces, however, did not yield statistically significant vMMN at occipital leads. Pearson's correlation and intra-class correlation analyses showed that the brain's reactions to the stimuli were highly stable within individuals across the experimental sets, whereas the mismatch responses were much more variable.

Magnetoencephalography Responses to Unpredictable and Predictable Rare Somatosensory Stimuli in Healthy Adult Humans

Mismatch brain responses to unpredicted rare stimuli are suggested to be a neural indicator of prediction error, but this has rarely been studied in the somatosensory modality. Here, we investigated how the brain responds to unpredictable and predictable rare events. Magnetoencephalography responses were measured in adults frequently presented with somatosensory stimuli (FRE) that were occasionally replaced by two consecutively presented rare stimuli [unpredictable rare stimulus (UR) and predictable rare stimulus (PR); p = 0.1 for each]. The FRE and PR were electrical stimulations administered to either the little finger or the forefinger in a counterbalanced manner between the two conditions. The UR was a simultaneous electrical stimulation to both the forefinger and the little finger (for a smaller subgroup, the UR and FRE were counterbalanced for the stimulus properties). The grand-averaged responses were characterized by two main components: one at 30-100 ms (M55) and the other at 130-230 ms (M150) latency. Source-level analysis was conducted for the primary somatosensory cortex (SI) and the secondary somatosensory cortex (SII). The M55 responses were larger for the UR and PR than for the FRE in both the SI and the SII areas and were larger for the UR than for the PR. For M150, both investigated areas showed increased activity for the UR and the PR compared to the FRE. Interestingly, although the UR was larger in stimulus energy (stimulation of two fingers at the same time) and had a larger prediction error potential than the PR, the M150 responses to these two rare stimuli did not differ in source strength in either the SI or the SII area. The results suggest that M55, but not M150, can possibly be associated with prediction error signals. These findings highlight the need for disentangling prediction error and rareness-related effects in future studies investigating prediction error signals.

Ketamine Alters Electrophysiological Responses to Emotional Faces in Major Depressive Disorder

BACKGROUND

The glutamatergic modulator ketamine rapidly reduces depressive symptoms in individuals with treatment-resistant major depressive disorder (MDD). However, ketamine's effects on emotional processing biases remain largely unknown, and understanding these processes may help elucidate ketamine's mechanism of action.

METHODS

Magnetoencephalography (MEG) was used to investigate ketamine's effects on early visual responses to affective stimuli in individuals with MDD (n=31) and healthy volunteers (HVs; n=24). Participants were enrolled in a double-blind, placebo-controlled, crossover clinical trial and were assessed at baseline and after subanesthetic-dose ketamine and placebo-saline infusions. During MEG recording, participants completed an emotional evaluation task in which they indicated the sex or emotional valence (happy-neutral or sad-angry) of facial stimuli. Source-localized event-related field (ERF) M100 and M170 amplitudes and latencies were extracted from regions of interest. Linear fixed effects models examined interactions between diagnosis, stimulus valence, and drug session for behavioral and MEG data.

RESULTS

In baseline behavioral analyses, MDD participants exhibited higher accuracy for sad-angry than happy-neutral faces, and HVs responded faster to happy-neutral than sad-angry faces. In the MEG post-infusion analyses, calcarine M100 amplitudes were larger in MDD than HV participants post-placebo but became more similar post-ketamine. Finally, fusiform M170 amplitudes were associated with antidepressant response in MDD participants.

LIMITATIONS

The modest sample size and the need to collapse across responses to happy and neutral faces to increase statistical power limit the generalizability of the findings.

CONCLUSIONS

Ketamine rapidly altered emotional stimulus processing in MDD, laying the groundwork for future investigations of biomarkers of antidepressant treatment response.

CLINICAL TRIAL

Clinicaltrials.gov, NCT#00088699.

Emotional visual mismatch negativity: a joint investigation of social and non-social dimensions in adults with autism

Unusual behaviors and brain activity to socio-emotional stimuli have been reported in Autism Spectrum Disorder (ASD). Atypical reactivity to change and intolerance of uncertainty are also present, but little is known on their possible impact on facial expression processing in autism. The visual mismatch negativity (vMMN) is an electrophysiological response automatically elicited by changing events such as deviant emotional faces presented among regular neutral faces. While vMMN has been found altered in ASD in response to low-level changes in simple stimuli, no study has investigated this response to visual social stimuli. Here two deviant expressions were presented, neutral and angry, embedded in a sequence of repetitive neutral stimuli. vMMN peak analyses were performed for latency and amplitude in early and late time windows. The ASD group presented smaller amplitude of the late vMMN to both neutral and emotional deviants compared to the typically developed adults (TD) group, and only the TD group presented a sustained activity related to emotional change (i.e., angry deviant). Source reconstruction of the vMMNs further revealed that any change processing elicited a reduced activity in ASD group compared to TD in the saliency network, while the specific processing emotional change elicited activity in the temporal region and in the insula. This study confirms atypical change processing in ASD and points to a specific difficulty in the processing of emotional changes, potentially playing a crucial role in social interaction deficits. Nevertheless, these results require to be further replicated with a greater sample size and generalized to other emotional expressions.

Electrical brain activity and facial electromyography responses to irony in dysphoric and non-dysphoric participants

We studied irony comprehension and emotional reactions to irony in dysphoric and control participants. Electroencephalography (EEG) and facial electromyography (EMG) were measured when spoken conversations were presented with pictures that provided either congruent (non-ironic) or incongruent (ironic) contexts. In a separate session, participants evaluated the congruency and valence of the stimuli. While both groups rated ironic stimuli funnier than non-ironic stimuli, the control group rated all the stimuli funnier than the dysphoric group. N400-like activity, P600, and EMG activity indicating smiling were larger after the ironic stimuli than the non-ironic stimuli for both groups. Further, in the dysphoric group the irony modulation was evident in the electrode cluster over the right hemisphere, while no such difference in lateralization was observed in the control group. The results suggest a depression-related alteration in the P600 response associated to irony comprehension, but no alterations were found in emotional reactivity specifically related to irony.

The Mediating Effects of Coping Style on the Effects of Breath Count Mindfulness Training on Depressive Symptoms among International Students in China

Mindfulness training has gained popularity in the scientific field and has been proposed as an efficient way for emotional regulation. Mindfulness-based cognitive therapy (MBCT) is designed especially for depressive people in reducing risk of depression relapse and is recommended in national guidelines as a treatment choice for relapse prevention in recurrent depression. The aim of the current study was to investigate the effects of mindfulness training on depressive symptoms of international students and probe into the mediating role of mindfulness in stressful events and depression. In addition, we introduced a new kind of mindfulness training, the breathing exercise-based mindfulness training, which is based on the integration of Buddhism and Daoism. Self-report questionnaires assessing the coping style, abnormal depressive behavior, and stressful live events were completed in 260 international students in China (mean age = 21.4 years). The results showed that (1) many international students showed depression symptoms, (2) stressful life events play a completely mediating role in the initiation of depression and anxiety, and (3) mindfulness training for 8 weeks significantly reduced the depressive symptoms, and it was also related to a positive coping style. This study has certain theoretical significance in exploring the mechanism of the occurrence and development of depression among international students and provides useful tools for this special group of international students. In addition, the international students can also learn Chinese culture through the training. These findings indicate that mindfulness training and positive coping style are interrelated with treating depressive symptoms for international students.

Event-related potentials to task-irrelevant sad faces as a state marker of depression

Negative bias in face processing has been demonstrated in depression, but there are no longitudinal investigations of negative bias in symptom reduction. We recorded event-related potentials (P1 and N170) to task-irrelevant facial expressions in depressed participants who were later provided with a psychological intervention and in never depressed control participants. Follow-up measurements were conducted for the depressed group two and 39 months later. Negative bias was found specifically in the depression group, and was demonstrated as enlarged P1 amplitude to sad faces, which normalized in the follow-up measurements when the participants had fewer symptoms. Because the P1 amplitude recorded at the baseline did not differ between the depression group that recovered and the group that did not recover after the intervention, this brain response did not show potential as a biomarker for treatment response. It could have potential, however, to serve as a state-marker of depression.

Prediction-based neural mechanisms for shielding the self from existential threat

The human mind has an automatic tendency to avoid awareness of its mortality. How this protective mechanism is implemented at the neuronal level is unknown. Here we test the hypothesis that prediction-based mechanisms mediate death-denial by shielding the self from existential threat. We provide evidence that self-specific predictive processes are downregulated during the perception of death-related linguistic stimuli and that this mechanism can predict fear-of-death. Using a magnetoencephalography visual mismatch paradigm, we show that the brain's automatic prediction response to deviancy is eliminated when death words and self-face representations are coupled, but remains present when coupled to other-face or to negative words. We further demonstrate a functional link between how death impacts self-image vs. Other-image, and show that it predicts fear-of-death. Finally, we confirm this effect in a behavioral active inference experiment showing that death-related words bias perceptual judgment on facial self and other morphed video clips. Together these results lay out, for the first time, a plausible neural-based mechanism of death-denial.

Feature-specific prediction errors for visual mismatch

Predictive coding (PC) theory posits that our brain employs a predictive model of the environment to infer the causes of its sensory inputs. A fundamental but untested prediction of this theory is that the same stimulus should elicit distinct precision weighted prediction errors (pwPEs) when different (feature-specific) predictions are violated, even in the absence of attention. Here, we tested this hypothesis using functional magnetic resonance imaging (fMRI) and a multi-feature roving visual mismatch paradigm where rare changes in either color (red, green), or emotional expression (happy, fearful) of faces elicited pwPE responses in human participants. Using a computational model of learning and inference, we simulated pwPE and prediction trajectories of a Bayes-optimal observer and used these to analyze changes in blood oxygen level dependent (BOLD) responses to changes in color and emotional expression of faces while participants engaged in a distractor task. Controlling for visual attention by eye-tracking, we found pwPE responses to unexpected color changes in the fusiform gyrus. Conversely, unexpected changes of facial emotions elicited pwPE responses in cortico-thalamo-cerebellar structures associated with emotion and theory of mind processing. Predictions pertaining to emotions activated fusiform, occipital and temporal areas. Our results are consistent with a general role of PC across perception, from low-level to complex and socially relevant object features, and suggest that monitoring of the social environment occurs continuously and automatically, even in the absence of attention.