Sound gaps boost emotional audiovisual integration independent of attention: Evidence from an ERP study

Early audiovisual integration in target processing under continuous noise: Behavioral and EEG evidence

Multisensory integration is interconnected across various information reception. The stage and mechanism of brain response to audiovisual integration have not been fully understood. In this study, we designed audiovisual and unisensory experiments to investigate task performance and electrophysiological characteristics associated with audiovisual integration in a continuous background interference environment using materials collected from the underwater environment. Behavioral results showed that the reaction time (RT) was shorter, and the accuracy was higher in the audiovisual experiment. The cumulative distribution function (CDF) results of RT indicated that audiovisual integration supported the co-activation model. Event-related potential (ERP) results revealed shorter latency of the P1 and N1 components in the audiovisual experiment. Microstate analysis indicated that the parietal-occipital area may play a key role in audiovisual integration. Moreover, event-related spectral perturbation (ERSP) results demonstrated the critical role of low-frequency oscillation in audiovisual integration at the early stage. Our findings support the view that the beneficial effect of audiovisual integration is predominantly upon the early stage of neural information processing, including task-independent information.

The Modulatory Effect of Exogenous Orienting on Audiovisual Emotional Integration: An ERP Study

BACKGROUND

In this study, we explored the interplay between exogenous orienting attention and emotional audiovisual integration (AVI) via electroencephalography (EEG).

METHODS

We designed a 2 (cue validity: valid, invalid) × 3 (emotion types: happiness, neutral and sadness) × 3 (modality: visual, auditory, audiovisual) discrimination task on the basis of the cue-target paradigm. Twenty-two participants (average age: 21.71 ± 1.84 years; 13 females, 9 males) were enrolled in this experiment. Participants were asked to respond to three emotional stimuli presented in different modalities by pressing a corresponding key.

RESULTS

The results indicated faster responses to multisensory stimuli than to unisensory stimuli and to the valid cue condition than to the invalid cue condition, which indicated multisensory advantage and cueing effect occurred. In addition, happiness stimuli induced the fastest response compared with neutral and sadness emotion stimuli. EEG findings indicated a reduction in audiovisual integration induced by valid exogenous orienting in the frontal, central and parietal lobe regions. Moreover, neutral emotional stimuli elicited greater audiovisual integration than stimuli expressing happiness and sadness did.

CONCLUSIONS

Overall, valid exogenous cues and emotional processing decreased audiovisual integration. The present study sheds light on how exogenous attention modulates emotional audiovisual integration and highlights the complex interactions among attention, sensory processing, and the emotional context in multisensory perception.

Microstate Analyses to Study face Processing in Healthy Individuals and Psychiatric Disorders: A Review of ERP Findings

Microstates represent brief periods of quasi-stable electroencephalography (EEG) scalp topography, offering insights into dynamic fluctuations in event-related potential (ERP) topographies. Despite this, there is a lack of a comprehensive systematic overview of microstate findings concerning cognitive face processing. This review aims to summarize ERP findings on face processing using microstate analyses and assess their effectiveness in characterizing face-related neural representations. A literature search was conducted for microstate ERP studies involving healthy individuals and psychiatric populations, utilizing PubMed, Google Scholar, Web of Science, PsychInfo, and Scopus databases. Twenty-two studies were identified, primarily focusing on healthy individuals (n = 16), with a smaller subset examining psychiatric populations (n = 6). The evidence reviewed in this study suggests that various microstates are consistently associated with distinct ERP stages involved in face processing, encompassing the processing of basic visual facial features to more complex functions such as analytical processing, facial recognition, and semantic representations. Furthermore, these studies shed light on atypical attentional neural mechanisms in Autism Spectrum Disorder (ASD), facial recognition deficits among emotional dysregulation disorders, and encoding and semantic dysfunctions in Post-Traumatic Stress Disorder (PTSD). In conclusion, this review underscores the practical utility of ERP microstate analyses in investigating face processing. Methodologies have evolved towards greater automation and data-driven approaches over time. Future research should aim to forecast clinical outcomes and conduct validation studies to directly demonstrate the efficacy of such analyses in inverse space.

EEG Microstates in Social and Affective Neuroscience

Social interactions require both the rapid processing of multifaceted socio-affective signals (e.g., eye gaze, facial expressions, gestures) and their integration with evaluations, social knowledge, and expectations. Researchers interested in understanding complex social cognition and behavior face a "black box" problem: What are the underlying mental processes rapidly occurring between perception and action and why are there such vast individual differences? In this review, we promote electroencephalography (EEG) microstates as a powerful tool for both examining socio-affective states (e.g., processing whether someone is in need in a given situation) and identifying the sources of heterogeneity in socio-affective traits (e.g., general willingness to help others). EEG microstates are identified by analyzing scalp field maps (i.e., the distribution of the electrical field on the scalp) over time. This data-driven, reference-independent approach allows for identifying, timing, sequencing, and quantifying the activation of large-scale brain networks relevant to our socio-affective mind. In light of these benefits, EEG microstates should become an indispensable part of the methodological toolkit of laboratories working in the field of social and affective neuroscience.

The impact of degraded vision on emotional perception of audiovisual stimuli: An event-related potential study

Emotion recognition will be challenged for individuals when visual signals are degraded in real-life scenarios. Recently, researchers have conducted many studies on the distinct neural activity between clear and degraded audiovisual stimuli. These findings addressed the "how" question, but the precise stage of the distinct activity that occurred remains unknown. Therefore, it is crucial to use event-related potential (ERP) to explore the "when" question, just the time course of the neural activity of degraded audiovisual stimuli. In the present research, we established two conditions: clear auditory + degraded visual (AcVd) and clear auditory + clear visual (AcVc) multisensory conditions. We enlisted 31 participants to evaluate the emotional valence of audiovisual stimuli. The resulting data were analyzed using ERP in time domains and Microstate analysis. Current results suggest that degraded vision impairs the early-stage processing of audiovisual stimuli, with the superior parietal lobule (SPL) regulating audiovisual processing in a top-down fashion. Additionally, our findings indicate that negative and positive stimuli elicit greater EPN compared to neutral stimuli, pointing towards a subjective motivation-related attentional regulation. To sum up, in the early stage of emotional audiovisual processing, the degraded visual signal affected the perception of the physical attributes of audiovisual stimuli and had a further influence on emotion extraction processing, leading to the different regulation of top-down attention resources in the later stage.