EEG Decoding of Dynamic Facial Expressions of Emotion: Evidence from SSVEP and Causal Cortical Network Dynamics

EEG correlates of static and dynamic face perception: The role of naturalistic motion

Much of our understanding of how the brain processes dynamic faces comes from research that compares static photographs to dynamic morphs, which exhibit simplified, computer-generated motion. By comparing static, video recorded, and dynamic morphed expressions, we aim to identify the neural correlates of naturalistic facial dynamism, using time-domain and time-frequency analysis. Dynamic morphs were made from the neutral and peak frames of video recorded transitions of happy and fearful expressions, which retained expression change and removed asynchronous and non-linear features of naturalistic facial motion. We found that dynamic morphs elicited increased N400 amplitudes and lower LPP amplitudes compared to other stimulus types. Video recordings elicited higher LPP amplitudes and greater frontal delta activity compared to other stimuli. Thematic analysis of participant interviews using a large language model revealed that participants found it difficult to assess the genuineness of morphed expressions, and easier to analyse the genuineness of happy compared to fearful expressions. Our findings suggest that animating real faces with artificial motion may violate expectations (N400) and reduce the social salience (LPP) of dynamic morphs. Results also suggest that delta oscillations in the frontal region may be involved with the perception of naturalistic facial motion in happy and fearful expressions. Overall, our findings highlight the sensitivity of neural mechanisms required for face perception to subtle changes in facial motion characteristics, which has important implications for neuroimaging research using faces with simplified motion.

Assessing the influence of visual stimulus properties on steady-state visually evoked potentials and pupil diameter

Steady-State Visual Evoked Potentials (SSVEPs) are brain responses measurable via electroencephalography (EEG) in response to continuous visual stimulation at a constant frequency. SSVEPs have been instrumental in advancing our understanding of human vision and attention, as well as in the development of brain-computer interfaces (BCIs). Ongoing questions remain about which type of visual stimulus causes the most potent SSVEP response. The current study investigated the effects of color, size, and flicker frequency on the signal-to-noise ratio of SSVEPs, complemented by pupillary light reflex measurements obtained through an eye-tracker. Six participants were presented with visual stimuli that differed in terms of color (white, red, green), shape (circles, squares, triangles), size (10,000 to 30,000 pixels), flicker frequency (8 to 25 Hz), and grouping (one stimulus at a time versus four stimuli presented in a 2 × 2 matrix to simulate a BCI). The results indicated that larger stimuli elicited stronger SSVEP responses and more pronounced pupil constriction. Additionally, the results revealed an interaction between stimulus color and flicker frequency, with red being more effective at lower frequencies and white at higher frequencies. Future SSVEP research could focus on the recommended waveform, interactions between SSVEP and power grid frequency, a wider range of flicker frequencies, a larger sample of participants, and a systematic comparison of the information transfer obtained through SSVEPs, pupil diameter, and eye movements.

Neural representations of perspectival shapes and attentional effects: Evidence from fMRI and MEG

Does the human brain represent perspectival shapes, i.e., viewpoint-dependent object shapes, especially in relatively higher-level visual areas such as the lateral occipital cortex? What is the temporal profile of the appearance and disappearance of neural representations of perspectival shapes? And how does attention influence these neural representations? To answer these questions, we employed functional magnetic resonance imaging (fMRI), magnetoencephalography (MEG), and multivariate decoding techniques to investigate spatiotemporal neural representations of perspectival shapes. Participants viewed rotated objects along with the corresponding objective shapes and perspectival shapes (i.e., rotated round, round, and oval) while we measured their brain activities. Our results revealed that shape classifiers trained on the basic shapes (i.e., round and oval) consistently identified neural representations in the lateral occipital cortex corresponding to the perspectival shapes of the viewed objects regardless of attentional manipulations. Additionally, this classification tendency toward the perspectival shapes emerged approximately 200 ms after stimulus presentation. Moreover, attention influenced the spatial dimension as the regions showing the perspectival shape classification tendency propagated from the occipital lobe to the temporal lobe. As for the temporal dimension, attention led to a more robust and enduring classification tendency towards perspectival shapes. In summary, our study outlines a spatiotemporal neural profile for perspectival shapes that suggests a greater degree of perspectival representation than is often acknowledged.

Effects of altered-reality training on interocular disinhibition in amblyopia

Training of viewing an altered-reality environment dichoptically has been found to reactivate human adult ocular dominance plasticity, allowing improvement of vision for amblyopia. One suspected mechanism for this training effect is ocular dominance rebalancing through interocular disinhibition. Here, we investigated whether the training modulated the neural responses reflecting interocular inhibition. Thirteen patients with amblyopia and 11 healthy controls participated in this study. Before and after six daily altered-reality training sessions, participants watched flickering video stimuli with their steady-state visually evoked potential (SSVEP) signals recorded simultaneously. We assessed the amplitude of SSVEP response at intermodulation frequencies, which was a potential neural indicator of interocular suppression. The results showed that training weakened the intermodulation response only in the amblyopic group, which was in agreement with the hypothesis that the training reduced interocular suppression specific to amblyopia. Moreover, even one month after the training ended, we could still observe this neural training effect. These findings provide preliminary neural evidence in support of the disinhibition account for treating amblyopia. We also explain these results with the ocular opponency model, which, to our knowledge, is the first time for this binocular rivalry model to be used in explaining long-term ocular dominance plasticity.