Electrophysiological correlates of face-evoked person knowledge

Are you for real? Decoding realistic AI-generated faces from neural activity

Can we trust our eyes? Until recently, we rarely had to question whether what we see is indeed what exists, but this is changing. Artificial neural networks can now generate realistic images that challenge our perception of what is real. This new reality can have significant implications for cybersecurity, counterfeiting, fake news, and border security. We investigated how the human brain encodes and interprets realistic artificially generated images using behaviour and brain imaging. We found that we could reliably decode AI generated faces using people's neural activity. However, while at a group level people performed near chance classifying real and realistic fakes, participants tended to interchange the labels, classifying real faces as realistic fakes and vice versa. Understanding this difference between brain and behavioural responses may be key in determining the 'real' in our new reality. Stimuli, code, and data for this study can be found at https://osf.io/n2z73/.

Early alpha/beta oscillations reflect the formation of face-related expectations in the brain

Although statistical regularities in the environment often go explicitly unnoticed, traces of implicit learning are evident in our neural activity. Recent perspectives have offered evidence that both pre-stimulus oscillations and peri-stimulus event-related potentials are reliable biomarkers of implicit expectations arising from statistical learning. What remains ambiguous, however, is the origination and development of these implicit expectations. To address this lack of knowledge and determine the temporal constraints of expectation formation, pre-stimulus increases in alpha/beta power were investigated alongside a reduction in the N170 and a suppression in peri-/post-stimulus gamma power. Electroencephalography was acquired from naive participants who engaged in a gender classification task. Participants were uninformed, that eight face images were sorted into four reoccurring pairs which were pseudorandomly hidden amongst randomly occurring face images. We found a reduced N170 for statistically expected images at left parietal and temporo-parietal electrodes. Furthermore, enhanced gamma power following the presentation of random images emphasized the bottom-up processing of these arbitrary occurrences. In contrast, enhanced alpha/beta power was evident pre-stimulus for expected relative to random faces. A particularly interesting finding was the early onset of alpha/beta power enhancement which peaked immediately after the depiction of the predictive face. Hence, our findings propose an approximate timeframe throughout which consistent traces of enhanced alpha/beta power illustrate the early prioritisation of top-down processes to facilitate the development of implicitly cued face-related expectations.

Face Recognition Deficits in a Patient With Alzheimer's Disease: Amnesia or Agnosia? The Importance of Electrophysiological Markers for Differential Diagnosis

Face recognition deficits are frequently reported in Alzheimer's disease (AD) and often attributed to memory impairment. However, it has been hypothesized that failure in identifying familiar people could also be due to deficits in higher-level perceptual processes, since there is evidence showing a reduced inversion effect for faces but not for cars in AD. To address the involvement of these higher processes, we investigated event-related potential (ERP) neural correlates of faces in a patient with AD showing a face recognition deficit. Eight healthy participants were tested as a control group. Participants performed different tasks following the stimulus presentation. In experiment 1, they should indicate whether the stimulus was either a face or a house or a scrambled image. In experiments 2 and 3, they should discriminate between upright and inverted faces (in experiment 2, stimuli were faces with neutral or fearful expressions, while in experiment 3, stimuli were famous or unfamiliar faces). Electrophysiological results reveal that the typical face-specific modulation of the N170 component, which is thought to reflect the structural encoding of faces, was not present in patient MCG, despite being affected by the emotional content of the face implicitly processed by MCG. Conversely, the N400 component, which is thought to reflect the recruitment of the memory trace of the face identity, was found to be implicitly modulated in MCG. These results may identify a possible role for gnosic processes in face recognition deficits in AD and suggest the importance of adopting an integrated approach to the AD diagnosis while considering electrophysiological markers.

Subliminal affective priming effect: Dissociated processes for intense versus normal facial expressions

Positive vs. negative intense-facial expressions are difficult to explicitly distinguish; yet, whether they dissociate when subliminally presented remains unclear. Through three experiments using affective priming paradigms, we assessed how intense facial expressions, when presented briefly (17 ms) and masked, influenced following neutral ambiguous words (Experiment 1) or visible facial expressions (Experiments 2&3). We also compared these results with those of using normal facial expressions as primes in each experiment. All experiments indicated masked affective priming effects (biasing valence judgement of neutral words or facilitating reaction time to faces with the same valence as the prime) in normal facial expression, but not those intense ones. Experiment 3 using event related potentials (ERPs) further revealed that two ERP components N250 and LPP were consistent with behavioral changes in the normal condition (larger when valences of primes and targets were different), but inconsistent in the intense condition. Taken together, our results provided behavioral and neural evidence for distinctive processing between normal and intense facial expressions under masked condition.

Perceptual and Semantic Phases of Face Identification Processing: A Multivariate Electroencephalography Study

Rapid identification of a familiar face requires an image-invariant representation of person identity. A varying sample of familiar faces is necessary to disentangle image-level from person-level processing. We investigated the time course of face identity processing using a multivariate electroencephalography analysis. Participants saw ambient exemplars of celebrity faces that differed in pose, lighting, hairstyle, and so forth. A name prime preceded a face on half of the trials to preactivate person-specific information, whereas a neutral prime was used on the remaining half. This manipulation helped dissociate perceptual- and semantic-based identification. Two time intervals within the post-face onset electroencephalography epoch were sensitive to person identity. The early perceptual phase spanned 110-228 msec and was not modulated by the name prime. The late semantic phase spanned 252-1000 msec and was sensitive to person knowledge activated by the name prime. Within this late phase, the identity response occurred earlier in time (300-600 msec) for the name prime with a scalp topography similar to the FN400 ERP. This may reflect a matching of the person primed in memory with the face on the screen. Following a neutral prime, the identity response occurred later in time (500-800 msec) with a scalp topography similar to the P600f ERP. This may reflect activation of semantic knowledge associated with the identity. Our results suggest that processing of identity begins early (110 msec), with some tolerance to image-level variations, and then progresses in stages sensitive to perceptual and then to semantic features.

Differential effects of face-realism and emotion on event-related brain potentials and their implications for the uncanny valley theory

Cartoon characters are omnipresent in popular media. While few studies have scientifically investigated their processing, in computer graphics, efforts are made to increase realism. Yet, close approximations of reality have been suggested to evoke sometimes a feeling of eeriness, the “uncanny valley” effect. Here, we used high-density electroencephalography to investigate brain responses to professionally stylized happy, angry, and neutral character faces. We employed six face-stylization levels varying from abstract to realistic and investigated the N170, early posterior negativity (EPN), and late positive potential (LPP) event-related components. The face-specific N170 showed a u-shaped modulation, with stronger reactions towards both most abstract and most realistic compared to medium-stylized faces. For abstract faces, N170 was generated more occipitally than for real faces, implying stronger reliance on structural processing. Although emotional faces elicited highest amplitudes on both N170 and EPN, on the N170 realism and expression interacted. Finally, LPP increased linearly with face realism, reflecting activity increase in visual and parietal cortex for more realistic faces. Results reveal differential effects of face stylization on distinct face processing stages and suggest a perceptual basis to the uncanny valley hypothesis. They are discussed in relation to face perception, media design, and computer graphics.

The Effects of Face Inversion and Face Race on the P100 ERP

Research about the neural basis of face recognition has investigated the timing and anatomical substrates of different stages of face processing. Scalp-recorded ERP studies of face processing have focused on the N170, an ERP with a peak latency of ∼170 msec that has long been associated with the initial structural encoding of faces. However, several studies have reported earlier ERP differences related to faces, suggesting that face-specific processes might occur before N170. Here, we examined the influence of face inversion and face race on the timing of face-sensitive scalp-recorded ERPs by examining neural responses to upright and inverted line-drawn and luminance-matched white and black faces in a sample of white participants. We found that the P100 ERP evoked by inverted faces was significantly larger than that evoked by upright faces. Although this inversion effect was statistically significant at 100 msec, the inverted-upright ERP difference peaked at 138 msec, suggesting that it might represent an activity in neural sources that overlap with P100. Inverse modeling of the inversion effect difference waveform suggested possible neural sources in pericalcarine extrastriate visual cortex and lateral occipito-temporal cortex. We also found that the inversion effect difference wave was larger for white faces. These results are consistent with behavioral evidence that individuals process the faces of their own races more configurally than faces of other races. Taken together, the inversion and race effects observed in the current study suggest that configuration influences face processing by at least 100 msec.