View-independent coding of face identity in frontal and temporal cortices is modulated by familiarity: an event-related fMRI study

Decoding dynamic faces and scenes without awareness under dis-continuous flash suppression

In the perceptual sciences, there is an ongoing debate about the depth of unconscious processing. Here, we address this issue by implementing three improvements with regards to paradigm, stimuli and analyses to explore the neural correlates of unconscious face processing. Our results demonstrated that conscious faces elicited broader univariate activations than conscious scenes. Such results were absent when faces/scenes were suppressed and invisible (n = 43). However, further whole-brain multivariate classification revealed that both static and dynamic invisible faces could be distinguished from scenes in the occipital-temporal region. ROI analysis showed that bilateral FFA and OFA could differentiate dynamic invisible faces from dynamic invisible scenes. These findings suggest that interocularly suppressed faces are still processed in-depth in the ventral visual stream. Therefore, our study highlights the importance of optimizing stimulus signal, experimental paradigm, and analysis to extract unconscious signals in the brain.

Familiarity enhances functional connectivity between visual and nonvisual regions of the brain during natural viewing

We explored the neural correlates of familiarity with people and places using a naturalistic viewing paradigm. Neural responses were measured using functional magnetic resonance imaging, while participants viewed a movie taken from Game of Thrones. We compared inter-subject correlations and functional connectivity in participants who were either familiar or unfamiliar with the TV series. Higher inter-subject correlations were found between familiar participants in regions, beyond the visual brain, that are typically associated with the processing of semantic, episodic, and affective information. However, familiarity also increased functional connectivity between face and scene regions in the visual brain and the nonvisual regions of the familiarity network. To determine whether these regions play an important role in face recognition, we measured responses in participants with developmental prosopagnosia (DP). Consistent with a deficit in face recognition, the effect of familiarity was significantly attenuated across the familiarity network in DP. The effect of familiarity on functional connectivity between face regions and the familiarity network was also attenuated in DP. These results show that the neural response to familiarity involves an extended network of brain regions and that functional connectivity between visual and nonvisual regions of the brain plays an important role in the recognition of people and places during natural viewing.

A neural marker of the human face identity familiarity effect

Human adults associate different views of an identity much better for familiar than for unfamiliar faces. However, a robust and consistent neural index of this behavioral face identity familiarity effect (FIFE)-not found in non-human primate species-is lacking. Here we provide such a neural FIFE index, measured implicitly and with one fixation per face. Fourteen participants viewed 70 s stimulation sequences of a large set (n = 40) of widely variable natural images of a face identity at a rate of 6 images/second (6 Hz). Different face identities appeared every 5th image (1.2 Hz). In a sequence, face images were either familiar (i.e., famous) or unfamiliar, participants performing a non-periodic task unrelated to face recognition. The face identity recognition response identified at 1.2 Hz over occipital-temporal regions in the frequency-domain electroencephalogram was 3.4 times larger for familiar than unfamiliar faces. The neural response to familiar faces-which emerged at about 180 ms following face onset-was significant in each individual but a case of prosopdysgnosia. Besides potential clinical and forensic applications to implicitly measure one's knowledge of a face identity, these findings open new perspectives to clarify the neurofunctional source of the FIFE and understand the nature of human face identity recognition.

Social conformity is associated with inter-trial electroencephalogram variability

Human society encompasses diverse social influences, and people experience events differently and may behave differently under such influence, including in forming an impression of others. However, little is known about the underlying neural relevance of individual differences in following others' opinions or social norms. In the present study, we designed a series of tasks centered on social influence to investigate the underlying relevance between an individual's degree of social conformity and their neural variability. We found that individual differences under the social influence are associated with the amount of inter-trial electroencephalogram (EEG) variability over multiple stages in a conformity task (making face judgments and receiving social influence). This association was robust in the alpha band over the frontal and occipital electrodes for negative social influence. We also found that inter-trial EEG variability is a very stable, participant-driven internal state measurement and could be interpreted as mindset instability. Overall, these findings support the hypothesis that higher inter-trial EEG variability may be related to higher mindset instability, which makes participants more vulnerable to exposed external social influence. The present study provides a novel approach that considers the stability of one's endogenous neural signal during tasks and links it to human social behaviors.

The representation of shape and texture in category-selective regions of ventral-temporal cortex

Neuroimaging studies using univariate and multivariate approaches have shown that the fusiform face area (FFA) and parahippocampal place area (PPA) respond selectively to images of faces and places. The aim of this study was to determine the extent to which this selectivity to faces or places is based on the shape or texture properties of the images. Faces and houses were filtered to manipulate their texture properties, while preserving the shape properties (spatial envelope) of the images. In Experiment 1, multivariate pattern analysis (MVPA) showed that patterns of fMRI response to faces and houses in FFA and PPA were predicted by the shape properties, but not by the texture properties of the image. In Experiment 2, a univariate analysis (fMR‐adaptation) showed that responses in the FFA and PPA were sensitive to changes in both the shape and texture properties of the image. These findings can be explained by the spatial scale of the representation of images in the FFA and PPA. At a coarser scale (revealed by MVPA), the neural selectivity to faces and houses is sensitive to variation in the shape properties of the image. However, at a finer scale (revealed by fMR‐adaptation), the neural selectivity is sensitive to the texture properties of the image. By combining these neuroimaging paradigms, our results provide insights into the spatial scale of the neural representation of faces and places in the ventral‐temporal cortex.

Modulation of proper name recall by transcranial direct current stimulation of the anterior temporal lobes

We often fail to recall another person's name. Proper names might be more difficult to memorize and retrieve than other pieces of knowledge, such as one's profession because they are processed differently in the brain. Neuroimaging and neuropsychological studies associate the bilateral anterior temporal lobes (ATL) in the retrieval of proper names and other person-related knowledge. Specifically, recalling a person's name is thought to be supported by the left ATL, whereas recalling specific information such as a person's occupation is suggested to be subserved by the right ATL. To clarify and further explore the causal relationship between both ATLs and proper name retrieval, we stimulated these regions with anodal, cathodal and sham transcranial direct current stimulation (tDCS) while the participants memorized surnames (e.g., Mr. Baker) and professions (e.g., baker) presented with a person’s face. The participants were then later asked to recall the surname and the profession. Left ATL anodal stimulation resulted in higher intrusion errors for surnames than sham, whereas right ATL anodal stimulation resulted in higher overall intrusion errors, both, surnames and professions, compared to cathodal stimulation. Cathodal stimulation of the left and right ATL had no significant effect on surname and profession recall. The results indicate that the left ATL plays a role in recalling proper names. On the other hand, the specific role of the right ATL remaines to be explored.

Lesion-behaviour mapping reveals multifactorial neurocognitive processes in recognition memory for unfamiliar faces

Face recognition abilities, which play a critical role in social interactions, involve face processing and identifying familiar faces, but also remembering one-off encounters with previously unfamiliar faces. Previous functional imaging and lesion studies have found evidence for temporal, frontal, and parietal contributions to episodic recognition memory for previously unfamiliar faces. However, the functional contributions of these regions remain unclear. We, therefore, conducted a systematic group analysis of this memory function using lesion-behavior mapping. 95 first-event stroke patients (53 with right- and 42 with left-hemisphere damage) in the sub-acute phase performed the Wechsler Memory Scale (WMS-III) face recognition memory subtest. We analyzed their performance relative to 75 healthy controls, using signal detection measures. To identify brain lesions specifically implicated in face recognition deficits, we used voxel-based lesion-behavior mapping (VLBM; an analysis comparing the performance of participants with and without damage affecting a given voxel). Behavioral analysis disclosed a pronounced impairment in the performance of patients with right hemisphere damage. Frontal damage was associated with an increased amount of false alarms (i.e., failed rejection of new face items) and overly liberal criterion setting, without affecting the recognition of studied faces. In contrast, parietal damage was associated with impaired recognition of studied faces, which was more pronounced in immediate than in delayed retrieval. These findings suggest the existence of multifactorial neurocognitive processes in recognition memory for unfamiliar faces.

Spatially Adjacent Regions in Posterior Cingulate Cortex Represent Familiar Faces at Different Levels of Complexity

Extensive research has shown that perceptual information of faces is processed in a network of hierarchically-organized areas within ventral temporal cortex. For familiar and famous faces, perceptual processing of faces is normally accompanied by extraction of semantic knowledge about the social status of persons. Semantic processing of familiar faces could entail progressive stages of information abstraction. However, the cortical mechanisms supporting multistage processing of familiar faces have not been characterized. Here, using an event-related fMRI experiment, familiar faces from four celebrity groups (actors, singers, politicians, and football players) and unfamiliar faces were presented to the human subjects (both males and females) while they were engaged in a face categorization task. We systematically explored the cortical representations for faces, familiar faces, subcategories of familiar faces, and familiar face identities using whole-brain univariate analysis, searchlight-based multivariate pattern analysis (MVPA), and functional connectivity analysis. Convergent evidence from all these analyses revealed a set of overlapping regions within posterior cingulate cortex (PCC) that contained decodable fMRI responses for representing different levels of semantic knowledge about familiar faces. Our results suggest a multistage pathway in PCC for processing semantic information of faces, analogous to the multistage pathway in ventral temporal cortex for processing perceptual information of faces.SIGNIFICANCE STATEMENT Recognizing familiar faces is an important component of social communications. Previous research has shown that a distributed network of brain areas is involved in processing the semantic information of familiar faces. However, it is not clear how different levels of semantic information are represented in the brain. Here, we evaluated the multivariate response patterns across the entire cortex to discover the areas that contain information for familiar faces, subcategories of familiar faces, and identities of familiar faces. The searchlight maps revealed that different levels of semantic information are represented in topographically adjacent areas within posterior cingulate cortex (PCC). The results suggest that semantic processing of faces is mediated through progressive stages of information abstraction in PCC.

Nature and nurture shape structural connectivity in the face processing brain network

Face processing is a key ability facilitating social cognition. Only a few studies explored how nature and nurture shape face processing ontogeny at the behavioral and neural level. Also, very little is known about the contributions of nature and nurture to the establishment of white matter fibers supporting this specific human ability. The main purpose of this study was to assess genetic and environmental influences on white matter bundles connecting atlas-defined and functionally-defined face-responsive areas in the brain. Diffusion weighted images from 408 twins (monozygotic = 264, dizygotic = 144) were obtained from the WU-Minn Human Connectome Project. Fractional anisotropy - a widely used measure of fiber quality - of seven white matter tracts in the face network and ten global white matter tracts was analyzed by means of Structural Equation Modeling for twin data. Results revealed small and moderate genetic effects on face network fiber quality in addition to their shared variance with global brain white matter integrity. Furthermore, a theoretically expected common latent factor accounted for limited genetic and larger environmental variance in multiple face network fibers. The findings suggest that both genetic and environmental factors explain individual differences in fiber quality within the face network, as compared with much larger genetic effects on global brain white matter quality. In addition to heritability, individual-specific environmental influences on the face processing brain network are large, a finding that suggests to connect nature and nurture views on this remarkably specific human ability.

From concepts to percepts in human and machine face recognition: A reply to Blauch, Behrmann & Plaut

Intact recognition of familiar faces is critical for appropriate social interactions. Thus, the human face processing system should be optimized for familiar face recognition. Blauch et al. (2020) used face recognition deep convolutional neural networks (DCNNs) that are trained to maximize recognition of the trained (familiar) identities, to model human unfamiliar and familiar face recognition. In line with this model, we discuss behavioral, neuroimaging and computational findings that indicate that human face recognition develops from the generation of identity-specific concepts of familiar faces that are learned in a supervised manner, to the generation of view-invariant identity-general perceptual representations. Face-trained DCNNs seem to share some fundamental similarities with this framework.

The 4D Space-Time Dimensions of Facial Perception

Facial information is a powerful channel for human-to-human communication. Characteristically, faces can be defined as biological objects that are four-dimensional (4D) patterns, whereby they have concurrently a spatial structure and surface as well as temporal dynamics. The spatial characteristics of facial objects contain a volume and surface in three dimensions (3D), namely breadth, height and importantly, depth. The temporal properties of facial objects are defined by how a 3D facial structure and surface evolves dynamically over time; where time is referred to as the fourth dimension (4D). Our entire perception of another’s face, whether it be social, affective or cognitive perceptions, is therefore built on a combination of 3D and 4D visual cues. Counterintuitively, over the past few decades of experimental research in psychology, facial stimuli have largely been captured, reproduced and presented to participants with two dimensions (2D), while remaining largely static. The following review aims to advance and update facial researchers, on the recent revolution in computer-generated, realistic 4D facial models produced from real-life human subjects. We delve in-depth to summarize recent studies which have utilized facial stimuli that possess 3D structural and surface cues (geometry, surface and depth) and 4D temporal cues (3D structure + dynamic viewpoint and movement). In sum, we have found that higher-order perceptions such as identity, gender, ethnicity, emotion and personality, are critically influenced by 4D characteristics. In future, it is recommended that facial stimuli incorporate the 4D space-time perspective with the proposed time-resolved methods.

Human Face-Selective Cortex Does Not Distinguish between Members of a Racial Outgroup

People often fail to individuate members of social outgroups, a phenomenon known as the outgroup homogeneity effect. Here, we used functional magnetic resonance imaging (fMRI) repetition suppression to investigate the neural representation underlying this effect. In a preregistered study, White human perceivers (N = 29) responded to pairs of faces depicting White or Black targets. In each pair, the second face depicted either the same target as the first face, a different target from the same race, or a scrambled face outline. We localized face-selective neural regions via an independent task, and demonstrated that neural activity in the fusiform face area (FFA) distinguished different faces only when targets belonged to the perceivers' racial ingroup (White). By contrast, face-selective cortex did not discriminate between other-race individuals. Moreover, across two studies (total N = 67) perceivers were slower to discriminate between different outgroup members and remembered them to a lesser extent. Together, these results suggest that the outgroup homogeneity effect arises when early-to-mid-level visual processing results in an erroneous overlap of representations of outgroup members.

The bottom-up and top-down processing of faces in the human occipitotemporal cortex

Although face processing has been studied extensively, the dynamics of how face-selective cortical areas are engaged remains unclear. Here, we uncovered the timing of activation in core face-selective regions using functional Magnetic Resonance Imaging and Magnetoencephalography in humans. Processing of normal faces started in the posterior occipital areas and then proceeded to anterior regions. This bottom-up processing sequence was also observed even when internal facial features were misarranged. However, processing of two-tone Mooney faces lacking explicit prototypical facial features engaged top-down projection from the right posterior fusiform face area to right occipital face area. Further, face-specific responses elicited by contextual cues alone emerged simultaneously in the right ventral face-selective regions, suggesting parallel contextual facilitation. Together, our findings chronicle the precise timing of bottom-up, top-down, as well as context-facilitated processing sequences in the occipital-temporal face network, highlighting the importance of the top-down operations especially when faced with incomplete or ambiguous input.

Emotional learning promotes perceptual predictions by remodeling stimulus representation in visual cortex

Emotions exert powerful effects on perception and memory, notably by modulating activity in sensory cortices so as to capture attention. Here, we examine whether emotional significance acquired by a visual stimulus can also change its cortical representation by linking neuronal populations coding for different memorized versions of the same stimulus, a mechanism that would facilitate recognition across different appearances. Using fMRI, we show that after pairing a given face with threat through conditioning, viewing this face activates the representation of another viewpoint of the same person, which itself was never conditioned, leading to robust repetition-priming across viewpoints in the ventral visual stream (including medial fusiform, lateral occipital, and anterior temporal cortex). We also observed a functional-anatomical segregation for coding view-invariant and view-specific identity information. These results indicate emotional signals may induce plasticity of stimulus representations in visual cortex, serving to generate new sensory predictions about different appearances of threat-associated stimuli.

Convergent evolution of face spaces across human face-selective neuronal groups and deep convolutional networks

The discovery that deep convolutional neural networks (DCNNs) achieve human performance in realistic tasks offers fresh opportunities for linking neuronal tuning properties to such tasks. Here we show that the face-space geometry, revealed through pair-wise activation similarities of face-selective neuronal groups recorded intracranially in 33 patients, significantly matches that of a DCNN having human-level face recognition capabilities. This convergent evolution of pattern similarities across biological and artificial networks highlights the significance of face-space geometry in face perception. Furthermore, the nature of the neuronal to DCNN match suggests a role of human face areas in pictorial aspects of face perception. First, the match was confined to intermediate DCNN layers. Second, presenting identity-preserving image manipulations to the DCNN abolished its correlation to neuronal responses. Finally, DCNN units matching human neuronal group tuning displayed view-point selective receptive fields. Our results demonstrate the importance of face-space geometry in the pictorial aspects of human face perception.

Deep convolutional neural networks (DCNNs) are able to identify faces on par with humans. Here, the authors record neuronal activity from higher visual areas in humans and show that face-selective responses in the brain show similarity to those in the intermediate layers of the DCNN.

Symmetrical Viewpoint Representations in Face-Selective Regions Convey an Advantage in the Perception and Recognition of Faces

Learning new identities is crucial for effective social interaction. A critical aspect of this process is the integration of different images from the same face into a view-invariant representation that can be used for recognition. The representation of symmetrical viewpoints has been proposed to be a key computational step in achieving view-invariance. The aim of this study was to determine whether the representation of symmetrical viewpoints in face-selective regions is directly linked to the perception and recognition of face identity. In Experiment 1, we measured fMRI responses while male and female human participants viewed images of real faces from different viewpoints (-90, -45, 0, 45, and 90° from full-face view). Within the face regions, patterns of neural response to symmetrical views (-45 and 45° or -90 and 90°) were more similar than responses to nonsymmetrical views in the fusiform face area and superior temporal sulcus, but not in the occipital face area. In Experiment 2, participants made perceptual similarity judgements to pairs of face images. Images with symmetrical viewpoints were reported as being more similar than nonsymmetric views. In Experiment 3, we asked whether symmetrical views also convey an advantage when learning new faces. We found that recognition was best when participants were tested with novel face images that were symmetrical to the learning viewpoint. Critically, the pattern of perceptual similarity and recognition across different viewpoints predicted the pattern of neural response in face-selective regions. Together, our results provide support for the functional value of symmetry as an intermediate step in generating view-invariant representations.SIGNIFICANCE STATEMENT The recognition of identity from faces is crucial for successful social interactions. A critical step in this process is the integration of different views into a unified, view-invariant representation. The representation of symmetrical views (e.g., left profile and right profile) has been proposed as an important intermediate step in computing view-invariant representations. We found view symmetric representations were specific to some face-selective regions, but not others. We also show that these neural representations influence the perception of faces. Symmetric views were perceived to be more similar and were recognized more accurately than nonsymmetric views. Moreover, the perception and recognition of faces at different viewpoints predicted patterns of response in those face regions with view symmetric representations.

What does the functional organization of cortico-hippocampal networks tell us about the functional organization of memory?

Historically, research on the cognitive processes that support human memory proceeded, to a large extent, independently of research on the neural basis of memory. Accumulating evidence from neuroimaging, however, has enabled the field to develop a broader and more integrative perspective. Here, we briefly outline how advances in cognitive neuroscience can potentially shed light on concepts and controversies in human memory research. We argue that research on the functional properties of cortico-hippocampal networks informs us about how memories might be organized in the brain, which, in turn, helps to reconcile seemingly disparate perspectives in cognitive psychology. Finally, we discuss several open questions and directions for future research.

TMS over right OFA affects individuation of faces but not of exemplars of objects

In addition to its well-documented role in processing of faces, the occipital face area in the right hemisphere (rOFA) may also play a role in identifying specific individuals within a class of objects. Here we explored this issue by using fMRI-guided TMS. In a first experiment, participants had to judge whether two sequentially presented images of faces or objects represented exactly the same exemplar or two different exemplars of the same class, while receiving online TMS over either the rOFA, the right lateral occipital cortex (rLO) or the Vertex (control). We found that, relative to Vertex, stimulation of rOFA impaired individuation of faces only, with no effect on objects; in contrast, TMS over rLO reduced individuation of objects but not of faces. In a second control experiment participants judged whether a picture representing a fragment of a stimulus belonged or not to the subsequently presented image of a whole stimulus (part-whole matching task). Our results showed that rOFA stimulation selectively disrupted performance with faces, whereas performance with objects (but not with faces) was selectively affected by TMS over rLO. Overall, our findings suggest that rOFA does not contribute to discriminate between exemplars of non-face objects.

Distinct neural processes for the perception of familiar versus unfamiliar faces along the visual hierarchy revealed by EEG

Humans recognize faces with ease, despite the complexity of the task and of the visual system which underlies it. Different spatial regions, including both the core and extended face processing networks, and distinct temporal stages of processing have been implicated in face recognition, but there is ongoing controversy regarding the extent to which the mechanisms for recognizing a familiar face differ from those for an unfamiliar face. Here, we used electroencephalogram (EEG) and flicker SSVEP, a high signal-to-noise approach, and searchlight decoding methods to elucidate the mechanisms mediating the processing of familiar and unfamiliar faces in the time domain. Familiar and unfamiliar faces were presented periodically at 15 Hz, 6 Hz and 3.75 Hz either upright or inverted in separate blocks, with the rationale that faster frequencies require shorter processing times per image and tap into fundamentally different levels of visual processing. The 15 Hz trials, likely to reflect early visual processing, exhibited enhanced neural responses for familiar over unfamiliar face trials, but only when the faces were upright. In contrast, decoding methods revealed similar classification accuracies for upright and inverted faces for both familiar and unfamiliar faces. For the 6 Hz frequency, familiar faces had lower amplitude responses than unfamiliar faces, and decoding familiarity was more accurate for upright compared with inverted faces. Finally, the 3.75 Hz frequency revealed no main effects of familiarity, but decoding showed significant correlations with behavioral ratings of face familiarity, suggesting that activity evoked by this slow presentation frequency reflected higher-level, cognitive aspects of familiarity processing. This three-way dissociation between frequencies reveals that fundamentally different stages of the visual hierarchy are modulated by face familiarity. The combination of experimental and analytical approaches used here represent a novel method for elucidating spatio-temporal characteristics within the visual system.

Temporal Contiguity Training Influences Behavioral and Neural Measures of Viewpoint Tolerance

Humans can often recognize faces across viewpoints despite the large changes in low-level image properties a shift in viewpoint introduces. We present a behavioral and an fMRI adaptation experiment to investigate whether this viewpoint tolerance is reflected in the neural visual system and whether it can be manipulated through training. Participants saw training sequences of face images creating the appearance of a rotating head. Half of the sequences showed faces undergoing veridical changes in appearance across the rotation (non-morph condition). The other half were non-veridical: during rotation, the face simultaneously morphed into another face. This procedure should successfully associate frontal face views with side views of the same or a different identity, and, according to the temporal contiguity hypothesis, thus enhance viewpoint tolerance in the non-morph condition and/or break tolerance in the morph condition. Performance on the same/different task in the behavioral experiment (N = 20) was affected by training. There was a significant interaction between training (associated/not associated) and identity (same/different), mostly reflecting a higher confusion of different identities when they were associated during training. In the fMRI study (N = 20), fMRI adaptation effects were found for same-viewpoint images of untrained faces, but no adaptation for untrained faces was present across viewpoints. Only trained faces which were not morphed during training elicited a slight adaptation across viewpoints in face-selective regions. However, both in the behavioral and in the neural data the effects were small and weak from a statistical point of view. Overall, we conclude that the findings are not inconsistent with the proposal that temporal contiguity can influence viewpoint tolerance, with more evidence for tolerance when faces are not morphed during training.

Processing of gaze direction within the N170/M170 time window: A combined EEG/MEG study

Gaze direction is an important social signal for human beings. Beside the role of gaze in attention orienting, direct gaze (that is, gaze directed toward an observer) is a highly relevant biological stimulus that elicits attention capture and increases face encoding. Brain imaging studies have emphasized the role of the superior temporal sulcus (STS) in the coding of gaze direction and in the integration of gaze and head cues of social attention. The dynamics of the processing and integration of these cues remains, however, unclear. In order to address this question, we used deviated and frontal faces with averted and direct gaze in a combined electro- and magneto- encephalography (EEG-MEG) study. We showed distinct effects of gaze direction on the N170 and M170 responses. There was an interaction between gaze direction and head orientation between 134 and 162ms in MEG and a main effect of gaze direction between 171 and 186ms in EEG. These effects involved the posterior and anterior regions of the STS respectively. Both effects also emphasized the sensitivity to direct gaze. These data highlight the central role of the STS in gaze processing.

An image-invariant neural response to familiar faces in the human medial temporal lobe

The ability to recognise familiar faces with ease across different viewing conditions contrasts with the inherent difficulty in the perception of unfamiliar faces across similar image manipulations. Models of face processing suggest that this difference is based on the neural representation for familiar faces being more invariant to changes in the image, than it is for unfamiliar faces. Here, we used an fMR-adaptation paradigm to investigate neural correlates of image-invariant face recognition in face-selective regions of the human brain. Participants viewed faces presented in a blocked design. Each block contained different images of the same identity or different images from different identities. Faces in each block were either familiar or unfamiliar to the participants. First, we defined face-selective regions by comparing the response to faces with the response to scenes and scrambled faces. Next, we asked whether any of these face-selective regions showed image-invariant adaptation to the identity of a face. The core face-selective regions showed image-invariant adaptation to familiar and unfamiliar faces. However, there was no difference in the adaptation to familiar compared to unfamiliar faces. In contrast, image-invariant adaptation for familiar faces, but not for unfamiliar faces, was found in face-selective regions of the medial temporal lobe (MTL). Taken together, our results suggest that the marked differences in the perception of familiar and unfamiliar faces may depend critically on neural processes in the medial temporal lobe.

Efficient visual information for unfamiliar face matching despite viewpoint variations: It's not in the eyes!

Faces are encountered in highly diverse angles in real-world settings. Despite this considerable diversity, most individuals are able to easily recognize familiar faces. The vast majority of studies in the field of face recognition have nonetheless focused almost exclusively on frontal views of faces. Indeed, a number of authors have investigated the diagnostic facial features for the recognition of frontal views of faces previously encoded in this same view. However, the nature of the information useful for identity matching when the encoded face and test face differ in viewing angle remains mostly unexplored. The present study addresses this issue using individual differences and bubbles, a method that pinpoints the facial features effectively used in a visual categorization task. Our results indicate that the use of features located in the center of the face, the lower left portion of the nose area and the center of the mouth, are significantly associated with individual efficiency to generalize a face's identity across different viewpoints. However, as faces become more familiar, the reliance on this area decreases, while the diagnosticity of the eye region increases. This suggests that a certain distinction can be made between the visual mechanisms subtending viewpoint invariance and face recognition in the case of unfamiliar face identification. Our results further support the idea that the eye area may only come into play when the face stimulus is particularly familiar to the observer.

The Role of Familiarity on Viewpoint Adaptation for Self-Face and Other-Face Images

An adaptation method was used to investigate whether self-face processing is dissociable from general face processing. We explored the viewpoint aftereffect with face images having different degrees of familiarity (never-before-seen faces, recently familiarized faces, personally familiar faces, and the participant’s own face). A face viewpoint aftereffect occurs after prolonged viewing of a face viewed from one side, with the result that the perceived viewing direction of a subsequently presented face image shown near the frontal view is biased in a direction which is the opposite of the adapting orientation. We found that (1) the magnitude of the viewpoint aftereffect depends on the level of familiarity of the adapting and test faces, (2) a cross-identity transfer of the viewpoint aftereffect is found between all categories of faces, but not between an unfamiliar adaptor face and the self-face test, and (3) learning affects the processing of the self-face in greater measure than any other category of faces. These results highlight the importance of familiarity on the face aftereffects, but they also suggest the possibility of separate representations for the self-face, on the one side, and for highly familiar faces, on the other.

ACC and IPL networks in the perception of the faces of parents during selective tasks

The perception of the faces of parents is different from that of other faces. Mother's face perception is associated with childbearing and parenting and has specific cognitive processing properties. Yet, the underlying mechanism remains unknown. We investigated the neural network that is involved in the perception of parent's face based upon parental face selective tasks that were completed by 23 normal adults. Resting state functional magnetic resonance imaging (fMRI) and a task block design with fixation cross sign control fMRI were used. The faces activated the right inferior frontal gyrus, the left and right middle frontal gyri, and the left and right inferior parietal lobule. In addition, the parental faces task resulted in more activation than the control tasks in the left and right anterior cingulate, left middle (superior) temporal gyrus, right and left inferior frontal gyrus. The anterior cingulate cortex (ACC) and the inferior parietal lobe (IPL) are central nodes of the brain network, while the parental face perception network includes the anterior cingulate cortex and superior temporal gyrus (ACC-STG)/parahippocampal network as well as the inferior frontal and inferior parietal lobule (IFG-IPL) network. The network of father's face perception involves the left inferior parietal lobule and left middle frontal gyrus/right middle frontal gyrus/right inferior frontal gyrus, while mother's face perception involves the right inferior parietal lobe and frontal network (MFG.L/IFG.R). The experiments showed that the ACC-STG/parahippocampal network and IFG-IPL network are neural networks used in parental face perception, and the ACC and IPL are central nodes in the network. The neural pathway of parental face perception is similar to the perception of a stranger's face, as both include the STG, but is different in that the perception of the stranger's face involves a connected STG and IPL as a specific neural pathway.

Effects of exposure to facial expression variation in face learning and recognition

Facial expression is a major source of image variation in face images. Linking numerous expressions to the same face can be a huge challenge for face learning and recognition. It remains largely unknown what level of exposure to this image variation is critical for expression-invariant face recognition. We examined this issue in a recognition memory task, where the number of facial expressions of each face being exposed during a training session was manipulated. Faces were either trained with multiple expressions or a single expression, and they were later tested in either the same or different expressions. We found that recognition performance after learning three emotional expressions had no improvement over learning a single emotional expression (Experiments 1 and 2). However, learning three emotional expressions improved recognition compared to learning a single neutral expression (Experiment 3). These findings reveal both the limitation and the benefit of multiple exposures to variations of emotional expression in achieving expression-invariant face recognition. The transfer of expression training to a new type of expression is likely to depend on a relatively extensive level of training and a certain degree of variation across the types of expressions.

Face recognition in schizophrenia disorder: A comprehensive review of behavioral, neuroimaging and neurophysiological studies

Facial emotion processing has been extensively studied in schizophrenia patients while general face processing has received less attention. The already published reviews do not address the current scientific literature in a complete manner. Therefore, here we tried to answer some questions that remain to be clarified, particularly: are the non-emotional aspects of facial processing in fact impaired in schizophrenia patients? At the behavioral level, our key conclusions are that visual perception deficit in schizophrenia patients: are not specific to faces; are most often present when the cognitive (e.g. attention) and perceptual demands of the tasks are important; and seems to worsen with the illness chronification. Although, currently evidence suggests impaired second order configural processing, more studies are necessary to determine whether or not holistic processing is impaired in schizophrenia patients. Neural and neurophysiological evidence suggests impaired earlier levels of visual processing, which might involve the deficits in interaction of the magnocellular and parvocellular pathways impacting on further processing. These deficits seem to be present even before the disorder out-set. Although evidence suggests that this deficit may be not specific to faces, further evidence on this question is necessary, in particularly more ecological studies including context and body processing.

Successful decoding of famous faces in the fusiform face area

What are the neural mechanisms of face recognition? It is believed that the network of face-selective areas, which spans the occipital, temporal, and frontal cortices, is important in face recognition. A number of previous studies indeed reported that face identity could be discriminated based on patterns of multivoxel activity in the fusiform face area and the anterior temporal lobe. However, given the difficulty in localizing the face-selective area in the anterior temporal lobe, its role in face recognition is still unknown. Furthermore, previous studies limited their analysis to occipito-temporal regions without testing identity decoding in more anterior face-selective regions, such as the amygdala and prefrontal cortex. In the current high-resolution functional Magnetic Resonance Imaging study, we systematically examined the decoding of the identity of famous faces in the temporo-frontal network of face-selective and adjacent non-face-selective regions. A special focus has been put on the face-area in the anterior temporal lobe, which was reliably localized using an optimized scanning protocol. We found that face-identity could be discriminated above chance level only in the fusiform face area. Our results corroborate the role of the fusiform face area in face recognition. Future studies are needed to further explore the role of the more recently discovered anterior face-selective areas in face recognition.

Spatiotemporal changes in neural response patterns to faces varying in visual familiarity

Increasing experience with a previously unfamiliar face improves human ability to recognize it in challenging and novel viewing conditions. Differential neural responses to familiar versus unfamiliar faces in multiple regions of the ventral-temporal and parietal cortex have been reported in previous work, but with limited attention to how behavioral and neural measures change with increasing familiarity. We examined changes in the spatial and temporal characteristics of neural response patterns elicited by faces that vary in their degree of visual familiarity. First, we developed a behavioral paradigm to familiarize participants to low-, medium-, and high-levels of familiarity with faces. Recognition of novel, naturalistic images of the learned individuals improved with increasing familiarity with faces. Next, a new set of participants learned faces using the behavioral paradigm, outside the fMRI scanner, and subsequently viewed blocks of whole-body images of the learned and novel people, inside the scanner. We found that the face-selective FFA and OFA, and a combination of the ventral-temporal areas (e.g., fusiform gyrus) and parietal areas (e.g., precuneus) contained patterns useful for classifying highly familiar versus unfamiliar faces. Classification along the temporal-sequence of the face blocks revealed an early separation of neural patterns elicited in response to highly familiar versus unfamiliar faces in the FFA and OFA, but not in other regions of interest. This indicates the potential for a rapid assessment of the "known versus unknown" status of faces in core face-selective regions of the brain. The present study provides a first look at the perceptual and neural correlates underlying experience gains with faces as they become familiar.

The social network-network: size is predicted by brain structure and function in the amygdala and paralimbic regions

The social brain hypothesis proposes that the large size of the primate neocortex evolved to support complex and demanding social interactions. Accordingly, recent studies have reported correlations between the size of an individual's social network and the density of gray matter (GM) in regions of the brain implicated in social cognition. However, the reported relationships between GM density and social group size are somewhat inconsistent with studies reporting correlations in different brain regions. One factor that might account for these discrepancies is the use of different measures of social network size (SNS). This study used several measures of SNS to assess the relationships SNS and GM density. The second goal of this study was to test the relationship between social network measures and functional brain activity. Participants performed a social closeness task using photos of their friends and unknown people. Across the VBM and functional magnetic resonance imaging analyses, individual differences in SNS were consistently related to structural and functional differences in three regions: the left amygdala, right amygdala and the right entorhinal/ventral anterior temporal cortex.

Beyond the FFA: The role of the ventral anterior temporal lobes in face processing

Extensive research has supported the existence of a specialized face-processing network that is distinct from the visual processing areas used for general object recognition. The majority of this work has been aimed at characterizing the response properties of the fusiform face area (FFA) and the occipital face area (OFA), which together are thought to constitute the core network of brain areas responsible for facial identification. Although accruing evidence has shown that face-selective patches in the ventral anterior temporal lobes (vATLs) are interconnected with the FFA and OFA, and that they play a role in facial identification, the relative contribution of these brain areas to the core face-processing network has remained unarticulated. Here we review recent research critically implicating the vATLs in face perception and memory. We propose that current models of face processing should be revised such that the ventral anterior temporal lobes serve a centralized role in the visual face-processing network. We speculate that a hierarchically organized system of face processing areas extends bilaterally from the inferior occipital gyri to the vATLs, with facial representations becoming increasingly complex and abstracted from low-level perceptual features as they move forward along this network. The anterior temporal face areas may serve as the apex of this hierarchy, instantiating the final stages of face recognition. We further argue that the anterior temporal face areas are ideally suited to serve as an interface between face perception and face memory, linking perceptual representations of individual identity with person-specific semantic knowledge.

Internal representations for face detection: an application of noise-based image classification to BOLD responses

What basic visual structures underlie human face detection and how can we extract such structures directly from the amplitude of neural responses elicited by face processing? Here, we address these issues by investigating an extension of noise-based image classification to BOLD responses recorded in high-level visual areas. First, we assess the applicability of this classification method to such data and, second, we explore its results in connection with the neural processing of faces. To this end, we construct luminance templates from white noise fields based on the response of face-selective areas in the human ventral cortex. Using behaviorally and neurally-derived classification images, our results reveal a family of simple but robust image structures subserving face representation and detection. Thus, we confirm the role played by classical face selective regions in face detection and we help clarify the representational basis of this perceptual function. From a theory standpoint, our findings support the idea of simple but highly diagnostic neurally-coded features for face detection. At the same time, from a methodological perspective, our work demonstrates the ability of noise-based image classification in conjunction with fMRI to help uncover the structure of high-level perceptual representations.

Task- and experience-dependent cortical selectivity to features informative for categorization

In this study, we bridge the gap between monkey electrophysiological recordings that showed selective responses to informative features and human fMRI data that demonstrated increased and selective responses to trained objects. Human participants trained with computer-generated fish stimuli. For each participant, two features of the fish were informative for category membership and two features were uninformative. After training, participants showed higher perceptual sensitivity to the informative dimensions. An fMRI adaptation paradigm revealed that during categorization the right inferior frontal gyrus and occipitotemporal cortex were selectively responsive to the informative features. These selective cortical responses were experience dependent; they were not present for the entire trained object, but specific for those features that were informative for categorization. Responses in the inferior frontal gyrus showed category selectivity. Moreover, selectivity to the informative features correlated with performance on the categorization task during scanning. This all suggests that the frontal cortex is involved in actively categorizing objects and that it uses informative features to do so while ignoring those features that do not contribute category information. Occipitotemporal cortex also showed selectivity to the informative features during the categorization task. Interestingly, this area showed a positive correlation of performance during training and selectivity to the informative features and a negative correlation with selectivity to the uninformative features. This indicates that training enhanced sensitivity to trained items and decreased sensitivity to uninformative features. The absence of sensitivity for informative features during a color change detection task indicates that there is a strong component of task-related processing of these features.

Decoding representations of face identity that are tolerant to rotation

In order to recognize the identity of a face we need to distinguish very similar images (specificity) while also generalizing identity information across image transformations such as changes in orientation (tolerance). Recent studies investigated the representation of individual faces in the brain, but it remains unclear whether the human brain regions that were found encode representations of individual images (specificity) or face identity (specificity plus tolerance). In the present article, we use multivoxel pattern analysis in the human ventral stream to investigate the representation of face identity across rotations in depth, a kind of transformation in which no point in the face image remains unchanged. The results reveal representations of face identity that are tolerant to rotations in depth in occipitotemporal cortex and in anterior temporal cortex, even when the similarity between mirror symmetrical views cannot be used to achieve tolerance. Converging evidence from different analysis techniques shows that the right anterior temporal lobe encodes a comparable amount of identity information to occipitotemporal regions, but this information is encoded over a smaller extent of cortex.

Anterior temporal face patches: a meta-analysis and empirical study

Evidence suggests the anterior temporal lobe (ATL) plays an important role in person identification and memory. In humans, neuroimaging studies of person memory report consistent activations in the ATL to famous and personally familiar faces and studies of patients report resection or damage of the ATL causes an associative prosopagnosia in which face perception is intact but face memory is compromised. In addition, high-resolution fMRI studies of non-human primates and electrophysiological studies of humans also suggest regions of the ventral ATL are sensitive to novel faces. The current study extends previous findings by investigating whether similar subregions in the dorsal, ventral, lateral, or polar aspects of the ATL are sensitive to personally familiar, famous, and novel faces. We present the results of two studies of person memory: a meta-analysis of existing fMRI studies and an empirical fMRI study using optimized imaging parameters. Both studies showed left-lateralized ATL activations to familiar individuals while novel faces activated the right ATL. Activations to famous faces were quite ventral, similar to what has been reported in previous high-resolution fMRI studies of non-human primates. These findings suggest that face memory-sensitive patches in the human ATL are in the ventral/polar ATL.

Early visual experience and the recognition of basic facial expressions: involvement of the middle temporal and inferior frontal gyri during haptic identification by the early blind

Face perception is critical for social communication. Given its fundamental importance in the course of evolution, the innate neural mechanisms can anticipate the computations necessary for representing faces. However, the effect of visual deprivation on the formation of neural mechanisms that underlie face perception is largely unknown. We previously showed that sighted individuals can recognize basic facial expressions by haptics surprisingly well. Moreover, the inferior frontal gyrus (IFG) and posterior superior temporal sulcus (pSTS) in the sighted subjects are involved in haptic and visual recognition of facial expressions. Here, we conducted both psychophysical and functional magnetic-resonance imaging (fMRI) experiments to determine the nature of the neural representation that subserves the recognition of basic facial expressions in early blind individuals. In a psychophysical experiment, both early blind and sighted subjects haptically identified basic facial expressions at levels well above chance. In the subsequent fMRI experiment, both groups haptically identified facial expressions and shoe types (control). The sighted subjects then completed the same task visually. Within brain regions activated by the visual and haptic identification of facial expressions (relative to that of shoes) in the sighted group, corresponding haptic identification in the early blind activated regions in the inferior frontal and middle temporal gyri. These results suggest that the neural system that underlies the recognition of basic facial expressions develops supramodally even in the absence of early visual experience.