Face processing in humans is compatible with a simple shape-based model of vision

Effects of Natural Scene Inversion on Visual-evoked Brain Potentials and Pupillary Responses: A Matter of Effortful Processing of Unfamiliar Configurations

The inversion of a picture of a face hampers the accuracy and speed at which observers can perceptually process it. Event-related potentials and pupillary responses, successfully used as biomarkers of face inversion in the past, suggest that the perception of visual features, that are organized in an unfamiliar manner, recruits demanding additional processes. However, it remains unclear whether such inversion effects generalize beyond face stimuli and whether indeed more mental effort is needed to process inverted images. Here we aimed to study the effects of natural scene inversion on visual evoked potentials and pupil dilations. We simultaneously measured responses of 47 human participants to presentations of images showing upright or inverted natural scenes. For inverted scenes, we observed relatively stronger occipito-temporo-parietal N1 peak amplitudes and larger pupil dilations (on top of an initial orienting response) than for upright scenes. This study revealed neural and physiological markers of natural scene inversion that are in line with inversion effects of other stimulus types and demonstrates the robustness and generalizability of the phenomenon that unfamiliar configurations of visual content require increased processing effort.

Local features drive identity responses in macaque anterior face patches

Humans and other primates recognize one another in part based on unique structural details of the face, including both local features and their spatial configuration within the head and body. Visual analysis of the face is supported by specialized regions of the primate cerebral cortex, which in macaques are commonly known as face patches. Here we ask whether the responses of neurons in anterior face patches, thought to encode face identity, are more strongly driven by local or holistic facial structure. We created stimuli consisting of recombinant photorealistic images of macaques, where we interchanged the eyes, mouth, head, and body between individuals. Unexpectedly, neurons in the anterior medial (AM) and anterior fundus (AF) face patches were predominantly tuned to local facial features, with minimal neural selectivity for feature combinations. These findings indicate that the high-level structural encoding of face identity rests upon populations of neurons specialized for local features.

Anterior face patches in the macaque have been assumed to represent face identity in a holistic manner. Here the authors show that the neural encoding of face identity in the anterior medial and anterior fundus face patches are instead driven principally by local features.

Configural properties of face portraits change between childhood and adulthood

Adult observers are sensitive to the configuration of facial features within a face, able to distinguish between relative differences in feature spacing, and detecting deviations from typical facial appearance. How does the representation of the typical configuration of facial features develop? While there is a great deal of work describing children’s developing abilities to detect differences in feature spacing across face images, there is substantially less work examining what children think constitutes a typical arrangement of facial features. In the current study, we investigated this issue using a production task in which adults and 5- to 10-year-old children created a face “portrait” by arranging the eyes, nose, and mouth of a standard face within an empty outline. Using this simple task, we found differences in face configuration across age groups, such that children of all ages made far larger errors than adult participants, expanding facial features outward from the center of the face more than adults. These results were not affected by face inversion, potentially implying a domain-general rather than face-specific process. We also found that children of all ages endorsed the correct configuration as a best likeness in a perceptual task. We discuss these results in terms of ongoing debate regarding the extent to which configural processing is a meaningful component of face recognition and the conclusions we can draw from production paradigms as compared to purely perceptual tasks.

Whole-object effects in visual word processing: Parallels with and differences from face recognition

Visual words and faces differ in their structural properties, but both are objects of high expertise. Holistic processing is said to characterize expert face recognition, but the extent to which whole-word processes contribute to word recognition is unclear, particularly as word recognition is thought to proceed by a component-based process. We review the evidence for experimental effects in word recognition that parallel those used to support holistic face processing, namely inversion effects, the part-whole task, and composite effects, as well as the status of whole-word processing in pure alexia and developmental dyslexia, contrasts between familiar and unfamiliar languages, and the differences between handwriting and typeset font. The observations support some parallels in whole-object influences between face and visual word recognition, but do not necessarily imply similar expert mechanisms. It remains to be determined whether and how the relative balance between part-based and whole-object processing differs for visual words and faces.

Orientation-invariance of individual differences in three face processing tasks

Numerous studies have reported impairments in perception and recognition, and, particularly, in part-integration of faces following picture-plane inversion. Whether these findings support the notion that inversion changes face processing qualitatively remains a topic of debate. To examine whether associations and dissociations of the human face processing ability depend on stimulus orientation, we measured face recognition with the Cambridge Face Memory Test (CFMT), along with experimental tests of face perception and selective attention to faces and non-face objects in a sample of 314 participants. Results showed strong inversion effects for all face-related tasks, and modest ones for non-face objects. Individual differences analysis revealed that the CFMT shared common variance with face perception and face-selective attention, however, independent of orientation. Regardless of whether predictor and criterion had same or different orientation, face recognition was best predicted by the same test battery. Principal component decomposition revealed a common factor for face recognition and face perception, a second common factor for face recognition and face-selective attention, and two unique factors. The patterns of factor loadings were nearly identical for upright and inverted presentation. These results indicate orientation-invariance of common variance in three domains of face processing. Since inversion impaired performance, but did not affect domain-related associations and dissociations, the findings suggest process-specific but orientation-general mechanisms. Specific limitations by constraints of individual differences analysis and test selection are discussed.

Inversion Impairs Expert Budgerigar Identity Recognition: A Face-Like Effect for a Nonface Object of Expertise

The face-inversion effect is the finding that picture-plane inversion disproportionately impairs face recognition compared to object recognition and is now attributed to greater orientation-sensitivity of holistic processing for faces but not common objects. Yet, expert dog judges have showed similar recognition deficits for inverted dogs and inverted faces, suggesting that holistic processing is not specific to faces but to the expert recognition of perceptually similar objects. Although processing changes in expert object recognition have since been extensively documented, no other studies have observed the distinct recognition deficits for inverted objects-of-expertise that people as face experts show for faces. However, few studies have examined experts who recognize individual objects similar to how people recognize individual faces. Here we tested experts who recognize individual budgerigar birds. The effect of inversion on viewpoint-invariant budgerigar and face recognition was compared for experts and novices. Consistent with the face-inversion effect, novices showed recognition deficits for inverted faces but not for inverted budgerigars. By contrast, experts showed equal recognition deficits for inverted faces and budgerigars. The results are consistent with the hypothesis that processes underlying the face-inversion effect are specific to the expert individuation of perceptually similar objects.

On Response Bias in the Face Congruency Effect for Internal and External Features

Some years ago Cheung et al. (2008) proposed the complete design (CD) for measuring the failure of selective attention in composite objects. Since the CD is a fully balanced design, analysis of response bias may reveal potential effects of the experimental manipulation, the stimulus material, and/or attributes of the observers. Here we used the CD to prove whether external features modulate perception of internal features with the context congruency paradigm (Nachson et al., 1995; Meinhardt-Injac et al., 2010) in a larger sample of N = 303 subjects. We found a large congruency effect (Cohen's d = 1.78), which was attenuated by face inversion (d = 1.32). The congruency relation also strongly modulated response bias. In incongruent trials the proportion of "different" responses was much larger than in congruent trials (d = 0.79), which was again attenuated by face inversion (d = 0.43). Because in incongruent trials the wholes formed by integrating external and internal features are always different, while in congruent trials same and different wholes occur with the same frequency, a congruency related bias effect is expected from holistic integration. Our results suggest two behavioral markers of holistic processing in the context congruency paradigm: a performance advantage in congruent compared to incongruent trials, and a tendency toward more "different" responses in incongruent, compared to congruent trials. Since the results for both markers differed only quantitatively in upright and inverted presentation, our findings indicate no change of the face processing mode by picture plane rotation. A potential transfer to the composite face paradigm is discussed.

A new approach to the diagnosis of deficits in processing faces: Potential application in autism research

Deficits in social communication are one of the behavioral signatures of autism spectrum disorder (ASD). Because faces are arguably the most important social stimuli that we encounter in everyday life, investigating the ability of individuals with ASD to process faces is thought to be important for understanding the nature of ASD. However, although a considerable body of evidence suggests that ASD individuals show specific impairments in face processing, a significant number of studies argue otherwise. Through a literature review, we found that this controversy is largely attributable to the different face tests used across different studies. Therefore, a more reliable and valid face test is needed. To this end, we performed a meta-analysis on data gleaned from a variety of face tests conducted on individuals with developmental prosopagnosia (DP) who suffer a selective deficit in face processing. Based on this meta-analysis, we selected an old/new face recognition test that relies on face memory as a standard diagnostic test for measuring specific face processing deficits. This test not only reliably reflects DP individuals' subjective experiences with faces in their daily lives, but also effectively differentiates deficits in face processing from deficits caused by other general problems. In addition, DP individuals' performance in this test predicts their performance in a variety of face tests that examine specific components of face processing (e.g., holistic processing of faces). Finally, this test can be easily administrated and is not overly sensitive to prior knowledge. In summary, this test can be used to evaluate face-processing ability, and it helped to resolve the controversy whether individuals with ASD exhibit face-processing deficits.

Faces in Context: Does Face Perception Depend on the Orientation of the Visual Scene?

The mechanisms held responsible for familiar face recognition are thought to be orientation dependent; inverted faces are more difficult to recognize than their upright counterparts. Although this effect of inversion has been investigated extensively, researchers have typically sliced faces from photographs and presented them in isolation. As such, it is not known whether the perceived orientation of a face is inherited from the visual scene in which it appears. Here, we address this question by measuring performance in a simultaneous same–different task while manipulating both the orientation of the faces and the scene. We found that the face inversion effect survived scene inversion. Nonetheless, an improvement in performance when the scene was upside down suggests that sensitivity to identity increased when the faces were more easily segmented from the scene. Thus, while these data identify congruency with the visual environment as a contributing factor in recognition performance, they imply different mechanisms operate on upright and inverted faces.

A specialized face-processing model inspired by the organization of monkey face patches explains several face-specific phenomena observed in humans

Converging reports indicate that face images are processed through specialized neural networks in the brain –i.e. face patches in monkeys and the fusiform face area (FFA) in humans. These studies were designed to find out how faces are processed in visual system compared to other objects. Yet, the underlying mechanism of face processing is not completely revealed. Here, we show that a hierarchical computational model, inspired by electrophysiological evidence on face processing in primates, is able to generate representational properties similar to those observed in monkey face patches (posterior, middle and anterior patches). Since the most important goal of sensory neuroscience is linking the neural responses with behavioral outputs, we test whether the proposed model, which is designed to account for neural responses in monkey face patches, is also able to predict well-documented behavioral face phenomena observed in humans. We show that the proposed model satisfies several cognitive face effects such as: composite face effect and the idea of canonical face views. Our model provides insights about the underlying computations that transfer visual information from posterior to anterior face patches.

The "parts and wholes" of face recognition: A review of the literature

It has been claimed that faces are recognized as a "whole" rather than by the recognition of individual parts. In a paper published in the Quarterly Journal of Experimental Psychology in 1993, Martha Farah and I attempted to operationalize the holistic claim using the part/whole task. In this task, participants studied a face and then their memory presented in isolation and in the whole face. Consistent with the holistic view, recognition of the part was superior when tested in the whole-face condition compared to when it was tested in isolation. The "whole face" or holistic advantage was not found for faces that were inverted, or scrambled, nor for non-face objects, suggesting that holistic encoding was specific to normal, intact faces. In this paper, we reflect on the part/whole paradigm and how it has contributed to our understanding of what it means to recognize a face as a "whole" stimulus. We describe the value of part/whole task for developing theories of holistic and non-holistic recognition of faces and objects. We discuss the research that has probed the neural substrates of holistic processing in healthy adults and people with prosopagnosia and autism. Finally, we examine how experience shapes holistic face recognition in children and recognition of own- and other-race faces in adults. The goal of this article is to summarize the research on the part/whole task and speculate on how it has informed our understanding of holistic face processing.

There Is a "U" in Clutter: Evidence for Robust Sparse Codes Underlying Clutter Tolerance in Human Vision

The ability to recognize objects in clutter is crucial for human vision, yet the underlying neural computations remain poorly understood. Previous single-unit electrophysiology recordings in inferotemporal cortex in monkeys and fMRI studies of object-selective cortex in humans have shown that the responses to pairs of objects can sometimes be well described as a weighted average of the responses to the constituent objects. Yet, from a computational standpoint, it is not clear how the challenge of object recognition in clutter can be solved if downstream areas must disentangle the identity of an unknown number of individual objects from the confounded average neuronal responses. An alternative idea is that recognition is based on a subpopulation of neurons that are robust to clutter, i.e., that do not show response averaging, but rather robust object-selective responses in the presence of clutter. Here we show that simulations using the HMAX model of object recognition in cortex can fit the aforementioned single-unit and fMRI data, showing that the averaging-like responses can be understood as the result of responses of object-selective neurons to suboptimal stimuli. Moreover, the model shows how object recognition can be achieved by a sparse readout of neurons whose selectivity is robust to clutter. Finally, the model provides a novel prediction about human object recognition performance, namely, that target recognition ability should show a U-shaped dependency on the similarity of simultaneously presented clutter objects. This prediction is confirmed experimentally, supporting a simple, unifying model of how the brain performs object recognition in clutter.

SIGNIFICANCE STATEMENT

The neural mechanisms underlying object recognition in cluttered scenes (i.e., containing more than one object) remain poorly understood. Studies have suggested that neural responses to multiple objects correspond to an average of the responses to the constituent objects. Yet, it is unclear how the identities of an unknown number of objects could be disentangled from a confounded average response. Here, we use a popular computational biological vision model to show that averaging-like responses can result from responses of clutter-tolerant neurons to suboptimal stimuli. The model also provides a novel prediction, that human detection ability should show a U-shaped dependency on target-clutter similarity, which is confirmed experimentally, supporting a simple, unifying account of how the brain performs object recognition in clutter.

The influence of natural contour and face size on the spatial frequency tuning for identifying upright and inverted faces

It has previously been proposed that holistic face processing is based on low spatial frequencies (SFs) whereas featural processing relies on higher SFs, a hypothesis still widespread in the face processing literature today (e.g. Peters et al. in Eur J Neurosci 37(9):1448-1457, 2013). Since upright faces are supposedly recognized through holistic processing and inverted faces, using features, it is easy to take the leap to suggest a qualitatively different SF tuning for the identification of upright and vs. inverted faces. However, two independent studies (e.g. Gaspar et al. in Vision Res 48(28):2817-2826, 2008; Willenbockel et al. in J Exp Psychol Human 36(1):122-135, 2010a) found the same SF tuning for both stimulus presentations. Since these authors used relatively small faces hiding the natural facial contour, it is possible that differences in the SF tuning for identifying upright and inverted faces were missed. The present study thus revisits the SF tuning for upright and inverted faces face identification using the SF Bubbles technique. Our results still indicate that the same SFs are involved in both upright and inverted face recognition regardless of these additional parameters (contour and size), thus contrasting with previous data obtained using different methods (e.g. Oruc and Barton in J Vis 10(12):20, 1-12, 2010). The possible reasons subtending this divergence are discussed.

The sensitivity to replacement and displacement of the eyes region in early adolescence, young and later adulthood

Recent evidence suggests a rather gradual developmental trajectory for processing vertical relational face information, lasting well into late adolescence (de Heering and Schlitz, 2008). Results from another recent study (Tanaka et al., 2014) indicate that children and young adolescents use a smaller spatial integration field for faces than do adults, which particularly affects assessment of long-range vertical relations. Here we studied sensitivity to replacement of eyes and eyebrows (F), variation of inter-eye distance (H), and eye height (V) in young adolescents (11-12 years), young (21-25 years), and middle-age adults (51-62 years). In order to provide a baseline for potential age effects the sensitivity to all three types of face manipulations was calibrated to equal levels for the young adults group. Both the young adolescents and the middle-age adults showed substantially lower sensitivity compared to young adults, but only the young adolescents had selective impairment for V relational changes. Their inversion effects were at similar levels for all types of face manipulations, while in both adult groups the inversion effects for V were considerably stronger than for H or F changes. These results suggest that young adolescents use a limited spatial integration field for faces, and have not reached a mature state in processing vertical configural cues. The H-V asymmetry of inversion effects found for both adult groups indicates that adults integrate across the whole face when they view upright stimuli. However, the notably lower sensitivity of middle-age adults for all types of face manipulations, which was accompanied by a strong general "same" bias, suggests early age-related decline in attending cues for facial difference.

Developmental changes in analytic and holistic processes in face perception

Although infants demonstrate sensitivity to some kinds of perceptual information in faces, many face capacities continue to develop throughout childhood. One debate is the degree to which children perceive faces analytically versus holistically and how these processes undergo developmental change. In the present study, school-aged children and adults performed a perceptual matching task with upright and inverted face and house pairs that varied in similarity of featural or 2(nd) order configural information. Holistic processing was operationalized as the degree of serial processing when discriminating faces and houses [i.e., increased reaction time (RT), as more features or spacing relations were shared between stimuli]. Analytical processing was operationalized as the degree of parallel processing (or no change in RT as a function of greater similarity of features or spatial relations). Adults showed the most evidence for holistic processing (most strongly for 2(nd) order faces) and holistic processing was weaker for inverted faces and houses. Younger children (6-8 years), in contrast, showed analytical processing across all experimental manipulations. Older children (9-11 years) showed an intermediate pattern with a trend toward holistic processing of 2(nd) order faces like adults, but parallel processing in other experimental conditions like younger children. These findings indicate that holistic face representations emerge around 10 years of age. In adults both 2(nd) order and featural information are incorporated into holistic representations, whereas older children only incorporate 2(nd) order information. Holistic processing was not evident in younger children. Hence, the development of holistic face representations relies on 2(nd) order processing initially then incorporates featural information by adulthood.

Face perception and processing in early infancy: inborn predispositions and developmental changes

From birth it is critical for our survival to identify social agents and conspecifics. Among others stimuli, faces provide the required information. The present paper will review the mechanisms subserving face detection and face recognition, respectively, over development. In addition, the emergence of the functional and neural specialization for face processing as an experience-dependent process will be documented. Overall, the present work highlights the importance of both inborn predispositions and the exposure to certain experiences, shortly after birth, to drive the system to become functionally specialized to process faces in the first months of life.

An other-race effect for configural and featural processing of faces: upper and lower face regions play different roles

We examined whether Asian individuals would show differential sensitivity to configural vs. featural changes to own- and other-race faces and whether such sensitivity would depend on whether the changes occurred in the upper vs. lower regions of the faces. We systematically varied the size of key facial features (eyes and mouth) of own-race Asian faces and other-race Caucasian faces, and the configuration (spacing) between the eyes and between the nose and mouth of the two types of faces. Results revealed that the other-race effect (ORE) is more pronounced when featural and configural spacing changes are in the upper region than in the lower region of the face. These findings reveal that information from the upper vs. lower region of the face contributes differentially to the ORE in face processing, and that processing of face race is influenced more by information location (i.e., upper vs. lower) than by information type (i.e., configural vs. featural).

The complete design in the composite face paradigm: role of response bias, target certainty, and feedback

Some years ago an improved design (the “complete design”) was proposed to assess the composite face effect in terms of a congruency effect, defined as the performance difference for congruent and incongruent target to no-target relationships (Cheung et al., 2008). In a recent paper Rossion (2013) questioned whether the congruency effect was a valid hallmark of perceptual integration, because it may contain confounds with face-unspecific interference effects. Here we argue that the complete design is well-balanced and allows one to separate face-specific from face-unspecific effects. We used the complete design for a same/different composite stimulus matching task with face and non-face objects (watches). Subjects performed the task with and without trial-by-trial feedback, and with low and high certainty about the target half. Results showed large congruency effects for faces, particularly when subjects were informed late in the trial about which face halves had to be matched. Analysis of response bias revealed that subjects preferred the “different” response in incongruent trials, which is expected when upper and lower face halves are integrated perceptually at the encoding stage. The results pattern was observed in the absence of feedback, while providing feedback generally attenuated the congruency effect, and led to an avoidance of response bias. For watches no or marginal congruency effects and a moderate global “same” bias were observed. We conclude that the congruency effect, when complemented by an evaluation of response bias, is a valid hallmark of feature integration that allows one to separate faces from non-face objects.

Development of holistic vs. featural processing in face recognition

According to a classic view developed by Carey and Diamond (1977), young children process faces in a piecemeal fashion before adult-like holistic processing starts to emerge at the age of around 10 years. This is known as the encoding switch hypothesis. Since then, a growing body of studies have challenged the theory. This article will provide a critical appraisal of this literature, followed by an analysis of some more recent developments. We will conclude, quite contrary to the classical view, that holistic processing is not only present in early child development, but could even precede the development of part-based processing.

Perceiving crowd attention: ensemble perception of a crowd's gaze

In nearly every interpersonal encounter, people readily gather socio-visual cues to guide their behavior. Intriguingly, social information is most effective in directing behavior when it is perceived in crowds. For example, the shared gaze of a crowd is more likely to direct attention than is a single person's gaze. Are people equipped with mechanisms to perceive a crowd's gaze as an ensemble? Here, we provide the first evidence that the visual system extracts a summary representation of a crowd's attention; observers rapidly pooled information from multiple crowd members to perceive the direction of a group's collective gaze. This pooling occurred in high-level stages of visual processing, with gaze perceived as a global-level combination of information from head and pupil rotation. These findings reveal an important and efficient mechanism for assessing crowd gaze, which could underlie the ability to perceive group intentions, orchestrate joint attention, and guide behavior.

A meta-analysis and review of holistic face processing

The concept of holistic processing is a cornerstone of face recognition research, yet central questions related to holistic processing remain unanswered, and debates have thus far failed to reach a resolution despite accumulating empirical evidence. We argue that a considerable source of confusion in this literature stems from a methodological problem. Specifically, 2 measures of holistic processing based on the composite paradigm (complete design and partial design) are used in the literature, but they often lead to qualitatively different results. First, we present a comprehensive review of the work that directly compares the 2 designs, and which clearly favors the complete design over the partial design. Second, we report a meta-analysis of holistic face processing according to both designs and use this as further evidence for one design over the other. The meta-analysis effect size of holistic processing in the complete design is nearly 3 times that of the partial design. Effect sizes were not correlated between measures, consistent with the suggestion that they do not measure the same thing. Our meta-analysis also examines the correlation between conditions in the complete design of the composite task, and suggests that in an individual differences context, little is gained by including a misaligned baseline. Finally, we offer a comprehensive review of the state of knowledge about holistic processing based on evidence gathered from the measure we favor based on the 1st sections of our review-the complete design-and outline outstanding research questions in that new context.

The face inversion effect in opponent-stimulus rivalry

The face inversion effect is regarded as a hallmark of face-specific processing, and can be observed in a large variety of visual tasks. Face inversion effects are also reported in binocular rivalry. However, it is unclear whether these effects are face-specific, and distinct from the general tendency of visual awareness to privilege upright objects. We studied continuous rivalry across more than 600 dominance epochs for each observer, having faces and houses rival against their inverted counterparts, and letting faces rival against houses in both upright and inverted orientation. We found strong inversion effects for faces and houses in both the frequency of dominance epochs and their duration. Inversion effects for faces, however, were substantially larger, reaching a 70:30 distribution of dominance times for upright versus inverted faces, while a 60:40 distribution was obtained for upright versus inverted houses. Inversion effects for faces reached a Cohen's d of 0.85, compared to a value of 0.33 for houses. Dominance times for rivalry of faces against houses had a 60:40 distribution in favor of faces, independent of the orientation of the objects. These results confirm the general tendency of visual awareness to prefer upright objects, and demonstrate the outstanding role of faces. Since effect size measures clearly distinguish face stimuli in opponent-stimulus rivalry, the method is highly recommended for testing the effects of face manipulations against non-face reference objects.

Explaining the face-inversion effect: the face-scheme incompatibility (FSI) model

The face-inversion effect (FIE) can be viewed as being based on two kinds of findings. According to the face(UI) effect, perception and recognition are better for faces presented upright (U) than for faces presented inverted (I). According to the face/object(UI) effect, inversion impairs the processing of faces more than the processing of nonfacial objects (e.g., buildings or cars). Part I of this article focuses on the face(UI) effect and the configural-processing hypothesis, which is considered the most popular explanatory hypothesis of the FIE. In this hypothesis, it is proposed that inversion impairs the processing of configural information (the spatial relations between features) but hardly (if at all) impairs the processing of featural information (e.g., eyes, nose, and mouth). Part II of the article starts from the conclusion reached in part I, that the configural-processing hypothesis has not succeeded in explaining a substantial number of the findings and in resolving certain theoretical problems. The part then goes on to outline a new alternative model, the face-scheme incompatibility (FSI) model, which contends with these theoretical problems, accounts for the configural-processing hypothesis, succeeds in explaining a considerable portion of the empirical findings related to the face(UI) effect, and proposes a relatively new research program on the concept of the face scheme. The basic assumption of the FSI model is that schemes and prototypes are involved in processing a visual stimulus of a face and in transforming it to a "meaning-bearing" face, and that different schemes are involved if the face is presented upright or inverted.

The Face Inversion Effect—Parts and Wholes: Individual Features and Their Configuration

The face inversion effect (FIE) is a reduction in recognition performance for inverted faces (compared to upright faces) that is greater than that typically observed with other stimulus types (e.g., houses). The work of Diamond and Carey, suggests that a special type of configural information, “second-order relational information” is critical in generating this inversion effect. However, Tanaka and Farah concluded that greater reliance on second-order relational information did not directly result in greater sensitivity to inversion, and they suggested that the FIE is not entirely due to a reliance on this type of configural information. A more recent review by McKone and Yovel provides a meta-analysis that makes a similar point. In this paper, we investigated the contributions made by configural and featural information to the FIE. Experiments 1a and1b investigated the link between configural information and the FIE. Remarkably, Experiment 1b showed that disruption of all configural information of the type considered in Diamond and Carey's analysis (both first and second order) was effective in reducing recognition performance, but did not significantly impact on the FIE. Experiments 2 and 3 revealed that face processing is affected by the orientation of individual features and that this plays a major role in producing the FIE. The FIE was only completely eliminated when we disrupted the single feature orientation information in addition to the configural information, by using a new type of transformation similar to Thatcherizing our sets of scrambled faces. We conclude by noting that our results for scrambled faces are consistent with an account that has recognition performance entirely determined by the proportion of upright facial features within a stimulus, and that any ability to make use of the spatial configuration of these features seems to benefit upright and inverted normal faces alike.

Toward a unified model of face and object recognition in the human visual system

Our understanding of the mechanisms and neural substrates underlying visual recognition has made considerable progress over the past 30 years. During this period, accumulating evidence has led many scientists to conclude that objects and faces are recognised in fundamentally distinct ways, and in fundamentally distinct cortical areas. In the psychological literature, in particular, this dissociation has led to a palpable disconnect between theories of how we process and represent the two classes of object. This paper follows a trend in part of the recognition literature to try to reconcile what we know about these two forms of recognition by considering the effects of learning. Taking a widely accepted, self-organizing model of object recognition, this paper explains how such a system is affected by repeated exposure to specific stimulus classes. In so doing, it explains how many aspects of recognition generally regarded as unusual to faces (holistic processing, configural processing, sensitivity to inversion, the other-race effect, the prototype effect, etc.) are emergent properties of category-specific learning within such a system. Overall, the paper describes how a single model of recognition learning can and does produce the seemingly very different types of representation associated with faces and objects.

When Intuition Fails to Align with Data: A Reply to

Holistic processing, a hallmark of face perception, is often measured in the so-called composite paradigm, in which participants are asked to match part of a stimulus while ignoring another part. In prior work, we recommended against the use of one version of the composite task we call the partial design, on the basis of confounds with response biases. Rossion wrote a lengthy piece that reviews the work that he has published using this design, raising a large number of criticisms, both about an alternative measure of holistic processing that we have used and advocated (which we call the complete design) and about our work in general. In this reply, we have limited our discussion to those issues that would be relevant to a researcher looking to decide which version of this composite paradigm to use, as we doubt a comprehensive reply would be of significant interest outside a very small circle.

Sensitivity to Information Conveyed by Horizontal Contours is Correlated with Face Identification Accuracy

We measured thresholds in a 1-of-10 face identification task in which stimuli were embedded in orientation-filtered Gaussian noise. For upright faces, the threshold elevation produced by the masking noise varied as a function of noise orientation: significantly greater masking was obtained with horizontal noise than with vertical noise. However, the orientation selectivity of masking was significantly less with inverted faces. The performance of an ideal observer was qualitatively similar to human observers viewing upright faces: the masking function exhibited a peak for horizontally oriented noise although the selectivity of masking was greater than what was observed in human observers. These results imply that significantly more information about facial identity was conveyed by horizontal contours than by vertical contours, and that human observers use this information more efficiently to identify upright faces than inverted faces. We also found a significant positive correlation between selectivity for horizontal information and face identification accuracy for upright, but not inverted faces. Finally, there was a significant positive correlation between horizontal tuning and the size of the face inversion effect. These results demonstrate that the use of information conveyed by horizontal contours is associated with face identification accuracy and the magnitude of the face inversion effect.

First fixations in face processing: the more diagnostic they are the smaller the face-inversion effect

Hills, Ross, and Lewis (2011) introduced the concept that the face-inversion effect may, in part, be carried by the first feature attended to, since the first feature fixated upon is different for upright and inverted faces. An eye-tracking study that directly assesses this hypothesis by using fixation crosses to guide attention to the eye or mouth region of the to-be-presented upright and inverted faces was devised. Recognition was better when the fixation cross appeared at the eye region than at the mouth region. The face-inversion effect was smaller when the eyes were cued than when the mouth was cued or when there was no cueing. The eye-tracking measures confirmed that the fixation crosses attracted the first fixation but did not affect other measures of eye-movements. Furthermore, the location of the first fixation predicted recognition accuracy: when the first fixation was to the eyes, recognition accuracy was higher than when the first fixation was to the mouth, irrespective of facial orientation. The results suggest that the first facial feature attended to is more predictive of recognition accuracy than the face orientation in which they are presented.

The face inversion effect following pitch and yaw rotations: investigating the boundaries of holistic processing

Upright faces are thought to be processed holistically. However, the range of views within which holistic processing occurs is unknown. Recent research by McKone (2008) suggests that holistic processing occurs for all yaw-rotated face views (i.e., full-face through to profile). Here we examined whether holistic processing occurs for pitch, as well as yaw, rotated face views. In this face recognition experiment: (i) participants made same/different judgments about two sequentially presented faces (either both upright or both inverted); (ii) the test face was pitch/yaw rotated by between 0° and 75° from the encoding face (always a full-face view). Our logic was as follows: if a particular pitch/yaw-rotated face view is being processed holistically when upright, then this processing should be disrupted by inversion. Consistent with previous research, significant face inversion effects (FIEs) were found for all yaw-rotated views. However, while FIEs were found for pitch rotations up to 45°, none were observed for 75° pitch rotations (rotated either above or below the full face). We conclude that holistic processing does not occur for all views of upright faces (e.g., not for uncommon pitch rotated views), only those that can be matched to a generic global representation of a face.

The fusiform face area is engaged in holistic, not parts-based, representation of faces

Numerous studies with functional magnetic resonance imaging have shown that the fusiform face area (FFA) in the human brain plays a key role in face perception. Recent studies have found that both the featural information of faces (e.g., eyes, nose, and mouth) and the configural information of faces (i.e., spatial relation among features) are encoded in the FFA. However, little is known about whether the featural information is encoded independent of or combined with the configural information in the FFA. Here we used multi-voxel pattern analysis to examine holistic representation of faces in the FFA by correlating spatial patterns of activation with behavioral performance in discriminating face parts with face configurations either present or absent. Behaviorally, the absence of face configurations (versus presence) impaired discrimination of face parts, suggesting a holistic representation in the brain. Neurally, spatial patterns of activation in the FFA were more similar among correct than incorrect trials only when face parts were presented in a veridical face configuration. In contrast, spatial patterns of activation in the occipital face area, as well as the object-selective lateral occipital complex, were more similar among correct than incorrect trials regardless of the presence of veridical face configurations. This finding suggests that in the FFA faces are represented not on the basis of individual parts but in terms of the whole that emerges from the parts.

Nonconscious influences from emotional faces: a comparison of visual crowding, masking, and continuous flash suppression

In the study of nonconscious processing, different methods have been used in order to render stimuli invisible. While their properties are well described, the level at which they disrupt nonconscious processing remains unclear. Yet, such accurate estimation of the depth of nonconscious processes is crucial for a clear differentiation between conscious and nonconscious cognition. Here, we compared the processing of facial expressions rendered invisible through gaze-contingent crowding (GCC), masking, and continuous flash suppression (CFS), three techniques relying on different properties of the visual system. We found that both pictures and videos of happy faces suppressed from awareness by GCC were processed such as to bias subsequent preference judgments. The same stimuli manipulated with visual masking and CFS did not bias significantly preference judgments, although they were processed such as to elicit perceptual priming. A significant difference in preference bias was found between GCC and CFS, but not between GCC and masking. These results provide new insights regarding the nonconscious impact of emotional features, and highlight the need for rigorous comparisons between the different methods employed to prevent perceptual awareness.

Recognizing Degraded Faces: The Contribution of Configural and Featural Cues

Human face-recognition processes must maintain high levels of performance under different viewing conditions. An important dimension of variability is image resolution, which is affected by distance, refractive errors, and light levels. Here, we investigate how changes in resolution modulate the visual-system's ability to detect featural versus configural changes in face images. It has been suggested that at lower spatial frequencies the visual system relies predominantly on configural information, yet, to our knowledge, no experiments have systematically examined this idea. We determined subjects' relative sensitivities to configural and featural changes for systematically degraded images. We show that overall configuration and local features are processed equally well at the different resolution levels, supporting the idea of a holistic face-representation that encompasses both feature shape information and information about the distance between the features. These data have also enabled us to derive lower bounds for the resolution needed to effectively use each type of information. Our data are replicated with a completely different face stimulus set, but are not replicated when subjects were shown houses instead of faces. Overall, these results suggest that at lower spatial frequencies, facial representations embody both configural and featural attributes equally, and provide a platform for investigating the essence of holistic facial representations for low-resolution images.

From single cells to social perception

Research describing the cellular coding of faces in non-human primates often provides the underlying physiological framework for our understanding of face processing in humans. Models of face perception, explanations of perceptual after-effects from viewing particular types of faces, and interpretation of human neuroimaging data rely on monkey neurophysiological data and the assumption that neurophysiological responses of humans are comparable to those recorded in the non-human primate. Here, we review studies that describe cells that preferentially respond to faces, and assess the link between the physiological characteristics of single cells and social perception. Principally, we describe cells recorded from the non-human primate, although a limited number of cells have been recorded in humans, and are included in order to appraise the validity of non-human physiological data for our understanding of human face and social perception.

Automatic Aspects in Face Perception

We examined the perceptual dependency of local facial information on the whole facial context. In Experiment 1 participants matched a predetermined facial feature that appeared in two sequentially presented faces judging whether it is identical or not, while ignoring an irrelevant dimension in the faces. This irrelevant dimension was either (a) compatible or incompatible with the target’s response and (b) same or different in either featural characteristics or metric distance between facial features in the two faces. A compatibility effect was observed for upright but not inverted faces, regardless of the type of change that differentiated between the faces in the irrelevant dimension. Even when the target was presented upright in the inverted faces, to attenuate perceptual load, no compatibility effect was found (Experiment 2). Finally, no compatibility effects were found for either upright or inverted houses (Experiment 3). These findings suggest that holistic face perception is mandatory.

The time course of face matching for featural and relational image manipulations

It was found recently that horizontal and vertical relationships of facial features are differently vulnerable to inversion (Goffaux & Rossion, 2007). When faces are upside down manipulations of vertical relations are difficult to detect, while only moderate performance deficits are found for manipulations of horizontal relations, or when features differ. We replicate the findings of Goffaux and Rossion, and record the temporal courses of face matching performance and the effects of inversion. For vertical relations and featural changes inversion effects arise immediately, starting with the first 50ms of processing. For horizontal relations inversion effects are absent at brief timings, but arise later, after about 200ms. The final magnitudes of inversion effects are different in all three conditions, reaching 23.4% for vertical displacement, 10.7% for feature replacement, and 5.8% for horizontal displacement. Our results underline the special status of horizontal in contrast to vertical feature relations, and indicate that local and global configural information is handled in parallel by specific routines, as proposed recently (Senunkova & Barton, 2008).

The nature of upright and inverted face representations: an adaptation-transfer study of configuration

It is considered that whole-face processing of spatial structure may only be possible in upright faces, with only local feature processing in inverted faces. We asked whether this was due to impoverished representations of inverted faces. We performed two experiments. In the first, we divided faces into segments to create 'exploded' faces with disrupted second-order structures, and 'scrambled' faces with altered first-order relations; in the second we shifted features within intact facial outlines to create equivalent disruptions of spatial structure. In both we assessed the transfer of adaptation between faces with altered structure and intact faces. Scrambled adaptors did not adapt upright or inverted intact faces, indicating that a whole-face configuration is required at either orientation. Both upright and inverted faces showed a similar decline in aftereffect magnitude when adapting faces had altered second-order structure, implying that this structure is present in both upright and inverted face representations. We conclude that inverted faces are not represented simply as a collection of features, but have a whole-face configuration with second-order structure, similar to upright faces. Thus the qualitative impairments induced by inversion are not due to degraded inverted facial representations, but may reflect limitations in perceptual mechanisms.

Configural Face Processing: A Meta-Analytic Survey

In the field of face processing, the configural hypothesis is defended by many researchers. It is often claimed that this thesis is robustly supported by a large number of experiments exploring the face-inversion effect, the composite face effect, the face superiority effect, and the negative face effect. However, this claim is generally based on a rudimentary and approximate vote-counting approach. In this paper, I use meta-analyses to examine the relevant literature in more depth. The analysis supports the vote-counting argument.

Inverted faces are (eventually) processed holistically

Face inversion effects are used as evidence that faces are processed differently from objects. Nevertheless, there is debate about whether processing differences between upright and inverted faces are qualitative or quantitative. We present two experiments comparing holistic processing of upright and inverted faces within the composite task, which requires participants to match one half of a test face while ignoring irrelevant variation in the other half of the test face. Inversion reduced overall performance but led to the same qualitative pattern of results as observed for upright faces (Experiment 1). However, longer presentation times were required to observe holistic effects for inverted compared to upright faces (Experiment 2). These results suggest that both upright and inverted faces are processed holistically, but inversion reduces overall processing efficiency.

A computational shape-based model of anger and sadness justifies a configural representation of faces

Research suggests that configural cues (second-order relations) play a major role in the representation and classification of face images; making faces a "special" class of objects, since object recognition seems to use different encoding mechanisms. It is less clear, however, how this representation emerges and whether this representation is also used in the recognition of facial expressions of emotion. In this paper, we show how configural cues emerge naturally from a classical analysis of shape in the recognition of anger and sadness. In particular our results suggest that at least two of the dimensions of the computational (cognitive) space of facial expressions of emotion correspond to pure configural changes. The first of these dimensions measures the distance between the eyebrows and the mouth, while the second is concerned with the height-width ratio of the face. Under this proposed model, becoming a face "expert" would mean to move from the generic shape representation to that based on configural cues. These results suggest that the recognition of facial expressions of emotion shares this expertise property with the other processes of face processing.