Scale invariant adaptation in fusiform face-responsive regions

Prior multisensory learning can facilitate auditory-only voice-identity and speech recognition in noise

Seeing the visual articulatory movements of a speaker, while hearing their voice, helps with understanding what is said. This multisensory enhancement is particularly evident in noisy listening conditions. Multisensory enhancement also occurs even in auditory-only conditions: auditory-only speech and voice-identity recognition are superior for speakers previously learned with their face, compared to control learning; an effect termed the "face-benefit." Whether the face-benefit can assist in maintaining robust perception in increasingly noisy listening conditions, similar to concurrent multisensory input, is unknown. Here, in two behavioural experiments, we examined this hypothesis. In each experiment, participants learned a series of speakers' voices together with their dynamic face or control image. Following learning, participants listened to auditory-only sentences spoken by the same speakers and recognised the content of the sentences (speech recognition, Experiment 1) or the voice-identity of the speaker (Experiment 2) in increasing levels of auditory noise. For speech recognition, we observed that 14 of 30 participants (47%) showed a face-benefit. 19 of 25 participants (76%) showed a face-benefit for voice-identity recognition. For those participants who demonstrated a face-benefit, the face-benefit increased with auditory noise levels. Taken together, the results support an audio-visual model of auditory communication and suggest that the brain can develop a flexible system in which learned facial characteristics are used to deal with varying auditory uncertainty.

Strength of predicted information content in the brain biases decision behavior

Prediction-for-perception theories suggest that the brain predicts incoming stimuli to facilitate their categorization.1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17 However, it remains unknown what the information contents of these predictions are, which hinders mechanistic explanations. This is because typical approaches cast predictions as an underconstrained contrast between two categories18,19,20,21,22,23,24-e.g., faces versus cars, which could lead to predictions of features specific to faces or cars, or features from both categories. Here, to pinpoint the information contents of predictions and thus their mechanistic processing in the brain, we identified the features that enable two different categorical perceptions of the same stimuli. We then trained multivariate classifiers to discern, from dynamic MEG brain responses, the features tied to each perception. With an auditory cueing design, we reveal where, when, and how the brain reactivates visual category features (versus the typical category contrast) before the stimulus is shown. We demonstrate that the predictions of category features have a more direct influence (bias) on subsequent decision behavior in participants than the typical category contrast. Specifically, these predictions are more precisely localized in the brain (lateralized), are more specifically driven by the auditory cues, and their reactivation strength before a stimulus presentation exerts a greater bias on how the individual participant later categorizes this stimulus. By characterizing the specific information contents that the brain predicts and then processes, our findings provide new insights into the brain's mechanisms of prediction for perception.

The Impact of Spatial Frequency on the Perception of Crowd Emotion: An fMRI Study

Recognizing the emotions of faces in a crowd is crucial for understanding overall behavior and intention as well as for smooth and friendly social interactions. However, it is unclear whether the spatial frequency of faces affects the discrimination of crowd emotion. Although high- and low-spatial-frequency information for individual faces is processed by distinct neural channels, there is a lack of evidence on how this applies to crowd faces. Here, we used functional magnetic resonance imaging (fMRI) to investigate neural representations of crowd faces at different spatial frequencies. Thirty-three participants were asked to compare whether a test face was happy or more fearful than a crowd face that varied in high, low, and broad spatial frequencies. Our findings revealed that fearful faces with low spatial frequencies were easier to recognize in terms of accuracy (78.9%) and response time (927 ms). Brain regions, such as the fusiform gyrus, located in the ventral visual stream, were preferentially activated in high spatial frequency crowds, which, however, were the most difficult to recognize behaviorally (68.9%). Finally, the right inferior frontal gyrus was found to be better activated in the broad spatial frequency crowds. Our study suggests that people are more sensitive to fearful crowd faces with low spatial frequency and that high spatial frequency does not promote crowd face recognition.

Intracerebral electrical stimulation of the face-selective right lateral fusiform gyrus transiently impairs face identity recognition

Neuroimaging and intracranial electrophysiological studies have consistently shown the largest and most consistent face-selective neural activity in the middle portion of the human right lateral fusiform gyrus ('fusiform face area(s)', FFA). Yet, direct evidence for the critical role of this region in face identity recognition (FIR) is still lacking. Here we report the first evidence of transient behavioral impairment of FIR during focal electrical stimulation of the right FFA. Upon stimulation of an electrode contact within this region, subject CJ, who shows typical FIR ability outside of stimulation, was transiently unable to point to pictures of famous faces among strangers and to match pictures of famous or unfamiliar faces presented simultaneously for their identity. Her performance at comparable tasks with other visual materials (written names, pictures of buildings) remained unaffected by stimulation at the same location. During right FFA stimulation, CJ consistently reported that simultaneously presented faces appeared as being the same identity, with little or no distortion of the spatial face configuration. Independent electrophysiological recordings showed the largest neural face-selective and face identity activity at the critical electrode contacts. Altogether, this extensive multimodal case report supports the causal role of the right FFA in FIR.

The representation of shape and texture in category-selective regions of ventral-temporal cortex

Neuroimaging studies using univariate and multivariate approaches have shown that the fusiform face area (FFA) and parahippocampal place area (PPA) respond selectively to images of faces and places. The aim of this study was to determine the extent to which this selectivity to faces or places is based on the shape or texture properties of the images. Faces and houses were filtered to manipulate their texture properties, while preserving the shape properties (spatial envelope) of the images. In Experiment 1, multivariate pattern analysis (MVPA) showed that patterns of fMRI response to faces and houses in FFA and PPA were predicted by the shape properties, but not by the texture properties of the image. In Experiment 2, a univariate analysis (fMR‐adaptation) showed that responses in the FFA and PPA were sensitive to changes in both the shape and texture properties of the image. These findings can be explained by the spatial scale of the representation of images in the FFA and PPA. At a coarser scale (revealed by MVPA), the neural selectivity to faces and houses is sensitive to variation in the shape properties of the image. However, at a finer scale (revealed by fMR‐adaptation), the neural selectivity is sensitive to the texture properties of the image. By combining these neuroimaging paradigms, our results provide insights into the spatial scale of the neural representation of faces and places in the ventral‐temporal cortex.

Data-point-wise spatiotemporal mapping of human ventral visual areas: Use of spatial frequency/luminance-modulated chromatic faces

Visual information involving facial identity and expression is crucial for social communication. Although the influence of facial features such as spatial frequency (SF) and luminance on face processing in visual areas has been studied extensively using grayscale stimuli, the combined effects of other features in this process have not been characterized. To determine the combined effects of different SFs and color, we created chromatic stimuli with low, high or no SF components, which bring distinct SF and color information into the ventral stream simultaneously. To obtain neural activity data with high spatiotemporal resolution we recorded face-selective responses (M170) using magnetoencephalography. We used a permutation test procedure with threshold-free cluster enhancement to assess statistical significance while resolving problems related to multiple comparisons and arbitrariness found in traditional statistical methods. We found that time windows with statistically significant threshold levels were distributed differently among the stimulus conditions. Face stimuli containing any SF components evoked M170 in the fusiform gyrus (FG), whereas a significant emotional effect on M170 was only observed with the original images. Low SF faces elicited larger activation of the FG and the inferior occipital gyrus than the original images, suggesting an interaction between low and high SF information processing. Interestingly, chromatic face stimuli without SF first activated color-selective regions and then the FG, indicating that facial color was processed according to a hierarchy in the ventral stream. These findings suggest complex effects of SFs in the presence of color information, reflected in M170, and unveil the detailed spatiotemporal dynamics of face processing in the human brain.

Visual mechanisms for voice-identity recognition flexibly adjust to auditory noise level

Recognising the identity of voices is a key ingredient of communication. Visual mechanisms support this ability: recognition is better for voices previously learned with their corresponding face (compared to a control condition). This so‐called ‘face‐benefit’ is supported by the fusiform face area (FFA), a region sensitive to facial form and identity. Behavioural findings indicate that the face‐benefit increases in noisy listening conditions. The neural mechanisms for this increase are unknown. Here, using functional magnetic resonance imaging, we examined responses in face‐sensitive regions while participants recognised the identity of auditory‐only speakers (previously learned by face) in high (SNR −4 dB) and low (SNR +4 dB) levels of auditory noise. We observed a face‐benefit in both noise levels, for most participants (16 of 21). In high‐noise, the recognition of face‐learned speakers engaged the right posterior superior temporal sulcus motion‐sensitive face area (pSTS‐mFA), a region implicated in the processing of dynamic facial cues. The face‐benefit in high‐noise also correlated positively with increased functional connectivity between this region and voice‐sensitive regions in the temporal lobe in the group of 16 participants with a behavioural face‐benefit. In low‐noise, the face‐benefit was robustly associated with increased responses in the FFA and to a lesser extent the right pSTS‐mFA. The findings highlight the remarkably adaptive nature of the visual network supporting voice‐identity recognition in auditory‐only listening conditions.

Getting to Know Someone: Familiarity, Person Recognition, and Identification in the Human Brain

In our everyday life, we continuously get to know people, dominantly through their faces. Several neuroscientific experiments showed that familiarization changes the behavioral processing and underlying neural representation of faces of others. Here, we propose a model of the process of how we actually get to know someone. First, the purely visual familiarization of unfamiliar faces occurs. Second, the accumulation of associated, nonsensory information refines person representation, and finally, one reaches a stage where the effortless identification of very well-known persons occurs. We offer here an overview of neuroimaging studies, first evaluating how and in what ways the processing of unfamiliar and familiar faces differs and, second, by analyzing the fMRI adaptation and multivariate pattern analysis results we estimate where identity-specific representation is found in the brain. The available neuroimaging data suggest that different aspects of the information emerge gradually as one gets more and more familiar with a person within the same network. We propose a novel model of familiarity and identity processing, where the differential activation of long-term memory and emotion processing areas is essential for correct identification.

Predicting Performances on Processing and Memorizing East Asian Faces from Brain Activities in Face-Selective Regions: A Neurocomputational Approach

For more than two decades, a network of face-selective brain regions has been identified as the core system for face processing, including occipital face area (OFA), fusiform face area (FFA), and posterior region of superior temporal sulcus (pSTS). Moreover, recent studies have suggested that the ventral route of face processing and memory should end at the anterior temporal lobes (i.e., vATLs), which may play an important role bridging face perception and face memory. It is not entirely clear, however, the extent to which neural activities in these face-selective regions can effectively predict behavioral performance on tasks that are frequently used to investigate face processing and face memory test that requires recognition beyond variation in pose and lighting, especially when non-Caucasian East Asian faces are involved. To address these questions, we first identified during a functional scan the core face network by asking participants to perform a one-back task, while viewing either static images or dynamic videos. Dynamic localizers were effective in identifying regions of interest (ROIs) in the core face-processing system. We then correlated the brain activities of core ROIs with performances on face-processing tasks (component, configural, and composite) and face memory test (Taiwanese Face Memory Test, TFMT) and found evidence for limited predictability. We next adopted an multi-voxel pattern analysis (MVPA) approach to further explore the predictability of face-selective brain regions on TFMT performance and found evidence suggesting that a basic visual processing area such as calcarine and an area for structural face processing such as OFA may play an even greater role in memorizing faces. Implications regarding how differences in processing demands between behavioral and neuroimaging tasks and cultural specificity in face-processing and memory strategies among participants may have contributed to the findings reported here are discussed.

Emotional learning promotes perceptual predictions by remodeling stimulus representation in visual cortex

Emotions exert powerful effects on perception and memory, notably by modulating activity in sensory cortices so as to capture attention. Here, we examine whether emotional significance acquired by a visual stimulus can also change its cortical representation by linking neuronal populations coding for different memorized versions of the same stimulus, a mechanism that would facilitate recognition across different appearances. Using fMRI, we show that after pairing a given face with threat through conditioning, viewing this face activates the representation of another viewpoint of the same person, which itself was never conditioned, leading to robust repetition-priming across viewpoints in the ventral visual stream (including medial fusiform, lateral occipital, and anterior temporal cortex). We also observed a functional-anatomical segregation for coding view-invariant and view-specific identity information. These results indicate emotional signals may induce plasticity of stimulus representations in visual cortex, serving to generate new sensory predictions about different appearances of threat-associated stimuli.

Exploring the functional nature of synaesthetic colour: Dissociations from colour perception and imagery

Individuals with grapheme-colour synaesthesia experience anomalous colours when reading achromatic text. These unusual experiences have been said to resemble 'normal' colour perception or colour imagery, but studying the nature of synaesthesia remains difficult. In the present study, we report novel evidence that synaesthetic colour impacts conscious vision in a way that is different from both colour perception and imagery. Presenting 'normal' colour prior to binocular rivalry induces a location-dependent suppressive bias reflecting local habituation. By contrast, a grapheme that evokes synaesthetic colour induces a facilitatory bias reflecting priming that is not constrained to the inducing grapheme's location. This priming does not occur in non-synaesthetes and does not result from response bias. It is sensitive to diversion of visual attention away from the grapheme, but resistant to sensory perturbation, reflecting a reliance on cognitive rather than sensory mechanisms. Whereas colour imagery in non-synaesthetes causes local priming that relies on the locus of imagined colour, imagery in synaesthetes caused global priming not dependent on the locus of imagery. These data suggest a unique psychophysical profile of high-level colour processing in synaesthetes. Our novel findings and method will be critical to testing theories of synaesthesia and visual awareness.

Brain Responses to Dynamic Facial Expressions: A Normative Meta-Analysis

Identifying facial expressions is crucial for social interactions. Functional neuroimaging studies show that a set of brain areas, such as the fusiform gyrus and amygdala, become active when viewing emotional facial expressions. The majority of functional magnetic resonance imaging (fMRI) studies investigating face perception typically employ static images of faces. However, studies that use dynamic facial expressions (e.g., videos) are accumulating and suggest that a dynamic presentation may be more sensitive and ecologically valid for investigating faces. By using quantitative fMRI meta-analysis the present study examined concordance of brain regions associated with viewing dynamic facial expressions. We analyzed data from 216 participants that participated in 14 studies, which reported coordinates for 28 experiments. Our analysis revealed bilateral fusiform and middle temporal gyri, left amygdala, left declive of the cerebellum and the right inferior frontal gyrus. These regions are discussed in terms of their relation to models of face processing.

Development of face recognition: Dynamic causal modelling of MEG data

Electrophysiological studies of adults indicate that brain activity is enhanced during viewing of repeated faces, at a latency of about 250 ms after the onset of the face (M250/N250). The present study aimed to determine if this effect was also present in preschool-aged children, whose brain activity was measured in a custom-sized pediatric MEG system. The results showed that, unlike adults, face repetition did not show any significant modulation of M250 amplitude in children; however children's M250 latencies were significantly faster for repeated than non-repeated faces. Dynamic causal modelling (DCM) of the M250 in both age groups tested the effects of face repetition within the core face network including the occipital face area (OFA), the fusiform face area (FFA), and the superior temporal sulcus (STS). DCM revealed that repetition of identical faces altered both forward and backward connections in children and adults; however the modulations involved inputs to both FFA and OFA in adults but only to OFA in children. These findings suggest that the amplitude-insensitivity of the immature M250 may be due to a weaker connection between the FFA and lower visual areas.

Behavioral and Neural Adaptation in Approach Behavior

People often make approachability decisions based on perceived facial trustworthiness. However, it remains unclear how people learn trustworthiness from a population of faces and whether this learning influences their approachability decisions. Here we investigated the neural underpinning of approach behavior and tested two important hypotheses: whether the amygdala adapts to different trustworthiness ranges and whether the amygdala is modulated by task instructions and evaluative goals. We showed that participants adapted to the stimulus range of perceived trustworthiness when making approach decisions and that these decisions were further modulated by the social context. The right amygdala showed both linear response and quadratic response to trustworthiness level, as observed in prior studies. Notably, the amygdala's response to trustworthiness was not modulated by stimulus range or social context, a possible neural dynamic adaptation. Together, our data have revealed a robust behavioral adaptation to different trustworthiness ranges as well as a neural substrate underlying approach behavior based on perceived facial trustworthiness.

Temporal voice areas exist in autism spectrum disorder but are dysfunctional for voice identity recognition

The ability to recognise the identity of others is a key requirement for successful communication. Brain regions that respond selectively to voices exist in humans from early infancy on. Currently, it is unclear whether dysfunction of these voice-sensitive regions can explain voice identity recognition impairments. Here, we used two independent functional magnetic resonance imaging studies to investigate voice processing in a population that has been reported to have no voice-sensitive regions: autism spectrum disorder (ASD). Our results refute the earlier report that individuals with ASD have no responses in voice-sensitive regions: Passive listening to vocal, compared to non-vocal, sounds elicited typical responses in voice-sensitive regions in the high-functioning ASD group and controls. In contrast, the ASD group had a dysfunction in voice-sensitive regions during voice identity but not speech recognition in the right posterior superior temporal sulcus/gyrus (STS/STG)-a region implicated in processing complex spectrotemporal voice features and unfamiliar voices. The right anterior STS/STG correlated with voice identity recognition performance in controls but not in the ASD group. The findings suggest that right STS/STG dysfunction is critical for explaining voice recognition impairments in high-functioning ASD and show that ASD is not characterised by a general lack of voice-sensitive responses.

How the human brain exchanges information across sensory modalities to recognize other people

Recognizing the identity of other individuals across different sensory modalities is critical for successful social interaction. In the human brain, face- and voice-sensitive areas are separate, but structurally connected. What kind of information is exchanged between these specialized areas during cross-modal recognition of other individuals is currently unclear. For faces, specific areas are sensitive to identity and to physical properties. It is an open question whether voices activate representations of face identity or physical facial properties in these areas. To address this question, we used functional magnetic resonance imaging in humans and a voice-face priming design. In this design, familiar voices were followed by morphed faces that matched or mismatched with respect to identity or physical properties. The results showed that responses in face-sensitive regions were modulated when face identity or physical properties did not match to the preceding voice. The strength of this mismatch signal depended on the level of certainty the participant had about the voice identity. This suggests that both identity and physical property information was provided by the voice to face areas. The activity and connectivity profiles differed between face-sensitive areas: (i) the occipital face area seemed to receive information about both physical properties and identity, (ii) the fusiform face area seemed to receive identity, and (iii) the anterior temporal lobe seemed to receive predominantly identity information from the voice. We interpret these results within a prediction coding scheme in which both identity and physical property information is used across sensory modalities to recognize individuals.

Predictive codes of familiarity and context during the perceptual learning of facial identities

Face recognition is a key component of successful social behaviour. However, the computational processes that underpin perceptual learning and recognition as faces transition from unfamiliar to familiar are poorly understood. In predictive coding, learning occurs through prediction errors that update stimulus familiarity, but recognition is a function of both stimulus and contextual familiarity. Here we show that behavioural responses on a two-option face recognition task can be predicted by the level of contextual and facial familiarity in a computational model derived from predictive-coding principles. Using fMRI, we show that activity in the superior temporal sulcus varies with the contextual familiarity in the model, whereas activity in the fusiform face area covaries with the prediction error parameter that updated facial familiarity. Our results characterize the key computations underpinning the perceptual learning of faces, highlighting that the functional properties of face-processing areas conform to the principles of predictive coding.

The functional architecture of the ventral temporal cortex and its role in categorization

Visual categorization is thought to occur in the human ventral temporal cortex (VTC), but how this categorization is achieved is still largely unknown. In this Review, we consider the computations and representations that are necessary for categorization and examine how the microanatomical and macroanatomical layout of the VTC might optimize them to achieve rapid and flexible visual categorization. We propose that efficient categorization is achieved by organizing representations in a nested spatial hierarchy in the VTC. This spatial hierarchy serves as a neural infrastructure for the representational hierarchy of visual information in the VTC and thereby enables flexible access to category information at several levels of abstraction.

Beyond the FFA: The role of the ventral anterior temporal lobes in face processing

Extensive research has supported the existence of a specialized face-processing network that is distinct from the visual processing areas used for general object recognition. The majority of this work has been aimed at characterizing the response properties of the fusiform face area (FFA) and the occipital face area (OFA), which together are thought to constitute the core network of brain areas responsible for facial identification. Although accruing evidence has shown that face-selective patches in the ventral anterior temporal lobes (vATLs) are interconnected with the FFA and OFA, and that they play a role in facial identification, the relative contribution of these brain areas to the core face-processing network has remained unarticulated. Here we review recent research critically implicating the vATLs in face perception and memory. We propose that current models of face processing should be revised such that the ventral anterior temporal lobes serve a centralized role in the visual face-processing network. We speculate that a hierarchically organized system of face processing areas extends bilaterally from the inferior occipital gyri to the vATLs, with facial representations becoming increasingly complex and abstracted from low-level perceptual features as they move forward along this network. The anterior temporal face areas may serve as the apex of this hierarchy, instantiating the final stages of face recognition. We further argue that the anterior temporal face areas are ideally suited to serve as an interface between face perception and face memory, linking perceptual representations of individual identity with person-specific semantic knowledge.

Repetition suppression of face-selective evoked and induced EEG recorded from human cortex

In functional MRI studies, repetition suppression refers to the reduction of hemodynamic activation to repeated stimulus presentation. For example, the repeated presentation of a face reduces the hemodynamic response evoked by faces in the fusiform gyrus. The neural events that underlie repetition suppression are not well understood. Indeed, in contrast to the hemodynamic response, the face-specific N200 recorded from subdural electrodes on the ventral occipitotemporal cortex, primarily along the fusiform gyrus, has been reported to be insensitive to face-identity repetition. We have previously described a face-specific broadband gamma (30-100 Hz) response at ventral face-specific N200 sites that is functionally dissociable from the N200. In this study, we investigate whether gamma and other components of the electroencephalogram spectrum are affected by face-identity repetition independently of the N200. Participants viewed sequentially presented identical faces. At sites on and around the fusiform gyrus, we found that face repetition modulated alpha (8-12 Hz), low-gamma (30-60 Hz), and high-gamma (60-100 Hz) synchrony, but not the N200. These findings provide evidence of a spatially co-localized progression of face processing. Whereas the N200 reflects an initial obligatory response that is less sensitive to face-identity repetition, the subsequent spectral fluctuations reflect more elaborative face processing and are thus sensitive to face novelty. It is notable that the observed modulations were different for different frequency bands. We observed repetition suppression of broadband gamma, but repetition enhancement of alpha synchrony. This difference is discussed with regard to an existing model of repetition suppression and behavioral repetition priming.

Processing of fear and anger facial expressions: the role of spatial frequency

Spatial frequency (SF) components encode a portion of the affective value expressed in face images. The aim of this study was to estimate the relative weight of specific frequency spectrum bandwidth on the discrimination of anger and fear facial expressions. The general paradigm was a classification of the expression of faces morphed at varying proportions between anger and fear images in which SF adaptation and SF subtraction are expected to shift classification of facial emotion. A series of three experiments was conducted. In Experiment 1 subjects classified morphed face images that were unfiltered or filtered to remove either low (<8 cycles/face), middle (12-28 cycles/face), or high (>32 cycles/face) SF components. In Experiment 2 subjects were adapted to unfiltered or filtered prototypical (non-morphed) fear face images and subsequently classified morphed face images. In Experiment 3 subjects were adapted to unfiltered or filtered prototypical fear face images with the phase component randomized before classifying morphed face images. Removing mid frequency components from the target images shifted classification toward fear. The same shift was observed under adaptation condition to unfiltered and low- and middle-range filtered fear images. However, when the phase spectrum of the same adaptation stimuli was randomized, no adaptation effect was observed. These results suggest that medium SF components support the perception of fear more than anger at both low and high level of processing. They also suggest that the effect at high-level processing stage is related more to high-level featural and/or configural information than to the low-level frequency spectrum.

Neural correlates of after-effects caused by adaptation to multiple face displays

Adaptation to a given face leads to face-related, specific after-effects. Recently, this topic has attracted a lot of attention because it clearly shows that adaptation occurs even at the higher stages of visual cortical processing. However, during our every-day life, faces do not appear in isolation, rather they are usually surrounded by other stimuli. Here, we used psychophysical and fMRI adaptation methods to test whether humans adapt to the gender properties of a composite multiple face stimulus as well. As adaptors we used stimuli composed of eight different individual faces, positioned peripherally in a ring around a fixation mark. We found that the gender discrimination of a subsequent centrally presented target face is significantly biased as a result of long-term adaptation to either male or female multiple face stimuli. Similar to our previous results with single-face adaptors (Kovács et al. in Neuroimage 43(1):156-164, 2008), a concurrent functional magnetic resonance imaging adaptation experiment revealed the strongest blood oxygen level-dependent signal adaptation bilaterally in the fusiform face area. Our results suggest that humans extract the statistical features of the multiple face stimulus and this process occurs at the level of occipito-temporal face processing.

The role of the amygdala in face perception and evaluation

Faces are one of the most significant social stimuli and the processes underlying face perception are at the intersection of cognition, affect, and motivation. Vision scientists have had a tremendous success of mapping the regions for perceptual analysis of faces in posterior cortex. Based on evidence from (a) single unit recording studies in monkeys and humans; (b) human functional localizer studies; and (c) meta-analyses of neuroimaging studies, I argue that faces automatically evoke responses not only in these regions but also in the amygdala. I also argue that (a) a key property of faces represented in the amygdala is their typicality; and (b) one of the functions of the amygdala is to bias attention to atypical faces, which are associated with higher uncertainty. This framework is consistent with a number of other amygdala findings not involving faces, suggesting a general account for the role of the amygdala in perception.

Direct structural connections between voice- and face-recognition areas

Currently, there are two opposing models for how voice and face information is integrated in the human brain to recognize person identity. The conventional model assumes that voice and face information is only combined at a supramodal stage (Bruce and Young, 1986; Burton et al., 1990; Ellis et al., 1997). An alternative model posits that areas encoding voice and face information also interact directly and that this direct interaction is behaviorally relevant for optimizing person recognition (von Kriegstein et al., 2005; von Kriegstein and Giraud, 2006). To disambiguate between the two different models, we tested for evidence of direct structural connections between voice- and face-processing cortical areas by combining functional and diffusion magnetic resonance imaging. We localized, at the individual subject level, three voice-sensitive areas in anterior, middle, and posterior superior temporal sulcus (STS) and face-sensitive areas in the fusiform gyrus [fusiform face area (FFA)]. Using probabilistic tractography, we show evidence that the FFA is structurally connected with voice-sensitive areas in STS. In particular, our results suggest that the FFA is more strongly connected to middle and anterior than to posterior areas of the voice-sensitive STS. This specific structural connectivity pattern indicates that direct links between face- and voice-recognition areas could be used to optimize human person recognition.

Mirror-image sensitivity and invariance in object and scene processing pathways

Electrophysiological and behavioral studies in many species have demonstrated mirror-image confusion for objects, perhaps because many objects are vertically symmetric (e.g., a cup is the same cup when seen in left or right profile). In contrast, the navigability of a scene changes when it is mirror reversed, and behavioral studies reveal high sensitivity to this change. Thus, we predicted that representations in object-selective cortex will be unaffected by mirror reversals, whereas representations in scene-selective cortex will be sensitive to such reversals. To test this hypothesis, we ran an event-related functional magnetic resonance imaging adaptation experiment in human adults. Consistent with our prediction, we found tolerance to mirror reversals in one object-selective region, the posterior fusiform sulcus, and a strong sensitivity to these reversals in two scene-selective regions, the transverse occipital sulcus and the retrosplenial complex. However, a more posterior object-selective region, the lateral occipital sulcus, showed sensitivity to mirror reversals, suggesting that the sense information that distinguishes mirror images is represented at earlier stages in the object-processing hierarchy. Moreover, one scene-selective region (the parahippocampal place area or PPA) was tolerant to mirror reversals. This last finding challenges the hypothesis that the PPA is involved in navigation and reorientation and suggests instead that scenes, like objects, are processed by distinct pathways guiding recognition and action.

1/f(p) Characteristics of the Fourier power spectrum affects ERP correlates of face learning and recognition

We investigated the influence of Fourier power spectrum (1/f(p)) characteristics on face learning while recording ERPs that are associated with the representation of faces. Two image sets with an altered 1/f(p) characteristics were created. The first set consisted of stimuli with a STEEP SLOPE (1/f(3.5)) and therefore enhanced low spatial frequencies (LSF) and attenuated high spatial frequencies (HSF). The second set consisted of stimuli with a SHALLOW SLOPE (1/f(2)), similar to complex natural scenes and artwork, resulting in enhanced HSF and attenuated LSF. Faces with a SHALLOW SLOPE elicited larger N170 and N250 amplitudes and larger old/new effects for central positivity in comparison to unmodified faces. The opposite effect was observed for faces with a STEEP SLOPE that led to slower reaction times. This result suggests that diminishing the ratio of fine detail (HSF) to coarse structures (LSF) impairs face learning, whereas increasing it facilitates neurocognitive correlates of face learning.

Modulation of face processing by emotional expression and gaze direction during intracranial recordings in right fusiform cortex

We recorded intracranial local field potentials from structurally intact human visual cortex during several face processing tasks in a patient before brain surgery. Intracranial local field potentials were measured from subdural electrodes implanted in a right fusiform region with face-sensitive activity and a more medial location in posterior parahippocampal gyrus with house-selective activity. This electrode implantation allowed us to compare neural responses with different facial properties within two adjacent but functionally distinct cortical regions. Five experiments were conducted to determine the temporal dynamics of perceptual (Experiments 1 and 5), emotional (Experiments 2 and 3), and social (Experiment 4) effects on face-specific responses in the right fusiform. Our findings showed an early negative deflection (N200) that primarily reflected category-selective perceptual encoding of facial information, whereas higher order effects of face individuation, emotional expression, and gaze direction produced selective modulations in the same face-specific region during a later time period (from 200 to 1000 msec after onset). These results shed new lights on the time course of face recognition mechanisms in human visual cortex and reveal for the first time anatomically overlapping but temporally distinct influences of identity or emotional/social factors on face processing in right fusiform gyrus, which presumably reflect top-down feedback effects from distant brain areas.

The neural substrates and timing of top-down processes during coarse-to-fine categorization of visual scenes: a combined fMRI and ERP study

Spatial frequencies in an image influence visual analysis across a distributed, hierarchically organized brain network. Low spatial frequency (LSF) information may rapidly reach high-order areas to allow an initial coarse parsing of the visual scene, which could then be "retroinjected" through feedback into lower level visual areas to guide finer analysis on the basis of high spatial frequency (HSF). To test this "coarse-to-fine" processing scheme and to identify its neural substrates in the human brain, we presented sequences of two spatial-frequency-filtered scenes in rapid succession (LSF followed by HSF or vice versa) during fMRI and ERPs in the same participants. We show that for low-to-high sequences (but not for high-to-low sequences), LSF produces a first increase of activity in prefrontal and temporo-parietal areas, followed by enhanced responses to HSF in primary visual cortex. This pattern is consistent with retroactive influences on low-level areas that process HSF after initial activation of higher order areas by LSF.

Uncovering the visual "alphabet": advances in our understanding of object perception

The ability to rapidly and accurately recognize visual stimuli represents a significant computational challenge. Yet, despite such complexity, the primate brain manages this task effortlessly. How it does so remains largely a mystery. The study of visual perception and object recognition was once limited to investigations of brain-damaged individuals or lesion experiments in animals. However, in the last 25years, new methodologies, such as functional neuroimaging and advances in electrophysiological approaches, have provided scientists with the opportunity to examine this problem from new perspectives. This review highlights how some of these recent technological advances have contributed to the study of visual processing and where we now stand with respect to our understanding of neural mechanisms underlying object recognition.

Repetition suppression in occipitotemporal cortex despite negligible visual similarity: evidence for postperceptual processing?

The reduced neural response in certain brain regions when a task-relevant stimulus is repeated ("repetition suppression", RS) is often attributed to facilitation of the cognitive processes performed in those regions. Repetition of visual objects is associated with RS in the ventral and lateral occipital/temporal regions, and is typically attributed to facilitation of visual processes, ranging from the extraction of shape to the perceptual identification of objects. In two fMRI experiments using a semantic classification task, we found RS in a left lateral occipital/inferior temporal region to a picture of an object when the name of that object had previously been presented in a separate session. In other words, we found RS despite negligible visual similarity between the initial and repeated occurrences of an object identity. There was no evidence that this RS was driven by the learning of task-specific responses to an object identity ("S-R learning"). We consider several explanations of this occipitotemporal RS, such as phonological retrieval, semantic retrieval, and visual imagery. Although no explanation if fully satisfactory, it is proposed that such effects most plausibly relate to the extraction of task-relevant information relating to object size, either through the extraction of sensory-specific semantic information or through visual imagery processes. Our findings serve to emphasize the potential complexity of processing within traditionally visual regions, at least as measured by fMRI.

From coarse to fine? Spatial and temporal dynamics of cortical face processing

Primary vision segregates information along 2 main dimensions: orientation and spatial frequency (SF). An important question is how this primary visual information is integrated to support high-level representations. It is generally assumed that the information carried by different SF is combined following a coarse-to-fine sequence. We directly addressed this assumption by investigating how the network of face-preferring cortical regions processes distinct SF over time. Face stimuli were flashed during 75, 150, or 300 ms and masked. They were filtered to preserve low SF (LSF), middle SF (MSF), or high SF (HSF). Most face-preferring regions robustly responded to coarse LSF, face information in early stages of visual processing (i.e., until 75 ms of exposure duration). LSF processing decayed as a function of exposure duration (mostly until 150 ms). In contrast, the processing of fine HSF, face information became more robust over time in the bilateral fusiform face regions and in the right occipital face area. The present evidence suggests the coarse-to-fine strategy as a plausible modus operandi in high-level visual cortex.

fMRI–Adaptation Studies of Viewpoint Tuning in the Extrastriate and Fusiform Body Areas

People are easily able to perceive the human body across different viewpoints, but the neural mechanisms underpinning this ability are currently unclear. In three experiments, we used functional MRI (fMRI) adaptation to study the view-invariance of representations in two cortical regions that have previously been shown to be sensitive to visual depictions of the human body—the extrastriate and fusiform body areas (EBA and FBA). The BOLD response to sequentially presented pairs of bodies was treated as an index of view invariance. Specifically, we compared trials in which the bodies in each image held identical poses (seen from different views) to trials containing different poses. EBA and FBA adapted to identical views of the same pose, and both showed a progressive rebound from adaptation as a function of the angular difference between views, up to ∼30°. However, these adaptation effects were eliminated when the body stimuli were followed by a pattern mask. Delaying the mask onset increased the response (but not the adaptation effect) in EBA, leaving FBA unaffected. We interpret these masking effects as evidence that view-dependent fMRI adaptation is driven by later waves of neuronal responses in the regions of interest. Finally, in a whole brain analysis, we identified an anterior region of the left inferior temporal sulcus (l-aITS) that responded linearly to stimulus rotation, but showed no selectivity for bodies. Our results show that body-selective cortical areas exhibit a similar degree of view-invariance as other object selective areas—such as the lateral occipitotemporal area (LO) and posterior fusiform gyrus (pFs).

Evaluating functional localizers: the case of the FFA

Functional localizers are routinely used in neuroimaging studies to test hypotheses about the function of specific brain areas. The specific tasks and stimuli used to localize particular regions vary widely from study to study even when the same cortical region is targeted. Thus, it is important to ask whether task and stimulus changes lead to differences in localization or whether localization procedures are largely immune to differences in tasks and contrasting stimuli. We present two experiments and a literature review that explore whether face localizer tasks yield differential localization in the fusiform gyrus as a function of task and contrasting stimuli. We tested standard localization tasks-passive viewing, 1-back, and 2-back memory tests--and did not find differences in localization based on task. We did, however, find differences in the extent, strength and patterns/reliabilities of the activation in the fusiform gyrus based on comparison stimuli (faces vs. houses compared to faces vs. scrambled stimuli).