Cortical correlates of face and scene inversion: a comparison

Integrative processing in artificial and biological vision predicts the perceived beauty of natural images

Previous research shows that the beauty of natural images is already determined during perceptual analysis. However, it is unclear which perceptual computations give rise to the perception of beauty. Here, we tested whether perceived beauty is predicted by spatial integration across an image, a perceptual computation that reduces processing demands by aggregating image parts into more efficient representations of the whole. We quantified integrative processing in an artificial deep neural network model, where the degree of integration was determined by the amount of deviation between activations for the whole image and its constituent parts. This quantification of integration predicted beauty ratings for natural images across four studies with different stimuli and designs. In a complementary functional magnetic resonance imaging study, we show that integrative processing in human visual cortex similarly predicts perceived beauty. Together, our results establish integration as a computational principle that facilitates perceptual analysis and thereby mediates the perception of beauty.

Parafoveal vision reveals qualitative differences between fusiform face area and parahippocampal place area

The center-periphery visual field axis guides early visual system organization with enhanced resources devoted to central vision leading to reduced peripheral performance relative to that of central vision (i.e., behavioral eccentricity effect) for many visual functions. The center-periphery organization extends to high-order visual cortex where, for example, the well-studied face-sensitive fusiform face area (FFA) shows sensitivity to central vision and the place-sensitive parahippocampal place area (PPA) shows sensitivity to peripheral vision. As we have recently found that face perception is more sensitive to eccentricity than place perception, here we examined whether these behavioral findings reflect differences in FFA's and PPA's sensitivities to eccentricity. We assumed FFA would show higher sensitivity to eccentricity than PPA would, but that both regions' modulation by eccentricity would be invariant to the viewed category. We parametrically investigated (fMRI, n = 32) how FFA's and PPA's activations are modulated by eccentricity (≤8°) and category (upright/inverted faces/houses) while keeping stimulus size constant. As expected, FFA showed an overall higher sensitivity to eccentricity than PPA. However, both regions' activation modulations by eccentricity were dependent on the viewed category. In FFA, a reduction of activation with growing eccentricity ("BOLD eccentricity effect") was found (with different amplitudes) for all categories. In PPA however, qualitatively different BOLD eccentricity effect modulations were found (e.g., at 8° mild BOLD eccentricity effect for houses but a reverse BOLD eccentricity effect for faces and no modulation for inverted faces). Our results emphasize that peripheral vision investigations are critical to further our understanding of visual processing.

Cross-cultural comparison of the neural correlates of true and false memory retrieval

Prior work has shown Americans have higher levels of memory specificity than East Asians. Neuroimaging studies have not investigated mechanisms that account for cultural differences at retrieval. In this study, we use fMRI to assess whether mnemonic discrimination, distinguishing novel from previously encountered stimuli, accounts for cultural differences in memory. Fifty-five American and 55 Taiwanese young adults completed an object recognition paradigm testing discrimination of old targets, similar lures and novel foils. Mnemonic discrimination was tested by comparing discrimination of similar lures from studied targets, and results showed the relationship between activity in left fusiform gyrus and behavioural discrimination between target and lure objects differed across cultural groups. Parametric modulation analyses of activity during lure correct rejections also indicated that groups differed in left superior parietal cortex response to variations in lure similarity. Additional analyses of old vs. new activity indicated that Americans and Taiwanese differ in the neural activity supporting general object recognition in the hippocampus, left inferior frontal gyrus and middle frontal gyrus. Results are juxtaposed against comparisons of the regions activated in common across the two cultures. Overall, Americans and Taiwanese differ in the extent to which they recruit visual processing and attention modulating brain regions.

Memorability of line drawings of scenes: the role of contour properties

Why are some images more likely to be remembered than others? Previous work focused on the influence of global, low-level visual features as well as image content on memorability. To better understand the role of local, shape-based contours, we here investigate the memorability of photographs and line drawings of scenes. We find that the memorability of photographs and line drawings of the same scenes is correlated. We quantitatively measure the role of contour properties and their spatial relationships for scene memorability using a Random Forest analysis. To determine whether this relationship is merely correlational or if manipulating these contour properties causes images to be remembered better or worse, we split each line drawing into two half-images, one with high and the other with low predicted memorability according to the trained Random Forest model. In a new memorability experiment, we find that the half-images predicted to be more memorable were indeed remembered better, confirming a causal role of shape-based contour features, and, in particular, T junctions in scene memorability. We performed a categorization experiment on half-images to test for differential access to scene content. We found that half-images predicted to be more memorable were categorized more accurately. However, categorization accuracy for individual images was not correlated with their memorability. These results demonstrate that we can measure the contributions of individual contour properties to scene memorability and verify their causal involvement with targeted image manipulations, thereby bridging the gap between low-level features and scene semantics in our understanding of memorability.

Face processing and early event-related potentials: replications and novel findings

This research explores early Event-Related Potentials (ERPs) sensitivity to facial stimuli, investigating various facial features aimed to unveil underlying neural mechanisms. Two experiments, each involving 15 undergraduate students, utilized a multidimensional stimulus set incorporating race, gender, age, emotional expression, face masks, and stimulus orientation. Findings highlight significant modulations in N170 and P200 amplitudes and latencies for specific attributes, replicating prior research and revealing novel insights. Notably, age-related facial feature variations, facial inversion, and the presence of face masks significantly impact neural responses. Several speculative explanations are proposed to elucidate these results: First, the findings lend support to the idea that the increased N170 amplitude observed with facial inversion is closely tied to the activation of object-sensitive neurons. This is further bolstered by a similar amplitude increase noted when masks (effective objects) are added to faces. Second, the absence of an additional amplitude increase, when inverting face images with face masks suggests that neural populations may have reached a saturation point, limiting further enhancement. Third, the study reveals that the latency deficit in N170 induced by facial inversion is even more pronounced in the subsequent ERP component, the P200, indicating that face inversion may impact multiple stages of face processing. Lastly, the significant increase in P200 amplitude, typically associated with face typicality, for masked faces in this study aligns with previous research that demonstrated elevated P200 amplitudes for scrambled faces. This suggests that obscured faces may be processed as typical, potentially representing a default state in face processing.

Is increased activation in the fusiform face area to Greebles a result of appropriate expertise training or caused by Greebles' face likeness?

Background

In 2011, Brants et al. trained eight individuals to become Greeble experts and found neuronal inversion effects [NIEs; i.e., higher fusiform face area (FFA) activity for upright, rather than inverted Greebles]. These effects were also found for faces, both before and after training. By claiming to have replicated the seminal Greeble training study by Gauthier and colleagues in 1999, Brants et al. interpreted these results as participants viewing Greebles as faces throughout training, contrary to the original argument of subjects becoming Greeble experts only after training. However, Brants et al.'s claim presents two issues. First, their behavioral training results did not replicate those of Gauthier and Tarr conducted in 1997 and 1998, raising concerns of whether the right training regime had been adopted. Second, both a literature review and meta-analysis of NIEs in the FFA suggest its impotency as an index of the face(-like) processing.

Objectives

To empirically evaluate these issues, the present study compared two documented training paradigms Gauthier and colleagues in 1997 and 1998, and compared their impact on the brain.

Methods

Sixteen NCKU undergraduate and graduate students (nine girls) were recruited. Sixty Greeble exemplars were categorized by two genders, five families, and six individual levels. The participants were randomly divided into two groups (one for Greeble classification at all three levels and the other for gender- and individual-level training). Several fMRI tasks were administered at various time points, specifically, before training (1st), during training (2nd), and typically no <24 h after reaching expertise criterion (3rd).

Results

The ROI analysis results showed significant increases in the FFA for Greebles, and a clear neural "adaptation," both only in the Gauthier97 group and only after training, reflecting clear modulation of extensive experiences following an "appropriate" training regime. In both groups, no clear NIEs for faces nor Greebles were found, which was also in line with the review of extant studies bearing this comparison.

Conclusion

Collectively, these results invalidate the assumptions behind Brants et al.'s findings.

Changing What You Like: Modifying Contour Properties Shifts Aesthetic Valuations of Scenes

To what extent do aesthetic experiences arise from the human ability to perceive and extract meaning from visual features? Ordinary scenes, such as a beach sunset, can elicit a sense of beauty in most observers. Although it appears that aesthetic responses can be shared among humans, little is known about the cognitive mechanisms that underlie this phenomenon. We developed a contour model of aesthetics that assigns values to visual properties in scenes, allowing us to predict aesthetic responses in adults from around the world. Through a series of experiments, we manipulate contours to increase or decrease aesthetic value while preserving scene semantic identity. Contour manipulations directly shift subjective aesthetic judgments. This provides the first experimental evidence for a causal relationship between contour properties and aesthetic valuation. Our findings support the notion that visual regularities underlie the human capacity to derive pleasure from visual information.

The face inversion effect or the face upright effect?

The face inversion effect (FIE) refers to the observation that presenting stimuli upside-down impairs the processing of faces disproportionally more than other mono-oriented objects. This has been taken as evidence that processing of faces and objects differ qualitatively. However, nearly all FIE studies are based on comparing individuation of upright faces, which most people are rather good at, with individuation of objects most people are much less familiar with individuating (e.g., radios and airplanes). Consequently, the FIE may mainly reflect differences between categories in how they are processed prior to inversion, with within-category discrimination of upright faces being a much more familiar task than within-category discrimination among members belonging to other object classes. We tested this hypothesis by comparing inversion effects for faces and objects using object recognition tasks that do not require within-category discrimination (object decision and old/new recognition memory tasks). In all tasks (seven with objects and two with faces) we find credible inversion effects, but in no instance were these effects significantly larger for faces than for objects. This suggests that the FIE can be a product of familiarity with the type of identification process required in the upright conditions rather than some process that is selectively affected for faces when stimuli are inverted.

Characterising and dissecting human perception of scene complexity

Humans can effortlessly assess the complexity of the visual stimuli they encounter. However, our understanding of how we do this, and the relevant factors that result in our perception of scene complexity remain unclear; especially for the natural scenes in which we are constantly immersed. We introduce several new datasets to further understanding of human perception of scene complexity. Our first dataset (VISC-C) contains 800 scenes and 800 corresponding two-dimensional complexity annotations gathered from human observers, allowing exploration for how complexity perception varies across a scene. Our second dataset, (VISC-CI) consists of inverted scenes (reflection on the horizontal axis) with corresponding complexity maps, collected from human observers. Inverting images in this fashion is associated with destruction of semantic scene characteristics when viewed by humans, and hence allows analysis of the impact of semantics on perceptual complexity. We analysed perceptual complexity from both a single-score and a two-dimensional perspective, by evaluating a set of calculable and observable perceptual features based upon grounded psychological research (clutter, symmetry, entropy and openness). We considered these factors' relationship to complexity via hierarchical regressions analyses, tested the efficacy of various neural models against our datasets, and validated our perceptual features against a large and varied complexity dataset consisting of nearly 5000 images. Our results indicate that both global image properties and semantic features are important for complexity perception. We further verified this by combining identified perceptual features with the output of a neural network predictor capable of extracting semantics, and found that we could increase the amount of explained human variance in complexity beyond that of low-level measures alone. Finally, we dissect our best performing prediction network, determining that artificial neurons learn to extract both global image properties and semantic details from scenes for complexity prediction. Based on our experimental results, we propose the "dual information" framework of complexity perception, hypothesising that humans rely on both low-level image features and high-level semantic content to evaluate the complexity of images.

Effects of Natural Scene Inversion on Visual-evoked Brain Potentials and Pupillary Responses: A Matter of Effortful Processing of Unfamiliar Configurations

The inversion of a picture of a face hampers the accuracy and speed at which observers can perceptually process it. Event-related potentials and pupillary responses, successfully used as biomarkers of face inversion in the past, suggest that the perception of visual features, that are organized in an unfamiliar manner, recruits demanding additional processes. However, it remains unclear whether such inversion effects generalize beyond face stimuli and whether indeed more mental effort is needed to process inverted images. Here we aimed to study the effects of natural scene inversion on visual evoked potentials and pupil dilations. We simultaneously measured responses of 47 human participants to presentations of images showing upright or inverted natural scenes. For inverted scenes, we observed relatively stronger occipito-temporo-parietal N1 peak amplitudes and larger pupil dilations (on top of an initial orienting response) than for upright scenes. This study revealed neural and physiological markers of natural scene inversion that are in line with inversion effects of other stimulus types and demonstrates the robustness and generalizability of the phenomenon that unfamiliar configurations of visual content require increased processing effort.

fMRI evidence that hyper-caricatured faces activate object-selective cortex

Many brain imaging studies have looked at the cortical responses to object categories and faces. A popular way to manipulate face stimuli is by using a "face space," a high dimensional representation of individual face images, with the average face located at the origin. However, how the brain responds to faces that deviate substantially from average has not been much explored. Increasing the distance from the average (leading to increased caricaturing) could increase neural responses in face-selective regions, an idea supported by results from non-human primates. Here, we used a face space based on principal component analysis (PCA) to generate faces ranging from average to heavily caricatured. Using functional magnetic resonance imaging (fMRI), we first independently defined face-, object- and scene-selective areas with a localiser scan and then measured responses to parametrically caricatured faces. We also included conditions in which the images of faces were inverted. Interestingly in the right fusiform face area (FFA), we found that the patterns of fMRI response were more consistent as caricaturing increased. However, we found no consistent effect of either caricature level or facial inversion on the average fMRI response in the FFA or face-selective regions more broadly. In contrast, object-selective regions showed an increase in both the consistency of response pattern and the average fMRI response with increasing caricature level. This shows that caricatured faces recruit processing from regions typically defined as object-selective, possibly through enhancing low-level properties that are characteristic of objects.

Do chimpanzees see a face on Mars? A search for face pareidolia in chimpanzees

We sometimes perceive meaningful patterns or images in random arrangements of colors and shapes. This phenomenon is called pareidolia and has recently been studied intensively, especially face pareidolia. In contrast, there are few comparative-cognitive studies on face pareidolia with nonhuman primates. This study explored behavioral evidence for face pareidolia in chimpanzees using visual search and matching tasks. Faces are processed in a configural manner, and their perception and recognition are hampered by inversion and misalignment of top and bottom parts. We investigated whether the same effect occurs in a visual search for face-like objects. The results showed an effect of misalignment. On the other hand, consistent results were not obtained with the photographs of fruits. When only the top or bottom half of the face-like object was presented, chimpanzees showed better performance for the top-half condition, suggesting the importance of the eye area in face pareidolia. In the positive-control experiments, chimpanzees received the same experiment using human faces and human participants with face-like objects and fruits. As a result, chimpanzees showed an inefficient search for inverted and misaligned faces and humans for manipulated face-like objects. Finally, to examine the role of face awareness, we tested matching a human face to a face-like object in chimpanzees but obtained no substantial evidence that they saw the face-like object as a "face." Based on these results, we discussed the extents and limits of face pareidolia in chimpanzees.

The Face Inversion Effect in Deep Convolutional Neural Networks

The face inversion effect (FIE) is a behavioral marker of face-specific processing that the recognition of inverted faces is disproportionately disrupted than that of inverted non-face objects. One hypothesis is that while upright faces are represented by face-specific mechanism, inverted faces are processed as objects. However, evidence from neuroimaging studies is inconclusive, possibly because the face system, such as the fusiform face area, is interacted with the object system, and therefore the observation from the face system may indirectly reflect influences from the object system. Here we examined the FIE in an artificial face system, visual geometry group network-face (VGG-Face), a deep convolutional neural network (DCNN) specialized for identifying faces. In line with neuroimaging studies on humans, a stronger FIE was found in VGG-Face than that in DCNN pretrained for processing objects. Critically, further classification error analysis revealed that in VGG-Face, inverted faces were miscategorized as objects behaviorally, and the analysis on internal representations revealed that VGG-Face represented inverted faces in a similar fashion as objects. In short, our study supported the hypothesis that inverted faces are represented as objects in a pure face system.

Postural adjustment as a function of scene orientation

Visual orientation plays an important role in postural control, but the specific characteristics of postural response to orientation remain unknown. In this study, we investigated the relationship between postural response and the subjective visual vertical (SVV) as a function of scene orientation. We presented a virtual room including everyday objects through a head-mounted display and measured head tilt around the naso-occipital axis. The room orientation varied from 165° counterclockwise to 180° clockwise around the center of display in 15° increments. In a separate session, we also conducted a rod adjustment task to record the participant's SVV in the tilted room. We applied a weighted vector sum model to head tilt and SVV error and obtained the weight of three visual cues to orientation: frame, horizon, and polarity. We found significant contributions for all visual cues to head tilt and SVV error. For SVV error, frame cues made the largest contribution, whereas polarity contribution made the smallest. For head tilt, there was no clear difference across visual cue types, although the order of contribution was similar to the SVV. These findings suggest that multiple visual cues to orientation are involved in postural control and imply different representations of vertical orientation across postural control and perception.

The effects of stimulus inversion on the neural representations of Chinese character and face recognition

This study investigates whether stimulus inversion influences neural responses of Chinese character recognition similarly to its effect on face recognition in category-selective and object-related brain areas using functional magnetic resonance imaging. Participants performed a one-back matching task for simple (one radical) and compound (two radicals) Chinese characters and faces with upright and inverted orientations. Inverted stimuli produced slower response times with stronger activity within the fusiform gyrus (FG) than upright stimuli for faces and Chinese characters. While common inversion-related activation was identified in the left FG among stimulus types, we observed a significant inter-regional correlation between the left FG and the intraparietal sulcus for face inversion. Importantly, analyses of region-of-interest (ROI) multivariate pattern classification showed that classifiers trained on face inversion can decode the representations of character inversion in the character-selective ROI. However, this was not true for face inversion in face-selective ROIs when the classifiers were trained on characters. Similar activity patterns for character and face inversion were observed in the object-related ROIs. We also showed higher decoding accuracy for upright stimuli in the face-selective ROI than in the character-selective ROI but this was not true for inverted ones or when patterns were examined in the object-related ROIs. Together, our results support shared and distinct configural representations for character and face recognition in category-selective and object-related brain areas.

Patterns of perceptual performance in developmental prosopagnosia: An in-depth case series

Developmental prosopagnosia (DP) is a syndrome characterized by lifelong impairment in face recognition in the absence of brain damage. A key question regarding DP concerns which process(es) might be affected to selectively/disproportionally impair face recognition. We present evidence from a group of DPs, combining an overview of previous results with additional analyses important for understanding their pattern of preserved and impaired perceptual abilities. We argue that for most of these individuals, the common denominator is a deficit in (rapid) processing of global shape information. We conclude that the deficit in this group of DPs is not face-selective, but that it may appear so because faces are more visually similar-and recognized at a more fine-grained level-than objects. Indeed, when the demand on perceptual differentiation and visual similarity are held constant for faces and objects, we find no evidence for a disproportionate deficit for faces in this group of DPs.

Cortical Pathways or Mechanism in the Face Inversion Effect in Patients with First-Episode Schizophrenia

OBJECTIVE

Impaired face perception is considered as a hallmark of social disability in schizophrenia. It is widely believed that inverted faces and upright faces are processed by distinct mechanisms. Previous studies have identified that individuals with schizophrenia display poorer face processing than controls. However, the mechanisms underlying the face inversion effect (FIE) in patients with first-episode schizophrenia (FSZ) remain unclear.

METHODS

We designed an fMRI task to investigate the FIE mechanism in patients with schizophrenia. Thirty-four patients with FSZ and thirty-five healthy controls (CON) underwent task-related fMRI scanning, clinical assessment, anhedonia experience examination, and social function and cognitive function evaluation.

RESULTS

The patients with FSZ exhibited distinct functional activity regarding upright and inverted face processing within the cortical face and non-face network. These results suggest that the differences in quantitative processing might mediate the FIE in schizophrenia. Compared with controls, affected patients showed impairments in processing both upright and inverted faces; and for these patients with FSZ, upright face processing was associated with more severe and broader impairment than inverted face processing. Reduced response in the left middle occipital gyrus for upright face processing was related to poorer performance of social function outcomes evaluated using the Personal and Social Performance Scale.

CONCLUSION

Our data suggested that patients with FSZ exhibited similar performance in processing inverted faces and upright faces, but were less efficient than controls; and for these patients, inverted faces are processed less efficiently than upright faces. We also provided a clue that the mechanism under abnormal FIE might be related to an aberrant activation of non-face-selective areas instead of abnormal activation of face-specific areas in patients with schizophrenia. Finally, our study indicated that the neural pathway for upright recognition might be relevant in determining the functional outcomes of this devastating disorder.

Investigating holistic face processing within and outside of face-responsive brain regions

It has been shown that human faces are processed holistically (i.e. as indecomposable wholes, rather than by their component parts) and this holistic face processing is linked to brain activity in face-responsive brain regions. Although several brain regions outside of the face-responsive network are also sensitive to relational processing and perceptual grouping, whether these non-face-responsive regions contribute to holistic processing remains unclear. Here, we investigated holistic face processing in the composite face paradigm both within and outside of face-responsive brain regions. We recorded participants' brain activity using fMRI while they performed a composite face task. Behavioural results indicate that participants tend to judge the same top face halves as different when they are aligned with different bottom face halves but not when they are misaligned, demonstrating a composite face effect. Neuroimaging results revealed significant differences in responses to aligned and misaligned faces in the lateral occipital complex (LOC), and trends in the anterior part of the fusiform face area (FFA2) and transverse occipital sulcus (TOS), suggesting that these regions are sensitive to holistic versus part-based face processing. Furthermore, the retrosplenial cortex (RSC) and the parahippocampal place area (PPA) showed a pattern of neural activity consistent with a holistic representation of face identity, which also correlated with the strength of the behavioural composite face effect. These results suggest that neural activity in brain regions both within and outside of the face-responsive network contributes to the composite-face effect.

Exploring website gist through rapid serial visual presentation

Background

Users can make judgments about web pages in a glance. Little research has explored what semantic information can be extracted from a web page within a single fixation or what mental representations users have of web pages, but the scene perception literature provides a framework for understanding how viewers can extract and represent diverse semantic information from scenes in a glance. The purpose of this research was (1) to explore whether semantic information about a web page could be extracted within a single fixation and (2) to explore the effects of size and resolution on extracting this information. Using a rapid serial visual presentation (RSVP) paradigm, Experiment 1 explored whether certain semantic categories of websites (i.e., news, search, shopping, and social networks/blogs) could be detected within a RSVP stream of web page stimuli. Natural scenes, which have been shown to be detectable within a single fixation in the literature, served as a baseline for comparison. Experiment 2 examined the effects of stimulus size and resolution on observers’ ability to detect the presence of website categories using similar methods.

Results

Findings from this research demonstrate that users have conceptual models of websites that allow detection of web pages from a fixation’s worth of stimulus exposure, when provided additional time for processing. For website categories other than search, detection performance decreased significantly when web elements were no longer discernible due to decreases in size and/or resolution. The implications of this research are that website conceptual models rely more on page elements and less on the spatial relationship between these elements.

Conclusions

Participants can detect websites accurately when they were displayed for less than a fixation and when the participants were allowed additional processing time. Subjective comments and stimulus onset asynchrony data suggested that participants likely relied on local features for the detection of website targets for several website categories. This notion was supported when the size and/or resolution of stimuli were decreased to the extent that web elements were indistinguishable. This demonstrates that schemas or conceptualizations of websites provided information sufficient to detect websites from approximately 140 ms of stimulus exposure.

The effects of face inversion on perceiving- and sensing-based change detection

Face perception is more difficult when faces are inverted compared to when they are upright. However, it is not known whether face inversion disrupts the ability to make perceiving-based discriminations (i.e., the ability to identify a specific feature change), or sensing-based discriminations (i.e., the ability to detect there was a change without the ability to identify what changed). In the current study, we used confidence-based receiver operating characteristics (ROCs) in a change detection test to examine the effect of face inversion on perceiving and sensing. In Experiment 1, face inversion led to a reduction in the probability of perceiving but did not impact sensing-based discriminations. In Experiment 2, we replicated these results, and verified that the findings based on ROC estimates paralleled participants' phenomenological experiences of perceiving and sensing. Furthermore, the perceiving-based face inversion effect was found to reflect a reduction in the ability to accurately report specific feature changes. These findings indicate that face inversion does not reduce the ability to sense there was a change in the absence of identification, but rather it reduces the ability to consciously identify specific characteristics of faces in service of perceiving-based discriminations. In addition, they suggest that sensing responds to global differences across the visual image, rather than to changes in holistic processing of the visual input. These results further our understanding of the face inversion effect and clarify the nature of the processes underlying visual perception. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Altered upright face recognition and presence of face inversion effect in temporal lobe epilepsy: An event-related potential study

AIM

Facial recognition can be assessed by examining an event-related potential component, namely the N170. The amplitude of the N170 is larger in response to inverted faces than to upright faces. To examine facial processing in patients with temporal lobe epilepsy (TLE), we investigated the amplitude of the N170, the face inversion effect, and the association between social functioning and face-specific configuration processing.

METHODS

Sixteen patients with TLE and 17 normal controls (NC) participated in this study. Event-related potentials in response to upright or inverted neutral faces and bicycles were recorded. Social functioning was assessed by the socioeconomic status of the participants using the 5-point Hollingshead-Redlich Scale.

RESULTS

Compared with NC, patients with TLE had decreased N170 amplitudes. The inversion effect was observed for face stimuli in both groups; however, no inversion effect was observed for bicycle stimuli. Additionally, in TLE patients, but not in NC, socioeconomic status was significantly correlated with the N170 amplitudes in response to upright faces.

CONCLUSION

In a social context, upright faces are processed as a whole. This process is impaired in TLE. Conversely, inverted faces are processed analytically. This function is normal in TLE. Abnormal face-specific configuration processing may contribute to lower social functioning in TLE.

Orientation-invariance of individual differences in three face processing tasks

Numerous studies have reported impairments in perception and recognition, and, particularly, in part-integration of faces following picture-plane inversion. Whether these findings support the notion that inversion changes face processing qualitatively remains a topic of debate. To examine whether associations and dissociations of the human face processing ability depend on stimulus orientation, we measured face recognition with the Cambridge Face Memory Test (CFMT), along with experimental tests of face perception and selective attention to faces and non-face objects in a sample of 314 participants. Results showed strong inversion effects for all face-related tasks, and modest ones for non-face objects. Individual differences analysis revealed that the CFMT shared common variance with face perception and face-selective attention, however, independent of orientation. Regardless of whether predictor and criterion had same or different orientation, face recognition was best predicted by the same test battery. Principal component decomposition revealed a common factor for face recognition and face perception, a second common factor for face recognition and face-selective attention, and two unique factors. The patterns of factor loadings were nearly identical for upright and inverted presentation. These results indicate orientation-invariance of common variance in three domains of face processing. Since inversion impaired performance, but did not affect domain-related associations and dissociations, the findings suggest process-specific but orientation-general mechanisms. Specific limitations by constraints of individual differences analysis and test selection are discussed.

Decoding natural scenes based on sounds of objects within scenes using multivariate pattern analysis

Scene recognition plays an important role in spatial navigation and scene classification. It remains unknown whether the occipitotemporal cortex could represent the semantic association between the scenes and sounds of objects within the scenes. In this study, we used the functional magnetic resonance imaging (fMRI) technique and multivariate pattern analysis to assess whether diff ; ;erent scenes could be discriminated based on the patterns evoked by sounds of objects within the scenes. We found that patterns evoked by scenes could be predicted with patterns evoked by sounds of objects within the scenes in the posterior fusiform area (pF), lateral occipital area (LO) and superior temporal sulcus (STS). The further functional connectivity analysis suggested significant correlations between pF, LO and parahippocampal place area (PPA) except that between STS and other three regions under the scene and sound conditions. A distinct network in processing scenes and sounds was discovered using a seed-to-voxel analysis with STS as the seed. This study may provide a cross-modal channel of scene decoding through the sounds of objects within the scenes in the occipitotemporal cortex, which could complement the single-modal channel of scene decoding based on the global scene properties or objects within the scenes.

Lack of Privileged Access to Awareness for Rewarding Social Scenes in Autism Spectrum Disorder

Reduced social motivation is hypothesised to underlie social behavioural symptoms of Autism Spectrum Disorder (ASD). The extent to which rewarding social stimuli are granted privileged access to awareness in ASD is currently unknown. We use continuous flash suppression to investigate whether individuals with and without ASD show privileged access to awareness for social over nonsocial rewarding scenes that are closely matched for stimulus features. Strong evidence for a privileged access to awareness for rewarding social over nonsocial scenes was observed in neurotypical adults. No such privileged access was seen in ASD individuals, and moderate support for the null model was noted. These results suggest that the purported deficits in social motivation in ASD may extend to early processing mechanisms.

Stimulus Dependent Dynamic Reorganization of the Human Face Processing Network

Using the "face inversion effect", a hallmark of face perception, we examined network mechanisms supporting face representation by tracking functional magnetic resonance imaging (fMRI) stimulus-dependent dynamic functional connectivity within and between brain networks associated with the processing of upright and inverted faces. We developed a novel approach adapting the general linear model (GLM) framework classically used for univariate fMRI analysis to capture stimulus-dependent fMRI dynamic connectivity of the face network. We show that under the face inversion manipulation, the face and non-face networks have complementary roles that are evident in their stimulus-dependent dynamic connectivity patterns as assessed by network decomposition into components or communities. Moreover, we show that connectivity patterns are associated with the behavioral face inversion effect. Thus, we establish "a network-level signature" of the face inversion effect and demonstrate how a simple physical transformation of the face stimulus induces a dramatic functional reorganization across related brain networks. Finally, we suggest that the dynamic GLM network analysis approach, developed here for the face network, provides a general framework for modeling the dynamics of blocked stimulus-dependent connectivity experimental designs and hence can be applied to a host of neuroimaging studies.

Feature-coding transitions to conjunction-coding with progression through human visual cortex

Identifying an object and distinguishing it from similar items depends upon the ability to perceive its component parts as conjoined into a cohesive whole, but the brain mechanisms underlying this ability remain elusive. The ventral visual processing pathway in primates is organized hierarchically: Neuronal responses in early stages are sensitive to the manipulation of simple visual features, whereas neuronal responses in subsequent stages are tuned to increasingly complex stimulus attributes. It is widely assumed that feature-coding dominates in early visual cortex whereas later visual regions employ conjunction-coding in which object representations are different from the sum of their simple feature parts. However, no study in humans has demonstrated that putative object-level codes in higher visual cortex cannot be accounted for by feature-coding and that putative feature codes in regions prior to ventral temporal cortex are not equally well characterized as object-level codes. Thus the existence of a transition from feature- to conjunction-coding in human visual cortex remains unconfirmed, and if a transition does occur its location remains unknown. By employing multivariate analysis of functional imaging data, we measure both feature-coding and conjunction-coding directly, using the same set of visual stimuli, and pit them against each other to reveal the relative dominance of one vs. the other throughout cortex. Our results reveal a transition from feature-coding in early visual cortex to conjunction-coding in both inferior temporal and posterior parietal cortices. This novel method enables the use of experimentally controlled stimulus features to investigate population-level feature and conjunction codes throughout human cortex.NEW & NOTEWORTHY We use a novel analysis of neuroimaging data to assess representations throughout visual cortex, revealing a transition from feature-coding to conjunction-coding along both ventral and dorsal pathways. Occipital cortex contains more information about spatial frequency and contour than about conjunctions of those features, whereas inferotemporal and parietal cortices contain conjunction coding sites in which there is more information about the whole stimulus than its component parts.

Change detection of meaningful objects in real-world scenes in adolescents with and without autism spectrum disorder

Previous research suggested that adolescents with autism spectrum disorder are better than typically developing children in detecting local, non-social details within complex visual scenes. To better understand these differences, we used the image database by Sareen et al., containing the size and on-screen location information of all changes in the images, in a change blindness paradigm. In this task, an original and a modified real-world scene, separated by a gray blank, alternate repeatedly until observers detect the change. Our results indicated that participants with and without autism spectrum disorder performed similarly when scenes were presented upright, but that only the performance of the typically developing adolescents became worse in the inverted scene condition. In this condition, the correlation between performance and both image difficulty and change predictability was significantly weaker in autism spectrum disorder than in typically developing participants. We suggest that these findings result from a more locally biased search strategy in people with autism spectrum disorder, compared to typically developing participants, in tasks in which the rapid processing of global information does not help to improve change detection performance. Finally, although we found change location, change size, and age to influence participant performance, none of these was directly linked to the observed group-level differences.

Patterns of response to scrambled scenes reveal the importance of visual properties in the organization of scene-selective cortex

Neuroimaging studies have found distinct patterns of neural response to different categories of scene in scene-selective regions of the human brain. However, it is not clear how information about scene category is represented in these regions. Images from different categories vary systematically in their visual properties as well as their semantic category. So, it is possible that patterns of neural response could reflect variation in visual properties. To address this question, we used fMRI to measure patterns of neural response to intact and scrambled scene categories. Although scrambling preserved many of their visual characteristics, perception of scene categories was severely impaired. Nevertheless, we found distinct patterns of response to different scene categories in the parahippocampal place area (PPA) and the occipital place area (OPA) for both intact and scrambled scenes. Moreover, intact and scrambled scenes produced highly similar patterns of response. Our finding that reliable and distinct patterns of response in scene-selective regions are still evident when categorical perception is impaired suggests that visual properties play an important role in the topographic organization of these regions.

The Effects of Face Inversion and Face Race on the P100 ERP

Research about the neural basis of face recognition has investigated the timing and anatomical substrates of different stages of face processing. Scalp-recorded ERP studies of face processing have focused on the N170, an ERP with a peak latency of ∼170 msec that has long been associated with the initial structural encoding of faces. However, several studies have reported earlier ERP differences related to faces, suggesting that face-specific processes might occur before N170. Here, we examined the influence of face inversion and face race on the timing of face-sensitive scalp-recorded ERPs by examining neural responses to upright and inverted line-drawn and luminance-matched white and black faces in a sample of white participants. We found that the P100 ERP evoked by inverted faces was significantly larger than that evoked by upright faces. Although this inversion effect was statistically significant at 100 msec, the inverted-upright ERP difference peaked at 138 msec, suggesting that it might represent an activity in neural sources that overlap with P100. Inverse modeling of the inversion effect difference waveform suggested possible neural sources in pericalcarine extrastriate visual cortex and lateral occipito-temporal cortex. We also found that the inversion effect difference wave was larger for white faces. These results are consistent with behavioral evidence that individuals process the faces of their own races more configurally than faces of other races. Taken together, the inversion and race effects observed in the current study suggest that configuration influences face processing by at least 100 msec.

Abnormalities in the Visual Processing of Viewing Complex Visual Stimuli Amongst Individuals With Body Image Concern

Individuals with body dysmorphic disorder (BDD) and clinically concerning body-image concern (BIC) appear to possess abnormalities in the way they perceive visual information in the form of a bias towards local visual processing. As inversion interrupts normal global processing, forcing individuals to process locally, an upright-inverted stimulus discrimination task was used to investigate this phenomenon. We examined whether individuals with nonclinical, yet high levels of BIC would show signs of this bias, in the form of reduced inversion effects (i.e., increased local processing). Furthermore, we assessed whether this bias appeared for general visual stimuli or specifically for appearance-related stimuli, such as faces and bodies. Participants with high-BIC (n = 25) and low-BIC (n = 30) performed a stimulus discrimination task with upright and inverted faces, scenes, objects, and bodies. Unexpectedly, the high-BIC group showed an increased inversion effect compared to the low-BIC group, indicating perceptual abnormalities may not be present as local processing biases, as originally thought. There was no significant difference in performance across stimulus types, signifying that any visual processing abnormalities may be general rather than appearance-based. This has important implications for whether visual processing abnormalities are predisposing factors for BDD or develop throughout the disorder.

The Neural Basis of Individual Face and Object Perception

We routinely need to process the identity of many faces around us, and how the brain achieves this is still the subject of much research in cognitive neuroscience. To date, insights on face identity processing have come from both healthy and clinical populations. However, in order to directly compare results across and within participant groups, and across different studies, it is crucial that a standard task is utilized which includes different exemplars (for example, non-face stimuli along with faces), is memory-neutral, and taps into identity matching across orientation and across viewpoint change. The goal of this study was to test a previously behaviourally tested face and object identity matching design in a healthy control sample whilst being scanned using fMRI. Specifically, we investigated categorical, orientation, and category-specific orientation effects while participants were focused on identity matching of simultaneously presented exemplar stimuli. Alongside observing category and orientation specific effects in a distributed set of brain regions, we also saw an interaction between stimulus category and orientation in the bilateral fusiform gyrus and bilateral middle occipital gyrus. Generally these clusters showed the pattern of a heightened response to inverted versus upright faces, and to upright, as compared to inverted shoes. These results are discussed in relation to previous studies and to potential future research within prosopagnosic individuals.

Emotional facial expressions evoke faster orienting responses, but weaker emotional responses at neural and behavioural levels compared to scenes: A simultaneous EEG and facial EMG study

In the current study, electroencephalography (EEG) was recorded simultaneously with facial electromyography (fEMG) to determine whether emotional faces and emotional scenes are processed differently at the neural level. In addition, it was investigated whether these differences can be observed at the behavioural level via spontaneous facial muscle activity. Emotional content of the stimuli did not affect early P1 activity. Emotional faces elicited enhanced amplitudes of the face-sensitive N170 component, while its counterpart, the scene-related N100, was not sensitive to emotional content of scenes. At 220-280ms, the early posterior negativity (EPN) was enhanced only slightly for fearful as compared to neutral or happy faces. However, its amplitudes were significantly enhanced during processing of scenes with positive content, particularly over the right hemisphere. Scenes of positive content also elicited enhanced spontaneous zygomatic activity from 500-750ms onwards, while happy faces elicited no such changes. Contrastingly, both fearful faces and negative scenes elicited enhanced spontaneous corrugator activity at 500-750ms after stimulus onset. However, relative to baseline EMG changes occurred earlier for faces (250ms) than for scenes (500ms) whereas for scenes activity changes were more pronounced over the whole viewing period. Taking into account all effects, the data suggests that emotional facial expressions evoke faster attentional orienting, but weaker affective neural activity and emotional behavioural responses compared to emotional scenes.

Neural basis of identity information extraction from noisy face images

Previous research has made significant progress in identifying the neural basis of the remarkably efficient and seemingly effortless face perception in humans. However, the neural processes that enable the extraction of facial information under challenging conditions when face images are noisy and deteriorated remains poorly understood. Here we investigated the neural processes underlying the extraction of identity information from noisy face images using fMRI. For each participant, we measured (1) face-identity discrimination performance outside the scanner, (2) visual cortical fMRI responses for intact and phase-randomized face stimuli, and (3) intrinsic functional connectivity using resting-state fMRI. Our whole-brain analysis showed that the presence of noise led to reduced and increased fMRI responses in the mid-fusiform gyrus and the lateral occipital cortex, respectively. Furthermore, the noise-induced modulation of the fMRI responses in the right face-selective fusiform face area (FFA) was closely associated with individual differences in the identity discrimination performance of noisy faces: smaller decrease of the fMRI responses was accompanied by better identity discrimination. The results also revealed that the strength of the intrinsic functional connectivity within the visual cortical network composed of bilateral FFA and bilateral object-selective lateral occipital cortex (LOC) predicted the participants' ability to discriminate the identity of noisy face images. These results imply that perception of facial identity in the case of noisy face images is subserved by neural computations within the right FFA as well as a re-entrant processing loop involving bilateral FFA and LOC.

Individual topographic variability is inherent to cortical physiology but task-related differences may be noise

The observation of highly variable sets of association neocortical areas across individuals, containing the estimated generators of Slow Potentials (SPs) and beta oscillations, lead to the persistence in individual analyses. This brought to notice an unexpected within individual topographic similarity between task conditions, despite our original interest in task-related differences. A recent related work explored the quantification of the similarity in beta topography between largely differing tasks. In this article, we used Independent Component Analysis (ICA) for the decomposition of beta activity from a visual attention task, and compared it with quiet resting, recorded by 128-channel EEG in 62 subjects. We statistically tested whether each ICA component obtained in one condition could be explained by a linear regression model based on the topographic patterns from the other condition, in each individual. Results were coherent with the previous report, showing a high topographic similarity between conditions. From an average of 12 beta component maps obtained for each task, over 80% were satisfactorily explained by the complementary task. Once more, the component maps including those considered unexplained, putatively “task-specific”, had their scalp distribution and estimated cortical sources highly variable across subjects. These findings are discussed along with other studies based on individual data and the present fMRI results, reinforcing the increasingly accepted view that individual variability in sets of active neocortical association areas is not noise, but intrinsic to cortical physiology. Actual ‘noise’, mainly stemming from group “brain averaging” and the emphasis on statistical differences as opposed to similarities, may explain the overall hardship in replication of the vast literature on supposed task-specific forms of activity, and the ever inconclusive status of a universal functional mapping of cortical association areas. A new hypothesis, that individuals may use the same idiosyncratic sets of areas, at least by their fraction of activity in the sub-delta and beta range, in various non-sensory-motor forms of conscious activities, is a corollary of the discussed variability.

Dissociable cortical pathways for qualitative and quantitative mechanisms in the face inversion effect

Humans' ability to recognize objects is remarkably robust across a variety of views unless faces are presented upside-down. Whether this face inversion effect (FIE) results from qualitative (distinct mechanisms) or quantitative processing differences (a matter of degree within common mechanisms) between upright and inverted faces has been intensely debated. Studies have focused on preferential responses to faces in face-specific brain areas, although face recognition also involves nonpreferential responses in non-face-specific brain areas. By using dynamic causal modeling with Bayesian model selection, here we show that dissociable cortical pathways are responsible for qualitative and quantitative mechanisms in the FIE in the distributed network for face recognition. When faces were upright, the early visual cortex (VC) and occipital and fusiform face areas (OFA, FFA) suppressed couplings to the lateral occipital cortex (LO), a primary locus of object processing. In contrast, they did not inhibit the LO when faces were inverted but increased couplings to the intraparietal sulcus, which has been associated with visual working memory. Furthermore, we found that upright and inverted face processing together involved the face network consisting of the VC, OFA, FFA, and inferior frontal gyrus. Specifically, modulatory connectivity within the common pathways (VC-OFA), implicated in the parts-based processing of faces, strongly correlated with behavioral FIE performance. The orientation-dependent dynamic reorganization of effective connectivity indicates that the FIE is mediated by both qualitative and quantitative differences in upright and inverted face processing, helping to resolve a central debate over the mechanisms of the FIE.

Spatio-temporal dynamics and laterality effects of face inversion, feature presence and configuration, and face outline

Although a crucial role of the fusiform gyrus (FG) in face processing has been demonstrated with a variety of methods, converging evidence suggests that face processing involves an interactive and overlapping processing cascade in distributed brain areas. Here we examine the spatio-temporal stages and their functional tuning to face inversion, presence and configuration of inner features, and face contour in healthy subjects during passive viewing. Anatomically-constrained magnetoencephalography (aMEG) combines high-density whole-head MEG recordings and distributed source modeling with high-resolution structural MRI. Each person's reconstructed cortical surface served to constrain noise-normalized minimum norm inverse source estimates. The earliest activity was estimated to the occipital cortex at ~100 ms after stimulus onset and was sensitive to an initial coarse level visual analysis. Activity in the right-lateralized ventral temporal area (inclusive of the FG) peaked at ~160 ms and was largest to inverted faces. Images containing facial features in the veridical and rearranged configuration irrespective of the facial outline elicited intermediate level activity. The M160 stage may provide structural representations necessary for downstream distributed areas to process identity and emotional expression. However, inverted faces additionally engaged the left ventral temporal area at ~180 ms and were uniquely subserved by bilateral processing. This observation is consistent with the dual route model and spared processing of inverted faces in prosopagnosia. The subsequent deflection, peaking at ~240 ms in the anterior temporal areas bilaterally, was largest to normal, upright faces. It may reflect initial engagement of the distributed network subserving individuation and familiarity. These results support dynamic models suggesting that processing of unfamiliar faces in the absence of a cognitive task is subserved by a distributed and interactive neural circuit.

Abnormalities in visual processing amongst students with body image concerns

Individuals with body dysmorphic disorder (BDD) appear to possess abnormalities in the way they observe and discriminate visual information. A pre-occupation with perceived defects in appearance has been attributed to a local visual processing bias. We studied the nature of visual bias in individuals who may be at risk of developing BDD – those with high body image concerns (BICs) – by using inverted stimulus discrimination. Inversion disrupts global, configural information in favor of local, feature-based processing. 40 individuals with high BIC and 40 low BIC controls performed a discrimination task with upright and inverted faces, bodies, and scenes. Individuals with high BIC discriminated inverted faces and bodies faster than controls, and were also more accurate when discriminating inverted bodies and scenes. This reduction in inversion effect for high BIC individuals may be due to a stimulus-general local, detail-focused processing bias, which may be associated with maladaptive fixation on small features in their appearance.

Resting-state fMRI reveals functional connectivity between face-selective perirhinal cortex and the fusiform face area related to face inversion

Studies examining the neural correlates of face perception and recognition in humans have revealed multiple brain regions that appear to play a specialized role in face processing. These include an anterior portion of perirhinal cortex (PrC) that appears to be homologous to the face-selective 'anterior face patch' recently reported in non-human primates. Electrical stimulation studies in the macaque indicate that the anterior face patch is strongly connected with other face-selective patches of cortex, even in the absence of face stimuli. The intrinsic functional connectivity of face-selective PrC and other regions of the face-processing network in humans are currently less well understood. Here, we examined resting-state fMRI connectivity across five face-selective regions in the right hemisphere that were identified with separate functional localizer scans: the PrC, amygdala (Amg), superior temporal sulcus, fusiform face area (FFA), and occipital face area. A partial correlation technique, controlling for fluctuations in occipitotemporal cortex that were not face specific, revealed connectivity between the PrC and the FFA, as well as the Amg. When examining the 'unique' connectivity of PrC within this face processing network, we found that the connectivity between the PrC and the FFA as well as that between the PrC and the Amg persisted even after controlling for potential mediating effects of other face-selective regions. Lastly, we examined the behavioral relevance of PrC connectivity by examining inter-individual differences in resting-state fluctuations in relation to differences in behavioral performance for a forced-choice recognition memory task that involved judgments on upright and inverted faces. This analysis revealed a significant correlation between the increased accuracy for upright faces (i.e., the face inversion effect) and the strength of connectivity between the PrC and the FFA. Together, these findings point to a high degree of functional integration of face-selective aspects of PrC in the face processing network with notable behavioral relevance.

The functional architecture for face-processing expertise: FMRI evidence of the developmental trajectory of the core and the extended face systems

Expertise in processing faces is a cornerstone of human social interaction. However, the developmental course of many key brain regions supporting face preferential processing in the human brain remains undefined. Here, we present findings from an FMRI study using a simple viewing paradigm of faces and objects in a continuous age sample covering the age range from 6 years through adulthood. These findings are the first to use such a sample paired with whole-brain FMRI analyses to investigate development within the core and extended face networks across the developmental spectrum from middle childhood to adulthood. We found evidence, albeit modest, for a developmental trend in the volume of the right fusiform face area (rFFA) but no developmental change in the intensity of activation. From a spatial perspective, the middle portion of the right fusiform gyrus most commonly found in adult studies of face processing was increasingly likely to be included in the FFA as age increased to adulthood. Outside of the FFA, the most striking finding was that children hyperactivated nearly every aspect of the extended face system relative to adults, including the amygdala, anterior temporal pole, insula, inferior frontal gyrus, anterior cingulate gyrus, and parietal cortex. Overall, the findings suggest that development is best characterized by increasing modulation of face-sensitive regions throughout the brain to engage only those systems necessary for task requirements.

The temporo-parietal junction contributes to global gestalt perception-evidence from studies in chess experts

In a recent neuroimaging study the comparison of intact vs. disturbed perception of global gestalt indicated a significant role of the temporo-parietal junction (TPJ) in the intact perception of global gestalt (Huberle and Karnath, 2012). This location corresponded well with the areas known to be damaged or impaired in patients with simultanagnosia after stroke or due to neurodegenerative diseases. It was concluded that the TPJ plays an important role in the integration of individual items to a holistic percept. Thus, increased BOLD signals should be found in this region whenever a task calls for the integration of multiple visual items. Behavioral experiments in chess experts suggested that their superior skills in comparison to chess novices are partly based on fast holistic processing of chess positions with multiple pieces. We thus analyzed BOLD data from four fMRI studies that compared chess experts with chess novices during the presentation of complex chess-related visual stimuli (Bilalić et al., 2010, 2011a,b, 2012). Three regions of interests were defined by significant TPJ clusters in the abovementioned study of global gestalt perception (Huberle and Karnath, 2012) and BOLD signal amplitudes in these regions were compared between chess experts and novices. These cross-paradigm ROI analyses revealed higher signals at the TPJ in chess experts in comparison to novices during presentations of complex chess positions. This difference was consistent across the different tasks in five independent experiments. Our results confirm the assumption that the TPJ region identified in previous work on global gestalt perception plays an important role in the processing of complex visual stimulus configurations.

Repetition probability does not affect fMRI repetition suppression for objects

Previously several functional magnetic resonance imaging (fMRI) studies point toward the role of perceptual expectations in determining adaptation or repetition suppression (RS) in humans. These studies showed that the probability of repetitions of faces within a block influences the magnitude of adaptation in face-related areas of the human brain (Summerfield et al., 2008). However, a current macaque single-cell/local field potential (LFP) recording study using objects as stimuli found no evidence for the modulation of the neural response by the repetition probability in the inferior temporal cortex (Kaliukhovich and Vogels, 2010). Here we examined whether stimulus repetition probability affects fMRI repetition suppression for nonface object stimuli in the human brain. Subjects were exposed to either two identical [repetition trials (RTs)] or two different [alternation trials (ATs)] object stimuli. Both types of trials were presented blocks consisting of either 75% [repetition blocks (RBs)] or 25% [alternation blocks (ABs)] of RTs. We found strong RS, i.e., a lower signal for RTs compared to ATs, in the object sensitive lateral occipital cortex as well as in the face-sensitive occipital and fusiform face areas. More importantly, however, there was no significant difference in the magnitude of RS between RBs and ABs in each of the areas. This is in agreement with the previous monkey single-unit/LFP findings and suggests that RS in the case of nonface visual objects is not modulated by the repetition probability in humans. Our results imply that perceptual expectation effects vary for different visual stimulus categories.

Fusing concurrent EEG-fMRI with dynamic causal modeling: application to effective connectivity during face perception

Despite the wealth of research on face perception, the interactions between core regions in the face-sensitive network of the visual cortex are not well understood. In particular, the link between neural activity in face-sensitive brain regions measured by fMRI and EEG markers of face-selective processing in the N170 component is not well established. In this study, we used dynamic causal modeling (DCM) as a data fusion approach to integrate concurrently acquired EEG and fMRI data during the perception of upright compared with inverted faces. Data features derived from single-trial EEG variability were used as contextual modulators on fMRI-derived estimates of effective connectivity between key regions of the face perception network. The overall construction of our model space was highly constrained by the effects of task and ERP parameters on our fMRI data. Bayesian model selection suggested that the occipital face area (OFA) acted as a central gatekeeper directing visual information to the superior temporal sulcus (STS), the fusiform face area (FFA), and to a medial region of the fusiform gyrus (mFG). The connection from the OFA to the STS was strengthened on trials in which N170 amplitudes to upright faces were large. In contrast, the connection from the OFA to the mFG, an area known to be involved in object processing, was enhanced for inverted faces particularly on trials in which N170 amplitudes were small. Our results suggest that trial-by-trial variation in neural activity at around 170 ms, reflected in the N170 component, reflects the relative engagement of the OFA to STS/FFA network over the OFA to mFG object processing network for face perception. Importantly, the DCMs predicted the observed data significantly better by including the modulators derived from the N170, highlighting the value of incorporating EEG-derived information to explain interactions between regions as a multi-modal data fusion method for combined EEG-fMRI.

Representational demands modulate involvement of perirhinal cortex in face processing

The classic view holds that the medial temporal lobes (MTL) are dedicated to declarative memory functioning. Recent evidence, however, suggests that perirhinal cortex (PrC), a structure within the anterior MTL, may also play a role in perceptual discriminations when representations of complex conjunctions of features, or of gestalt-characteristics of objects must be generated. Interestingly, neuroimaging and electrophysiological recordings in nonhuman primates have also revealed a face patch in the anterior collateral sulcus with preferential responses to face stimuli in various task contexts. In the present fMRI study, we investigated the representational demands that influence PrC involvement in different types of judgments on human faces. Holding stimulus complexity constant, we independently manipulated the nature of the task and the orientation of the stimuli presented (through face inversion). Aspects of right PrC showed increased responses in a forced-choice recognition-memory and a perceptual-oddity task, as compared to a feature-search task that was included to probe visual detection of an isolated face feature. Effects of stimulus orientation in right PrC were observed when the recognition-memory condition for upright faces was compared with all other experimental conditions, including recognition-memory for inverted faces-a result that can be related to past work on the role of PrC in object unitization. Notably, both effects in right PrC paralleled activity patterns in broader networks of regions that also included the right fusiform gyrus and the amygdala, regions frequently implicated in face processing in prior research. As such, the current findings do not support the view that reference to a prior study episode clearly distinguishes the role of PrC from that of more posterior ventral visual pathway regions. They add to a growing body of evidence suggesting that the functional role of specific MTL structures may be best understood in terms of the representations that are required by the task and the stimuli at hand.

fMRI analysis of contrast polarity in face-selective cortex in humans and monkeys

Recognition is strongly impaired when the normal contrast polarity of faces is reversed. For instance, otherwise-familiar faces become very difficult to recognize when viewed as photographic negatives. Here, we used fMRI to demonstrate related properties in visual cortex: 1) fMRI responses in the human Fusiform Face Area (FFA) decreased strongly (26%) to contrast-reversed faces across a wide range of contrast levels (5.3-100% RMS contrast), in all subjects tested. In a whole brain analysis, this contrast polarity bias was largely confined to the Fusiform Face Area (FFA; p<0.0001), with possible involvement of a left occipital face-selective region. 2) It is known that reversing facial contrast affects three image properties in parallel (absorbance, shading, and specular reflection). Here, comparison of FFA responses to those in V1 suggests that the contrast polarity bias is produced in FFA only when all three component properties were reversed simultaneously, which suggests a prominent non-linearity in FFA processing. 3) Across a wide range (180°) of illumination source angles, 3D face shapes without texture produced response constancy in FFA, without a contrast polarity bias. 4) Consistent with psychophysics, analogous fMRI biases for normal contrast polarity were not produced by non-face objects, with image statistics similar to the face stimuli. 5) Using fMRI, we also demonstrated a contrast polarity bias in awake behaving macaque monkeys, in the cortical region considered homologous to human FFA. Thus common cortical mechanisms may underlie facial contrast processing across ~25 million years of primate evolution.