Visual representation of eye gaze is coded by a nonopponent multichannel system

Similar neural adaptation mechanisms underlying face gender and tilt aftereffects

Visual aftereffects have been found for a wide variety of stimuli, ranging from oriented lines to human faces, but previous results suggested that face aftereffects were qualitatively different from orientation (tilt) aftereffects. Using computational models, we predicted that these differences were due to the limited range of faces used in previous studies. Here we report psychophysical results verifying this prediction. We used the same paradigm to test tilt aftereffects (TAE) and face gender aftereffects (FAE) and found that they exhibited qualitatively similar aftereffect curves, when a sufficiently large range of test faces was used. Overall, the results suggest that similar adaptation mechanisms may underlie both high-level and low-level visual processing.

Visual adaptation and face perception

The appearance of faces can be strongly affected by the characteristics of faces viewed previously. These perceptual after-effects reflect processes of sensory adaptation that are found throughout the visual system, but which have been considered only relatively recently in the context of higher level perceptual judgements. In this review, we explore the consequences of adaptation for human face perception, and the implications of adaptation for understanding the neural-coding schemes underlying the visual representation of faces. The properties of face after-effects suggest that they, in part, reflect response changes at high and possibly face-specific levels of visual processing. Yet, the form of the after-effects and the norm-based codes that they point to show many parallels with the adaptations and functional organization that are thought to underlie the encoding of perceptual attributes like colour. The nature and basis for human colour vision have been studied extensively, and we draw on ideas and principles that have been developed to account for norms and normalization in colour vision to consider potential similarities and differences in the representation and adaptation of faces.

From single cells to social perception

Research describing the cellular coding of faces in non-human primates often provides the underlying physiological framework for our understanding of face processing in humans. Models of face perception, explanations of perceptual after-effects from viewing particular types of faces, and interpretation of human neuroimaging data rely on monkey neurophysiological data and the assumption that neurophysiological responses of humans are comparable to those recorded in the non-human primate. Here, we review studies that describe cells that preferentially respond to faces, and assess the link between the physiological characteristics of single cells and social perception. Principally, we describe cells recorded from the non-human primate, although a limited number of cells have been recorded in humans, and are included in order to appraise the validity of non-human physiological data for our understanding of human face and social perception.

Adaptation and visual coding

Visual coding is a highly dynamic process and continuously adapting to the current viewing context. The perceptual changes that result from adaptation to recently viewed stimuli remain a powerful and popular tool for analyzing sensory mechanisms and plasticity. Over the last decade, the footprints of this adaptation have been tracked to both higher and lower levels of the visual pathway and over a wider range of timescales, revealing that visual processing is much more adaptable than previously thought. This work has also revealed that the pattern of aftereffects is similar across many stimulus dimensions, pointing to common coding principles in which adaptation plays a central role. However, why visual coding adapts has yet to be fully answered.

Adaptation and the perception of facial age

We examined how the perceived age of adult faces is affected by adaptation to younger or older adult faces. Observers viewed images of a synthetic male face simulating ageing over a modelled range from 15 to 65 years. Age was varied by changing shape cues or textural cues. Age level was varied in a staircase to find the observer's subjective category boundary between "old" and "young". These boundaries were strongly biased by adaptation to the young or old face, with significant aftereffects induced by either shape or textural cues. A further experiment demonstrated comparable aftereffects for photorealistic images of average older or younger adult faces, and found that aftereffects showed some selectivity for a change in gender but also strongly transferred across gender. This transfer shows that adaptation can adjust to the attribute of age somewhat independently of other facial attributes. These findings suggest that perceived age, like many other natural facial dimensions, is highly susceptible to adaptation, and that this adaptation can be carried by both the structural and textural changes that normally accompany facial ageing.

Adaptation to different mouth shapes influences visual perception of ambiguous lip speech

We investigated the effects of adaptation to mouth shapes associated with different spoken sounds (sustained /m/ or /u/) on visual perception of lip speech. Participants were significantly more likely to label ambiguous faces on an /m/-to-/u/ continuum as saying /u/ following adaptation to /m/ mouth shapes than they were in a preadaptation test. By contrast, participants were significantly less likely to label the ambiguous faces as saying /u/ following adaptation to /u/ mouth shapes than they were in a preadaptation test. The magnitude of these aftereffects was equivalent when the same individual was shown in the adaptation and test phases of the experiment and when different individuals were presented in the adaptation and test phases. These findings present novel evidence that adaptation to natural variations in facial appearance influences face perception, and they extend previous research on face aftereffects to visual perception of lip speech.

Neural abnormalities in early-onset and adolescence-onset conduct disorder

CONTEXT

Conduct disorder (CD) is characterized by severe antisocial behavior that emerges in childhood (early-onset CD [EO-CD]) or adolescence (adolescence-onset CD [AO-CD]). Early-onset CD is proposed to have a neurodevelopmental basis, whereas AO-CD is thought to emerge owing to social mimicry of deviant peers. However, this developmental taxonomic theory is debated after reports of neuropsychological impairments in both CD subtypes. A critical, although unaddressed, issue is whether these subtypes present similar or distinct neurophysiological profiles. Hence, we investigated neurophysiological responses to emotional and neutral faces in regions associated with antisocial behavior (ie, the amygdala, ventromedial prefrontal cortex, insula, and orbitofrontal cortex) in individuals with EO-CD and AO-CD and in healthy control subjects.

OBJECTIVE

To investigate whether EO-CD and AO-CD subjects show neurophysiological abnormalities.

DESIGN

Case-control study.

SETTING

Government research institute, university department.

PARTICIPANTS

Seventy-five male adolescents and young adults aged 16 to 21 years, including 27 with EO-CD, 25 with AO-CD, and 23 healthy controls. Main Outcome Measure Neural activations measured by functional magnetic resonance imaging while participants viewed angry, sad, and neutral faces.

RESULTS

Comparing angry vs neutral faces, participants with both CD subtypes displayed reduced responses in regions associated with antisocial behavior compared with controls; differences between the CD subtypes were not significant. Comparing each expression with fixation baseline revealed an abnormal (increased) amygdala response to neutral but not angry faces in both groups of CD relative to controls. For sad vs neutral faces, reduced amygdala activation was observed in EO-CD relative to AO-CD and control participants. Comparing each expression with fixation revealed hypoactive amygdala responses to sadness in individuals with EO-CD relative to AO-CD participants and controls. These findings were not accounted for by attention-deficit/hyperactivity disorder symptoms.

CONCLUSIONS

Neurophysiological abnormalities are observed in both CD subtypes, contrary to the developmental taxonomic theory of CD. Additional amygdala hypofunction in relation to sad expressions might indicate why EO-CD is more severe and persistent than AO-CD.

Following gaze: gaze-following behavior as a window into social cognition

In general, individuals look where they attend and next intend to act. Many animals, including our own species, use observed gaze as a deictic (“pointing”) cue to guide behavior. Among humans, these responses are reflexive and pervasive: they arise within a fraction of a second, act independently of task relevance, and appear to undergird our initial development of language and theory of mind. Human and nonhuman animals appear to share basic gaze-following behaviors, suggesting the foundations of human social cognition may also be present in nonhuman brains.

Connectivity analysis reveals a cortical network for eye gaze perception

Haxby et al. (Haxby JV, Hoffman EA, Gobbini MI. 2000. The distributed human neural system for face perception. Trends Cogn Sci. 4:223–233.) proposed that eye gaze processing results from an interaction between a “core” face-specific system involved in visual analysis and an “extended” system involved in spatial attention, more generally. However, the full gaze perception network has remained poorly specified. In the context of a functional magnetic resonance imaging study, we used psychophysiological interactions (PPIs) to identify brain regions that showed differential connectivity (correlation) with core face perception structures (posterior superior temporal sulcus [pSTS] and fusiform gyrus [FG]) when viewing gaze shifts relative to control eye movements (opening/closing the eyes). The PPIs identified altered connectivity between the pSTS and MT/V5, intraparietal sulcus, frontal eye fields, superior temporal gyrus (STG), supramarginal gyrus, and middle frontal gyrus (MFG). The FG showed altered connectivity with the same areas of the STG and MFG, demonstrating the contribution of both dorsal and ventral core face areas to gaze perception. We propose that this network provides an interactive system that alerts us to seen changes in other agents’ gaze direction, makes us aware of their altered focus of spatial attention, and prepares a corresponding shift in our own attention.

What shape are the neural response functions underlying opponent coding in face space? A psychophysical investigation

Recent evidence has shown that face space represents facial identity information using two-pool opponent coding. Here we ask whether the shape of the monotonic neural response functions underlying such coding is linear (i.e. face space codes all equal-sized physical changes with equal sensitivity) or nonlinear (e.g. face space shows greater coding sensitivity around the average face). Using adaptation aftereffects and pairwise discrimination tasks, our results for face attributes of eye height and mouth height demonstrate linear shape; including for bizarre faces far outside the normal range. We discuss how linear coding explains some results in the previous literature, including failures to find that adaptation enhances face discrimination, and suggest possible reasons why face space can maintain detailed coding of values far outside the normal range. We also discuss specific nonlinear coding models needed to explain other findings, and conclude face space appears to use a mixture of linear and nonlinear representations.

The role of gaze direction in face viewpoint aftereffect

Face viewpoint aftereffect is a visual illusion that, after adaptation to a face side view, the perceived view direction of the same face subsequently presented near its front view is biased in a direction opposite to that of the adapted view. Eye gaze is a unique component in face not only because its direction is relatively independent of face view direction, but also because it is a primary cue for conveying social attention. Here, we studied the contribution of gaze direction adaptation to the formation of face viewpoint aftereffect. We found that a tiny (in terms of relative area) change of gaze direction in adapting face stimuli could induce a dramatic reduction in the magnitude of face viewpoint aftereffect. However, vertical inversion of the face stimuli almost abolished the reduction. Implications of these findings about face view representation and gaze direction representation are discussed.

About turn: the visual representation of human body orientation revealed by adaptation

Body orientation provides an important cue to other individuals' focus of attention, particularly when one is viewing them at a distance. Single-cell recording in macaques has identified cells in the superior temporal sulcus that show a view-selective response to particular body orientations. Whether similar separable coding is found in humans is not known, and there is currently no functional account of the visual representation of seen body orientation. This study addressed this issue using visual adaptation. Experiment 1 demonstrated distinct channels that code left- and right-oriented bodies. Experiment 2 investigated whether the visual representation of body orientation is best accounted for by an opponent-coding system, which has been shown to account for the visual representation of facial identity, or by a multichannel system, which provides the optimal account of coding line orientation and direction of motion. Our results provide evidence for multichannel coding of seen body orientation, with separate channels (or neuronal populations) selectively tuned to different body directions.