Gaze constancy in upright and inverted faces

The integration of head and body cues during the perception of social interactions

Humans spend a large proportion of time participating in social interactions. The ability to accurately detect and respond to human interactions is vital for social functioning, from early childhood through to older adulthood. This detection ability arguably relies on integrating sensory information from the interactants. Within the visual modality, directional information from a person's eyes, head, and body are integrated to inform where another person is looking and who they are interacting with. To date, social cue integration research has focused largely on the perception of isolated individuals. Across two experiments, we investigated whether observers integrate body information with head information when determining whether two people are interacting, and manipulated frame of reference (one of the interactants facing observer vs. facing away from observer) and the eye-region visibility of the interactant. Results demonstrate that individuals integrate information from the body with head information when perceiving dyadic interactions, and that integration is influenced by the frame of reference and visibility of the eye-region. Interestingly, self-reported autistics traits were associated with a stronger influence of body information on interaction perception, but only when the eye-region was visible. This study investigated the recognition of dyadic interactions using whole-body stimuli while manipulating eye visibility and frame of reference, and provides crucial insights into social cue integration, as well as how autistic traits affect cue integration, during perception of social interactions.

The roles of gaze and head orientation in face categorization during rapid serial visual presentation

Little is known about how perceived gaze direction and head orientation may influence human categorization of visual stimuli as faces. To address this question, a sequence of unsegmented natural images, each containing a random face or a non-face object, was presented in rapid succession (stimulus duration: 91.7 ms per image) during which human observers were instructed to respond immediately to every face presentation. Faces differed in gaze and head orientation in 7 combinations - full-front views with perceived gaze (1) directed to the observer, (2) averted to the left, or (3) averted to the right, left ¾ side views with (4) direct gaze or (5) averted gaze, and right ¾ side views with (6) direct gaze or (7) averted gaze - were presented randomly throughout the sequence. We found highly accurate and rapid behavioural responses to all kinds of faces. Crucially, both perceived gaze direction and head orientation had comparable, non-interactive effects on response times, where direct gaze was responded faster than averted gaze by 48 ms and full-front view faster than ¾ side view also by 48 ms on average. Presentations of full-front faces with direct gaze led to an additive speed advantage of 96 ms to ¾ faces with averted gaze. The results reveal that the effects of perceived gaze direction and head orientation on the speed of face categorization probably depend on the degree of social relevance of the face to the viewer.

A sparkle in the eye: Illumination cues and lightness constancy in the perception of eye contact

In social interactions, our sense of when we have eye contact with another person relies on the distribution of luminance across their eye region, reflecting the position of the darker iris within the lighter sclera of the human eye. This distribution of luminance can be distorted by the lighting conditions, consistent with the fundamental challenge that the visual system faces in distinguishing the nature of a surface from the pattern of light falling upon it. Here we perform a set of psychophysics experiments in human observers to investigate how illumination impacts on the perception of eye contact. First, we find that simple changes in the direction of illumination can produce systematic biases in our sense of when we have eye contact with another person. Second, we find that the visual system uses information about the lighting conditions to partially discount or 'explain away' the effects of illumination in this context, leading to a significantly more robust sense of when we have eye contact with another person. Third, we find that perceived eye contact is affected by specular reflections from the eye surface in addition to shading patterns, implicating eye glint as a potential cue to gaze direction. Overall, this illustrates how our interpretation of social signals relies on visual mechanisms that both compensate for the effects of illumination on retinal input and potentially exploit novel cues that illumination can produce.

Task Dependent Effects of Head Orientation on Perceived Gaze Direction

The perception of gaze direction involves the integration of a number of sensory cues exterior to the eye-region. The orientation of the head is one such cue, which has an overall repulsive effect on the perceived direction of gaze. However, in a recent experiment, we found the measured effect of head orientation on perceived gaze direction differed within subjects, depending on whether a single- or two-interval task design was employed. This suggests a potential difference in the way the orientation of the head is integrated into the perception of gaze direction across tasks. Four experiments were conducted to investigate this difference. The first two experiments showed that the difference was not the result of some interaction between stimuli in the two-interval task, but rather, a difference between the types of judgment being made across tasks, where observers were making a directional (left/right) judgment in the single-interval task, and a non-directional (direct/indirect gaze) judgment in the two-interval task. A third experiment showed that this difference does not arise from observers utilizing a non-directional cue to direct gaze (the circularity of the pupil/iris) in making their non-directional judgments. The fourth experiment showed no substantial differences in the duration of evidence accumulation and processing between judgments, suggesting that observers are not integrating different sensory information across tasks. Together these experiments show that the sensory information from head orientation is flexibly weighted in the perception of gaze direction, and that the purpose of the observer, in sampling gaze information, can influence the consequent perception of gaze direction.

Perceptual integration of head and eye cues to gaze direction in schizophrenia

The perceptual mechanisms that underlie social experience in schizophrenia are increasingly becoming a target of empirical research. In the context of low-level vision, there is evidence for a reduction in the integration of sensory features in schizophrenia (e.g. increased thresholds for contour detection and motion coherence). In the context of higher-level vision, comparable differences in the integration of sensory features of the face could in theory impair the recognition of important social cues. Here we examine how the sense of where other people are looking relies upon the integration of eye-region cues and head-region cues. Adults with schizophrenia viewed face images designed to elicit the 'Wollaston illusion', a perceptual phenomenon in which the perceived gaze direction associated with a given pair of eyes is modulated by the surrounding sensory context. We performed computational modelling of these psychophysical data to quantify individual differences in the use of facial cues to gaze direction. We find that adults with schizophrenia exhibit a robust perceptual effect whereby their sense of other people's direction of gaze is strongly biased by sensory cues relating to head orientation in addition to eye region information. These results indicate that the visual integration of facial cues to gaze direction in schizophrenia is intact, helping to constrain theories of reduced integrative processing in higher-level and lower-level vision. In addition, robust gaze processing was evident in the tested participants despite reduced performance on a theory of mind task designed to assess higher-level social cognition.

Context Modulates Congruency Effects in Selective Attention to Social Cues

Head and gaze directions are used during social interactions as essential cues to infer where someone attends. When head and gaze are oriented toward opposite directions, we need to extract socially meaningful information despite stimulus conflict. Recently, a cognitive and neural mechanism for filtering-out conflicting stimuli has been identified while performing non-social attention tasks. This mechanism is engaged proactively when conflict is anticipated in a high proportion of trials and reactively when conflict occurs infrequently. Here, we investigated whether a similar mechanism is at play for limiting distraction from conflicting social cues during gaze or head direction discrimination tasks in contexts with different probabilities of conflict. Results showed that, for the gaze direction task only (Experiment 1), inverse efficiency (IE) scores for distractor-absent trials (i.e., faces with averted gaze and centrally oriented head) were larger (indicating worse performance) when these trials were intermixed with congruent/incongruent distractor-present trials (i.e., faces with averted gaze and tilted head in the same/opposite direction) relative to when the same distractor-absent trials were shown in isolation. Moreover, on distractor-present trials, IE scores for congruent (vs. incongruent) head-gaze pairs in blocks with rare conflict were larger than in blocks with frequent conflict, suggesting that adaptation to conflict was more efficient than adaptation to infrequent events. However, when the task required discrimination of head orientation while ignoring gaze direction, performance was not impacted by both block-level and current trial congruency (Experiment 2), unless the cognitive load of the task was increased by adding a concurrent task (Experiment 3). Overall, our study demonstrates that during attention to social cues proactive cognitive control mechanisms are modulated by the expectation of conflicting stimulus information at both the block- and trial-sequence level, and by the type of task and cognitive load. This helps to clarify the inherent differences in the distracting potential of head and gaze cues during speeded social attention tasks.

Functional Mechanisms Encoding Others' Direction of Gaze in the Human Nervous System

The direction of others' gaze is a strong social signal to their intentions and future behavior. Pioneering electrophysiological research identified cell populations in the primate visual cortex that are tuned to specific directions of observed gaze, but the functional architecture of this system is yet to be precisely specified. Here, we develop a computational model of how others' gaze direction is flexibly encoded across sensory channels within the gaze system. We incorporate the divisive normalization of sensory responses—a computational mechanism that is thought to be widespread in sensory systems but has not been examined in the context of social vision. We demonstrate that the operation of divisive normalization in the gaze system predicts a surprising and distinctive pattern of perceptual changes after sensory adaptation to gaze stimuli and find that these predictions closely match the psychophysical effects of adaptation in human observers. We also find that opponent coding, broadband multichannel, and narrowband multichannel models of sensory coding make distinct predictions regarding the effects of adaptation in a normalization framework and find evidence in favor of broadband multichannel coding of gaze. These results reveal the functional principles that govern the neural encoding of gaze direction and support the notion that divisive normalization is a canonical feature of nervous system function. Moreover, this research provides a strong foundation for testing recent computational theories of neuropsychiatric conditions in which gaze processing is compromised, such as autism and schizophrenia.

Detecting and identifying offset gaze

A number of experiments have demonstrated that observers can accurately identify stimuli that they fail to detect (Rollman and Nachmias, 1972; Harris and Fahle, 1995; Allik et al. 1982, 2014). Using a 2x2AFC double judgements procedure, we demonstrated an analogous pattern of performance in making judgements about the direction of eye gaze. Participants were shown two faces in succession: one with direct gaze and one with gaze offset to the left or right. We found that they could identify the direction of gaze offset (left/right) better than they could detect which face contained the offset gaze. A simple Thurstonian model, under which the detection judgement is shown to be more computationally complex, was found to explain the empirical data. A further experiment incorporated metacognitive ratings into the double judgements procedure to measure observers' metacognitive awareness (Meta-d') across the two judgements and to assess whether observers were aware of the evidence for offset gaze when detection performance was at and below threshold. Results suggest that metacognitive awareness is tied to performance, with approximately equal Meta-d' across the two judgements, when sensitivity is taken into account. These results show that both performance and metacognitive awareness rely not only on the strength of sensory evidence but also on the computational complexity of the decision, which determines the relative distance of that evidence from the decision axes.

A bias-minimising measure of the influence of head orientation on perceived gaze direction

The orientation of the head is an important cue for gaze direction, and its role has been explained in a dual route model. The model incorporates both an attractive and a repulsive effect of head orientation, which act to support accurate gaze perception across large changes in natural stimuli. However, in all previous studies of which we are aware, measurements of the influence of head orientation on perceived gaze direction were obtained using a single-interval methodology, which may have been affected by response bias. Here we compare the single-interval methodology with a two-interval (bias-minimising) design. We find that although measures obtained using the two-interval design showed a stronger attractive effect of head orientation than previous studies, the influence of head orientation on perceived gaze direction still represents a genuine perceptual effect. Measurements obtained using the two-interval design were also shown to be more stable across sessions one week apart. These findings suggest the two-interval design should be used in future experiments, especially if comparing groups who may systematically differ in their biases, such as patients with schizophrenia or autism.

Asymmetry in Gaze Direction Discrimination Between the Upper and Lower Visual Fields

Previous research has shown that gaze direction can only be accurately discriminated within parafoveal limits (∼5° eccentricity) along the horizontal visual field. Beyond this eccentricity, head orientation seems to influence gaze discrimination more than iris cues. The present study examined gaze discrimination performance in the upper visual field (UVF) and lower visual field (LVF), and whether head orientation affects gaze judgments beyond parafoveal vision. Direct and averted gaze faces, in frontal and deviated head orientations, were presented for 150 ms along the vertical meridian while participants maintained central fixation during gaze discrimination judgments. Gaze discrimination was above chance level at all but one eccentricity for the two gaze-head congruent conditions. In contrast, for the incongruent conditions, gaze was discriminated above chance only from -1.5° to +3°, with an asymmetry between the UVF and LVF. Beyond foveal vision, response rates were biased toward head orientation rather than iris eccentricity, occurring in the LVF for both head orientations, and in the UVF for frontal head views. These findings suggest that covert processing of gaze direction involves the integration of eyes and head cues, with congruency of these two social cues driving response differences between the LVF and the UVF.