Viewers extract the mean from images of the same person: A route to face learning

A cognitive template for human face detection

Faces are highly informative social stimuli, yet before any information can be accessed, the face must first be detected in the visual field. A detection template that serves this purpose must be able to accommodate the wide variety of face images we encounter, but how this generality could be achieved remains unknown. In this study, we investigate whether statistical averages of previously encountered faces can form the basis of a general face detection template. We provide converging evidence from a range of methods-human similarity judgements and PCA-based image analysis of face averages (Experiment 1-3), human detection behaviour for faces embedded in complex scenes (Experiment 4 and 5), and simulations with a template-matching algorithm (Experiment 6 and 7)-to examine the formation, stability and robustness of statistical image averages as cognitive templates for human face detection. We integrate these findings with existing knowledge of face identification, ensemble coding, and the development of face perception.

Multiple images captured from a single encounter do not promote face learning

Viewing multiple images of a newly encountered face improves recognition of that identity in new instances. Studies examining face learning have presented high-variability (HV) images that incorporate changes that occur from moment-to-moment (e.g., head orientation and expression) and over time (e.g., lighting, hairstyle, and health). We examined whether low-variability (LV) images (i.e., images that incorporate only moment-to-moment changes) also promote generalisation of learning such that novel instances are recognised. Participants viewed a single image, six LV images, or six HV images of a target identity before being asked to recognise novel images of that identity in a face matching task (training stimuli remained visible) or a memory task (training stimuli were removed). In Experiment 1 (n = 71), participants indicated which image(s) in 8-image arrays belonged to the target identity. In Experiment 2 (n = 73), participants indicated whether sequentially presented images belonged to the target identity. Relative to the single-image condition, sensitivity to identity improved and response biases were less conservative in the HV condition; we found no evidence of generalisation of learning in the LV condition regardless of testing protocol. Our findings suggest that day-to-day variability in appearance plays an essential role in acquiring expertise with a novel face.

Knocking out the LRRK2 gene increases sensitivity to wavelength information in rats

Leucine-rich repeat kinase 2 (LRRK2) is a gene related to familial Parkinson's disease (PD). It has been associated with nonmotor symptoms such as disturbances in the visual system affecting colour discrimination and contrast sensitivity. This study examined how deficiency of LRRK2 impacts visual processing in adult rats. Additionally, we investigated whether these changes can be modelled in wild-type rats by administering the LRRK2 inhibitor PFE360. Visual evoked potentials (VEPs) and steady-state visual evoked potentials (SSVEPs) were recorded in the visual cortex and superior colliculus of female LRRK2-knockout and wild-type rats to study how the innate absence of LRRK2 changes visual processing. Exposing the animals to stimulation at five different wavelengths revealed an interaction between genotype and the response to stimulation at different wavelengths. Differences in VEP amplitudes and latencies were robust and barely impacted by the presence of the LRRK2 inhibitor PFE360, suggesting a developmental effect. Taken together, these results indicate that alterations in visual processing were related to developmental deficiency of LRRK2 and not acute deficiency of LRRK2, indicating a role of LRRK2 in the functional development of the visual system and synaptic transmission.

Ensemble coding of facial identity is robust, but may not contribute to face learning

Ensemble coding - the rapid extraction of a perceptual average - has been proposed as a potential mechanism underlying face learning. We tested this proposal across five pre-registered experiments in which four ambient images of an identity were presented in the study phase. In Experiments 1 and 2a-c, participants were asked whether a test image was in the study array; these experiments examined the robustness of ensemble coding. Experiment 1 replicated ensemble coding in an online sample; participants recognize images from the study array and the average of those images. Experiments 2a-c provide evidence that ensemble coding meets several criteria of a possible learning mechanism: It is robust to changes in head orientation (± 60o), survives a short (30s) delay, and persists when images of two identities are interleaved during the study phase. Experiment 3 examined whether ensemble coding is sufficient for face learning (i.e., facilitates recognition of novel images of a target identity). Each study array comprised four ambient images (variability + average), a single image, or an average of four images (average only). Participants were asked whether a novel test image showed the identity from a study array. Performance was best in the four-image condition, with no difference between the single-image and average-only conditions. We conclude that ensemble coding of facial identity is robust but that the perceptual average per se is not sufficient for face learning.

Refinement of face representations by exposure reveals different time scales of biases in face processing

Experience modulates face processing abilities so that face discrimination and recognition improve with development, especially for more frequently experienced faces (e.g., own-race faces). Although advanced models describe how experience generally modulates perception, the mechanism by which exposure refines internal perceptual representations of faces is unknown. To address this issue, we investigated the effect of short- and long-term experienced stimulus history on face processing. Participants performed same-different judgments in a serial discrimination task where two consecutive faces were drawn from a distribution of morphed faces. Use of stimulus statistics was measured by testing the gravitation of face representations towards the mean of a range of morphed faces around which they were sampled (regression-to-the-mean). The results demonstrated regression of face representations towards the experienced mean and the retention of stimulus statistics over days. In trials where regression facilitated discrimination, the bias diminished the otherwise disadvantage of other-race over own-races faces. The dynamics of the perceptual bias, probed by trial-by-trial performance, further indicated different timescales of the bias, depending on perceptual expertise: people with weak face-recognition skills showed the use of a stable reference, built on long-term statistics accumulated over many trials, along with an updating of this reference by recent trials. In contrast, the strong face recognizers showed a different pattern where sequential effects mostly contributed to discrimination, with relatively minimal reliance on the long-term average for other-race faces. The findings suggest a mechanism by which exposure refines face representations and reveal, for the first time in adults, associations between levels of specialization of perceptual representations and the extent to which these representations become narrowly tuned.

Modeling Biological Face Recognition with Deep Convolutional Neural Networks

Deep convolutional neural networks (DCNNs) have become the state-of-the-art computational models of biological object recognition. Their remarkable success has helped vision science break new ground, and recent efforts have started to transfer this achievement to research on biological face recognition. In this regard, face detection can be investigated by comparing face-selective biological neurons and brain areas to artificial neurons and model layers. Similarly, face identification can be examined by comparing in vivo and in silico multidimensional "face spaces." In this review, we summarize the first studies that use DCNNs to model biological face recognition. On the basis of a broad spectrum of behavioral and computational evidence, we conclude that DCNNs are useful models that closely resemble the general hierarchical organization of face recognition in the ventral visual pathway and the core face network. In two exemplary spotlights, we emphasize the unique scientific contributions of these models. First, studies on face detection in DCNNs indicate that elementary face selectivity emerges automatically through feedforward processing even in the absence of visual experience. Second, studies on face identification in DCNNs suggest that identity-specific experience and generative mechanisms facilitate this particular challenge. Taken together, as this novel modeling approach enables close control of predisposition (i.e., architecture) and experience (i.e., training data), it may be suited to inform long-standing debates on the substrates of biological face recognition.

From Pictures to the People in Them: Averaging Within-Person Variability Leads to Face Familiarization

Familiar faces can be confidently recognized despite sometimes radical changes in their appearance. Exposure to within-person variability-differences in facial characteristics over successive encounters-contributes to face familiarization. Research also suggests that viewers create mental averages of the different views of faces they encounter while learning them. Averaging over within-person variability is thus a promising mechanism for face familiarization. In Experiment 1, 153 Canadian undergraduates (88 female; age: M = 21 years, SD = 5.24) learned six target identities from eight different photos of each target interspersed among 32 distractor identities. Face-matching accuracy improved similarly irrespective of awareness of the target's identity, confirming that target faces presented among distractors can be learned incidentally. In Experiment 2, 170 Canadian undergraduates (125 female; age: M = 22.6 years, SD = 6.02) were tested using a novel indirect measure of learning. The results show that viewers update a mental average of a person's face as it becomes learned. Our findings are the first to show how averaging within-person variability over time leads to face familiarization.

Incomplete faces are completed using a more average face

Facial first impressions are known to influence how we behave towards others. As a result of the COVID-19 pandemic, we often view incomplete faces due to the commonplace wearing of face masks. Previous research has shown that perceptions of attractiveness are often increased due to these coverings, with initial evidence suggesting that this may be caused by viewers using a mental representation of the average face to complete any missing information. Here, we directly address this hypothesis by presenting participants with incomplete faces (either the lower or upper half removed) and asking them to decide how they thought the actual, full face looked. Participants were able to manipulate the missing half of the face onscreen by increasing or decreasing the averageness of its shape. Our results demonstrated that participants did not select the original versions of the faces but instead chose more average versions when manipulating both the lower and upper face. Further, the typicality of the original image influenced responses, with less typical faces (in comparison with more typical ones) being completed using an even more average version of the missing half of the faces. Taken together, these findings provide the first direct evidence that people utilise an average/typical internal representation when inferring information about incomplete faces. This result has theoretical importance in terms of visual perception, as well as real-world relevance in a time where face masks are commonplace due to the COVID-19 pandemic.

The aftereffect of the ensemble average of facial expressions on subsequent facial expression recognition

An ensemble or statistical summary can be extracted from facial expressions presented in different spatial locations simultaneously. However, how such complicated objects are represented in the mind is not clear. It is known that the aftereffect of facial expressions, in which prolonged viewing of facial expressions biases the perception of subsequent facial expressions of the same category, occurs only when a visual representation is formed. Using this methodology, we examined whether an ensemble can be represented with visualized information. Experiment 1 revealed that the presentation of multiple facial expressions biased the perception of subsequent facial expressions to less happy as much as the presentation of a single face did. Experiment 2 compared the presentation of faces comprising strong and weak intensities of emotional expressions with an individual face as the adaptation stimulus. The results indicated that the perceptual biases were found after the presentation of four faces and a strong single face, but not after the weak single face presentation. Experiment 3 employed angry expressions, a distinct category from the test expression used as an adaptation stimulus; no aftereffect was observed. Finally, Experiment 4 clearly demonstrated the perceptual bias with a higher number of faces. Altogether, these results indicate that an ensemble average extracted from multiple faces leads to the perceptual bias, and this effect is similar in terms of its properties to that of a single face. This supports the idea that an ensemble of faces is represented with visualized information as a single face.

What happens to our representation of identity as familiar faces age? Evidence from priming and identity aftereffects

Matching identity in images of unfamiliar faces is error prone, but we can easily recognize highly variable images of familiar faces – even images taken decades apart. Recent theoretical development based on computational modelling can account for how we recognize extremely variable instances of the same identity. We provide complementary behavioural data by examining older adults’ representation of older celebrities who were also famous when young. In Experiment 1, participants completed a long‐lag repetition priming task in which primes and test stimuli were the same age or different ages. In Experiment 2, participants completed an identity after effects task in which the adapting stimulus was an older or young photograph of one celebrity and the test stimulus was a morph between the adapting identity and a different celebrity; the adapting stimulus was the same age as the test stimulus on some trials (e.g., both old) or a different age (e.g., adapter young, test stimulus old). The magnitude of priming and identity after effects were not influenced by whether the prime and adapting stimulus were the same age or different age as the test face. Collectively, our findings suggest that humans have one common mental representation for a familiar face (e.g., Paul McCartney) that incorporates visual changes across decades, rather than multiple age‐specific representations. These findings make novel predictions for state‐of‐the‐art algorithms (e.g., Deep Convolutional Neural Networks).

Learning faces from variability: Four- and five-year-olds differ from older children and adults

Children under 6 years of age have difficulty recognizing a familiar face across changes in appearance and telling the face apart from similar-looking people. Understanding the process by which newly encountered faces become familiar can provide insights into these difficulties. Exposure to the ways in which a person varies in appearance is one mechanism by which adults and older children (≥6 years) learn new faces. We provide the first investigation of whether this mechanism for face learning functions in younger children. Children aged 4 and 5 years were read two storybooks featuring an unfamiliar character. Participants viewed six images of the character in one story and one image of the character in the other story. After each story, children were asked to identify novel images of the character that were intermixed with images of a similar-looking distractor. Like older children, 4- and 5-year-olds were more sensitive to identity in the 6-image condition, but they also adapted a less conservative criterion. Young children identified more images of the character after viewing six images versus one image. However, many also incorrectly identified more images of the distractor after viewing six images versus one image, an effect not previously found for older children and adults. These results suggest that this mechanism for face learning is not fully refined before 6 years of age.

Naturalistic Face Learning in Infants and Adults

Everyday face recognition presents a difficult challenge because faces vary naturally in appearance as a result of changes in lighting, expression, viewing angle, and hairstyle. We know little about how humans develop the ability to learn faces despite natural facial variability. In the current study, we provide the first examination of attentional mechanisms underlying adults' and infants' learning of naturally varying faces. Adults (n = 48) and 6- to 12-month-old infants (n = 48) viewed videos of models reading a storybook; the facial appearance of these models was either high or low in variability. Participants then viewed the learned face paired with a novel face. Infants showed adultlike prioritization of face over nonface regions; both age groups fixated the face region more in the high- than low-variability condition. Overall, however, infants showed less ability to resist contextual distractions during learning, which potentially contributed to their lack of discrimination between the learned and novel faces. Mechanisms underlying face learning across natural variability are discussed.

Within-person variability can improve the identification of unfamiliar faces across changes in viewpoint

Unfamiliar face identification is concerningly error prone, especially across changes in viewing conditions. Within-person variability has been shown to improve matching performance for unfamiliar faces, but this has only been demonstrated using images of a front view. In this study, we test whether the advantage of within-person variability from front views extends to matching to target images of a face rotated in view. Participants completed either a simultaneous matching task (Experiment 1) or a sequential matching task (Experiment 2) in which they were tested on their ability to match the identity of a face shown in an array of either one or three ambient front-view images, with a target image shown in front, three-quarter, or profile view. While the effect was stronger in Experiment 2, we found a consistent pattern in match trials across both experiments in that there was a multiple image matching benefit for front, three-quarter, and profile-view targets. We found multiple image effects for match trials only, indicating that providing observers with multiple ambient images confers an advantage for recognising different images of the same identity but not for discriminating between images of different identities. Signal detection measures also indicate a multiple image advantage despite a more liberal response bias for multiple image trials. Our results show that within-person variability information for unfamiliar faces can be generalised across views and can provide insights into the initial processes involved in the representation of familiar faces.

Individual Faces Were Not Discarded During Extracting Mean Emotion Representations

Individuals can perceive the mean emotion or mean identity of a group of faces. It has been considered that individual representations are discarded when extracting a mean representation; for example, the "element-independent assumption" asserts that the extraction of a mean representation does not depend on recognizing or remembering individual items. The "element-dependent assumption" proposes that the extraction of a mean representation is closely connected to the processing of individual items. The processing mechanism of mean representations and individual representations remains unclear. The present study used a classic member-identification paradigm and manipulated the exposure time and set size to investigate the effect of attentional resources allocated to individual faces on the processing of both the mean emotion representation and individual representations in a set and the relationship between the two types of representations. The results showed that while the precision of individual representations was affected by attentional resources, the precision of the mean emotion representation did not change with it. Our results indicate that two different pathways may exist for extracting a mean emotion representation and individual representations and that the extraction of a mean emotion representation may have higher priority. Moreover, we found that individual faces in a group could be processed to a certain extent even under extremely short exposure time and that the precision of individual representations was relatively poor but individual representations were not discarded.

Similar use of shape and texture cues for own- and other-race faces during face learning and recognition

Although the other-race effect (ORE; superior recognition of own- relative to other-race faces) is well established, the mechanisms underlying it are not well understood. We examined whether the ORE is attributable to differential use of shape and texture cues for own- vs. other-race faces. Shape cues are particularly important for detecting that an own-race face is unfamiliar, whereas texture cues are more important for recognizing familiar and newly learned own-race faces. We compared the influence of shape and texture cues on Caucasian participants' recognition of Caucasian and East Asian faces using two complementary approaches. In Experiment 1, participants studied veridical, shape-caricatured, or texture-caricatured faces and then were asked to recognize them in an old/new recognition task. In Experiment 2, all study faces were veridical and we independently removed the diagnosticity of shape (or texture) cues in the test phase by replacing original shape (or texture) with average shape (or texture). Despite an overall own-race advantage, participants' use of shape and texture cues was comparable for own- and other-race faces. These results suggest that the other-race effect is not attributable to qualitative differences in the use of shape and texture cues.

Multiple-image arrays in face matching tasks with and without memory

Previous research has shown that exposure to within-person variability facilitates face learning. A different body of work has examined potential benefits of providing multiple images in face matching tasks. Viewers are asked to judge whether a target face matches a single face image (as when checking photo-ID) or multiple face images of the same person. The evidence here is less clear, with some studies finding a small multiple-image benefit, and others finding no advantage. In four experiments, we address this discrepancy in the benefits of multiple images from learning and matching studies. We show that multiple-image arrays only facilitate face matching when arrays precede targets. Unlike simultaneous face matching tasks, sequential matching and learning tasks involve memory and require abstraction of a stable representation of the face from the array, for subsequent comparison with a target. Our results show that benefits from multiple-image arrays occur only when this abstraction is required, and not when array and target images are available at once. These studies reconcile apparent differences between face learning and face matching and provide a theoretical framework for the study of within-person variability in face perception.

Seeing through disguise: Getting to know you with a deep convolutional neural network

People use disguise to look unlike themselves (evasion) or to look like someone else (impersonation). Evasion disguise challenges human ability to see an identity across variable images; Impersonation challenges human ability to tell people apart. Personal familiarity with an individual face helps humans to see through disguise. Here we propose a model of familiarity based on high-level visual learning mechanisms that we tested using a deep convolutional neural network (DCNN) trained for face identification. DCNNs generate a face space in which identities and images co-exist in a unified computational framework, that is categorically structured around identity, rather than retinotopy. This allows for simultaneous manipulation of mechanisms that contrast identities and cluster images. In Experiment 1, we measured the DCNN's baseline accuracy (unfamiliar condition) for identification of faces in no disguise and disguise conditions. Disguise affected DCNN performance in much the same way it affects human performance for unfamiliar faces in disguise (cf. Noyes & Jenkins, 2019). In Experiment 2, we simulated familiarity for individual identities by averaging the DCNN-generated representations from multiple images of each identity. Averaging improved DCNN recognition of faces in evasion disguise, but reduced the ability of the DCNN to differentiate identities of similar appearance. In Experiment 3, we implemented a contrast learning technique to simultaneously teach the DCNN appearance variation and identity contrasts between different individuals. This facilitated identification with both evasion and impersonation disguise. Familiar face recognition requires an ability to group images of the same identity together and separate different identities. The deep network provides a high-level visual representation for face recognition that supports both of these mechanisms of face learning simultaneously.

Logarithmic encoding of ensemble time intervals

Although time perception is based on the internal representation of time, whether the subjective timeline is scaled linearly or logarithmically remains an open issue. Evidence from previous research is mixed: while the classical internal-clock model assumes a linear scale with scalar variability, there is evidence that logarithmic timing provides a better fit to behavioral data. A major challenge for investigating the nature of the internal scale is that the retrieval process required for time judgments may involve a remapping of the subjective time back to the objective scale, complicating any direct interpretation of behavioral findings. Here, we used a novel approach, requiring rapid intuitive ‘ensemble’ averaging of a whole set of time intervals, to probe the subjective timeline. Specifically, observers’ task was to average a series of successively presented, auditory or visual, intervals in the time range 300–1300 ms. Importantly, the intervals were taken from three sets of durations, which were distributed such that the arithmetic mean (from the linear scale) and the geometric mean (from the logarithmic scale) were clearly distinguishable. Consistently across the three sets and the two presentation modalities, our results revealed subjective averaging to be close to the geometric mean, indicative of a logarithmic timeline underlying time perception.

Classification of α-synuclein-induced changes in the AAV α-synuclein rat model of Parkinson's disease using electrophysiological measurements of visual processing

Biomarkers suitable for early diagnosis and monitoring disease progression are the cornerstone of developing disease-modifying treatments for neurodegenerative diseases such as Parkinson's disease (PD). Besides motor complications, PD is also characterized by deficits in visual processing. Here, we investigate how virally-mediated overexpression of α-synuclein in the substantia nigra pars compacta impacts visual processing in a well-established rodent model of PD. After a unilateral injection of vector, human α-synuclein was detected in the striatum and superior colliculus (SC). In parallel, there was a significant delay in the latency of the transient VEPs from the affected side of the SC in late stages of the disease. Inhibition of leucine-rich repeat kinase using PFE360 failed to rescue the VEP delay and instead increased the latency of the VEP waveform. A support vector machine classifier accurately classified rats according to their `disease state' using frequency-domain data from steady-state visual evoked potentials (SSVEP). Overall, these findings indicate that the latency of the rodent VEP is sensitive to changes mediated by the increased expression of α-synuclein and especially when full overexpression is obtained, whereas the SSVEP facilitated detection of α-synuclein across reflects all stages of PD model progression.

Individual Differences in Serial Dependence of Facial Identity are Associated with Face Recognition Abilities

Serial dependence is a perceptual bias where current perception is biased towards prior visual input. This bias occurs when perceiving visual attributes, such as facial identity, and has been argued to play an important functional role in vision, stabilising the perception of objects through integration. In face identity recognition, this bias could assist in building stable representations of facial identity. If so, then individual variation in serial dependence could contribute to face recognition ability. To investigate this possibility, we measured both the strength of serial dependence and the range over which individuals showed this bias (the tuning) in 219 adults, using a new measure of serial dependence of facial identity. We found that better face recognition was associated with stronger serial dependence and narrower tuning, that is, showing serial dependence primarily when sequential faces were highly similar. Serial dependence tuning was further found to be a significant predictor of face recognition abilities independently of both object recognition and face identity aftereffects. These findings suggest that the extent to which serial dependence is used selectively for similar faces is important to face recognition. Our results are consistent with the view that serial dependence plays a functional role in face recognition.

Listeners form average-based representations of individual voice identities

Models of voice perception propose that identities are encoded relative to an abstracted average or prototype. While there is some evidence for norm-based coding when learning to discriminate different voices, little is known about how the representation of an individual's voice identity is formed through variable exposure to that voice. In two experiments, we show evidence that participants form abstracted representations of individual voice identities based on averages, despite having never been exposed to these averages during learning. We created 3 perceptually distinct voice identities, fully controlling their within-person variability. Listeners first learned to recognise these identities based on ring-shaped distributions located around the perimeter of within-person voice spaces - crucially, these distributions were missing their centres. At test, listeners' accuracy for old/new judgements was higher for stimuli located on an untrained distribution nested around the centre of each ring-shaped distribution compared to stimuli on the trained ring-shaped distribution.

Social Evaluation of Faces Across Gender and Familiarity

Models of social evaluation aim to capture the information people use to form first impressions of unfamiliar others. However, little is currently known about the relationship between perceived traits across gender. In Study 1, we asked viewers to provide ratings of key social dimensions (dominance, trustworthiness, etc.) for multiple images of 40 unfamiliar identities. We observed clear sex differences in the perception of dominance—with negative evaluations of high dominance in unfamiliar females but not males. In Study 2, we used the social evaluation context to investigate the key predictions about the importance of pictorial information in familiar and unfamiliar face processing. We compared the consistency of ratings attributed to different images of the same identities and demonstrated that ratings of images depicting the same familiar identity are more tightly clustered than those of unfamiliar identities. Such results imply a shift from image rating to person rating with increased familiarity, a finding which generalises results previously observed in studies of identification.

Face averages and multiple images in a live matching task

We know from previous research that unfamiliar face matching (determining whether two simultaneously presented images show the same person or not) is very error-prone. A small number of studies in laboratory settings have shown that the use of multiple images or a face average, rather than a single image, can improve face matching performance. Here, we tested 1,999 participants using four-image arrays and face averages in two separate live matching tasks. Matching a single image to a live person resulted in numerous errors (79.9% accuracy across both experiments), and neither multiple images (82.4% accuracy) nor face averages (76.9% accuracy) improved performance. These results are important when considering possible alterations which could be made to photo-ID. Although multiple images and face averages have produced measurable improvements in performance in recent laboratory studies, they do not produce benefits in a real-world live face matching context.

Encoding differences affect the number and precision of own-race versus other-race faces stored in visual working memory

Other-race faces are discriminated and recognized less accurately than own-race faces. Despite a wealth of research characterizing this other-race effect (ORE), little is known about the nature of the representations of own-race versus other-race faces. This is because traditional measures of this ORE provide a binary measure of discrimination or recognition (correct/incorrect), failing to capture potential variation in the quality of face representations. We applied a novel continuous-response paradigm to independently measure the number of own-race and other-race face representations stored in visual working memory (VWM) and the precision with which they are stored. Participants reported target own-race or other-race faces on a circular face space that smoothly varied along the dimension of identity. Using probabilistic mixture modeling, we found that following ample encoding time, the ORE is attributable to differences in the probability of a face being maintained in VWM. Reducing encoding time, a manipulation that is more sensitive to encoding limitations, caused a loss of precision or an increase in variability of VWM for other-race but not own-race faces. These results suggest that the ORE is driven by the inefficiency with which other-race faces are rapidly encoded in VWM and provide novel insights about how perceptual experience influences the representation of own-race and other-race faces in VWM.

Memory of Ensemble Representation Was Independent of Attention

The hierarchical view of working memory suggested that object ensemble could also be stored into working memory by treating ensemble properties as single "unit." However, it remains unclear whether ensemble representation in working memory is vulnerable to attention demanding. The present study designed a dual-task paradigm constituting of a memory retaining task and an attention-demanding arrow flanker task. Participants were firstly presented an array (4 or 9) of facial images with neutral expressions and then shown a left- or right-orientated arrow surrounded by four congruent or incongruent oriented arrows or short lines. Participants judged the orientation of the target arrow and then indicated whether a probe facial image was present or absent in the preceding facial array. The probe face consisted of four conditions: (1) a morphed average face of prior face set, (2) a morphed average face of another face set, (3) an exemplar face of prior set, and (4) an exemplar face of another face set. Results confirmed that participants implicitly coded the average facial image of preceding set and retained in working memory. More importantly, the memory representation of ensemble property (e.g., average facial identity) was independent of flanker type. In sum, this study provided further evidence of the hierarchical view of working memory and suggested that attention was not a pre-requisite for the retaining of ensemble properties in working memory.

Becoming Familiar With a Newly Encountered Face: Evidence of an Own-Race Advantage

Adults' ability to match identity in images of unfamiliar faces is impaired for other- compared with own-race faces; their ability to match identity in images of familiar faces is independent of face race. Exposure to within-person variability in appearance plays a key role in face learning. Past research suggests that children need exposure to higher levels of variability than adults to learn a new face-a difference that has been attributed to experience. We predicted that adults' limited experience with other-race faces would result in their needing exposure to higher levels of variability when learning other- compared with own-race faces. We introduced adults to four new identities (two own-race; two other-race) in one of the three conditions: a single image, a low-variability video (filmed on 1 day), or a high-variability video (filmed across 3 days). Adults' ability to recognize new instances of learned identities improved in the low-variability condition for own-race faces but only in the high-variability condition for other-race faces. We discuss learning mechanisms that might drive this difference-a difference we attribute to experience.

Reprint of "Investigating ensemble perception of emotions in autistic and typical children and adolescents"

Ensemble perception, the ability to assess automatically the summary of large amounts of information presented in visual scenes, is available early in typical development. This ability might be compromised in autistic children, who are thought to present limitations in maintaining summary statistics representations for the recent history of sensory input. Here we examined ensemble perception of facial emotional expressions in 35 autistic children, 30 age- and ability-matched typical children and 25 typical adults. Participants received three tasks: a) an ‘ensemble’ emotion discrimination task; b) a baseline (single-face) emotion discrimination task; and c) a facial expression identification task. Children performed worse than adults on all three tasks. Unexpectedly, autistic and typical children were, on average, indistinguishable in their precision and accuracy on all three tasks. Computational modelling suggested that, on average, autistic and typical children used ensemble-encoding strategies to a similar extent; but ensemble perception was related to non-verbal reasoning abilities in autistic but not in typical children. Eye-movement data also showed no group differences in the way children attended to the stimuli. Our combined findings suggest that the abilities of autistic and typical children for ensemble perception of emotions are comparable on average.

Ensemble coding of face identity is present but weaker in congenital prosopagnosia

Individuals with congenital prosopagnosia (CP) are impaired at identifying individual faces but do not appear to show impairments in extracting the average identity from a group of faces (known as ensemble coding). However, possible deficits in ensemble coding in a previous study (CPs n = 4) may have been masked because CPs relied on pictorial (image) cues rather than identity cues. Here we asked whether a larger sample of CPs (n = 11) would show intact ensemble coding of identity when availability of image cues was minimised. Participants viewed a "set" of four faces and then judged whether a subsequent individual test face, either an exemplar or a "set average", was in the preceding set. Ensemble coding occurred when matching (vs. mismatching) averages were mistakenly endorsed as set members. We assessed both image- and identity-based ensemble coding, by varying whether test faces were either the same or different images of the identities in the set. CPs showed significant ensemble coding in both tasks, indicating that their performance was independent of image cues. As a group, CPs' ensemble coding was weaker than controls in both tasks, consistent with evidence that perceptual processing of face identity is disrupted in CP. This effect was driven by CPs (n= 3) who, in addition to having impaired face memory, also performed particularly poorly on a measure of face perception (CFPT). Future research, using larger samples, should examine whether deficits in ensemble coding may be restricted to CPs who also have substantial face perception deficits.

Unfamiliar Face Matching With Frontal and Profile Views

Research has systematically examined how laboratory participants and real-world practitioners decide whether two face photographs show the same person or not using frontal images. In contrast, research has not examined face matching using profile images. In Experiment 1, we ask whether matching unfamiliar faces is easier with frontal compared with profile views. Participants completed the original, frontal version of the Glasgow Face Matching Test, and also an adapted version where all face pairs were presented in profile. There was no difference in performance across the two tasks, suggesting that both views were similarly useful for face matching. Experiments 2 and 3 examined whether matching unfamiliar faces is improved when both frontal and profile views are provided. We compared face matching accuracy when both a frontal and a profile image of each face were presented, with accuracy using each view alone. Surprisingly, we found no benefit when both views were presented together in either experiment. Overall, these results suggest that either frontal or profile views provide substantially overlapping information regarding identity or participants are unable to utilise both sources of information when making decisions. Each of these conclusions has important implications for face matching research and real-world identification development.

Ensemble coding of face identity is not independent of the coding of individual identity

Information about a group of similar objects can be summarized into a compressed code, known as ensemble coding. Ensemble coding of simple stimuli (e.g., groups of circles) can occur in the absence of detailed exemplar coding, suggesting dissociable processes. Here, we investigate whether a dissociation would still be apparent when coding facial identity, where individual exemplar information is much more important. We examined whether ensemble coding can occur when exemplar coding is difficult, as a result of large sets or short viewing times, or whether the two types of coding are positively associated. We found a positive association, whereby both ensemble and exemplar coding were reduced for larger groups and shorter viewing times. There was no evidence for ensemble coding in the absence of exemplar coding. At longer presentation times, there was an unexpected dissociation, where exemplar coding increased yet ensemble coding decreased, suggesting that robust information about face identity might suppress ensemble coding. Thus, for face identity, we did not find the classic dissociation—of access to ensemble information in the absence of detailed exemplar information—that has been used to support claims of distinct mechanisms for ensemble and exemplar coding.

Getting to know you: The development of mechanisms underlying face learning

Nearly every study investigating the development of face recognition has focused on the ability to tell people apart using one or two tightly controlled images to represent each identity. Such research ignores the challenge of recognizing the same person despite variability in appearance. Whereas natural variation in appearance makes unfamiliar faces difficult to recognize, by 6 years of age people easily recognize multiple images of familiar faces. Two mechanisms are proposed to underlie the process by which adults become familiar with newly encountered faces. We provide the first examination of the development of these mechanisms during childhood (6-11 years). In Experiment 1, we examined children's (6- to 10-year-olds') and adults' ability to engage in ensemble coding-the ability to rapidly extract an average representation of an identity from several instances. In Experiment 2, we examined children's ability to use within-person variability in appearance to recognize novel instances of a newly encountered identity. We created a child-friendly perceptual matching task, and the number of images to which participants were exposed varied across targets. Although children were less accurate than adults overall in Experiment 2, we found no age-related improvement in either ensemble coding or the ability to benefit from exposure to within-person variability in appearance when learning a new face, suggesting that both abilities are developed by 6 years of age. We discuss the implications of these findings for understanding the nature of mechanisms underlying face learning and other developmental processes such as language and music.

Does direction of walking impact binocular rivalry between competing patterns of optic flow?

When dissimilar monocular images are viewed simultaneously by the two eyes, stable binocular vision gives way to unstable vision characterized by alternations in dominance between the two images in a phenomenon called binocular rivalry. These alternations in perception reveal the existence of inhibitory interactions between neural representations associated with conflicting visual inputs. Binocular rivalry has been studied since the days of Wheatstone, but one recent strategy is to investigate its susceptibility to influences caused by one's own motor activity. This paper focused on the activity of walking, which produces an expected, characteristic direction of optic flow dependent upon the direction of one's walking. In a set of experiments, we employed virtual reality technology to present dichoptic stimuli to observers who walked forward, backward, or were sitting. Optic flow was presented to a given eye, and was sometimes congruent with the direction of walking, sometimes incongruent, and sometimes random, except when the participant was sitting. Our results indicate that, while walking had a reliable influence on rivalry dynamics, the predominance of congruent or incongruent motion did not.

The reference frame for encoding and retention of motion depends on stimulus set size

The goal of this study was to investigate the reference frames used in perceptual encoding and storage of visual motion information. In our experiments, observers viewed multiple moving objects and reported the direction of motion of a randomly selected item. Using a vector-decomposition technique, we computed performance during smooth pursuit with respect to a spatiotopic (nonretinotopic) and to a retinotopic component and compared them with performance during fixation, which served as the baseline. For the stimulus encoding stage, which precedes memory, we found that the reference frame depends on the stimulus set size. For a single moving target, the spatiotopic reference frame had the most significant contribution with some additional contribution from the retinotopic reference frame. When the number of items increased (Set Sizes 3 to 7), the spatiotopic reference frame was able to account for the performance. Finally, when the number of items became larger than 7, the distinction between reference frames vanished. We interpret this finding as a switch to a more abstract nonmetric encoding of motion direction. We found that the retinotopic reference frame was not used in memory. Taken together with other studies, our results suggest that, whereas a retinotopic reference frame may be employed for controlling eye movements, perception and memory use primarily nonretinotopic reference frames. Furthermore, the use of nonretinotopic reference frames appears to be capacity limited. In the case of complex stimuli, the visual system may use perceptual grouping in order to simplify the complexity of stimuli or resort to a nonmetric abstract coding of motion information.

What makes a face photo a 'good likeness'?

Photographs of people are commonly said to be 'good likenesses' or 'poor likenesses', and this is a concept that we readily understand. Despite this, there has been no systematic investigation of what makes an image a good likeness, or of which cognitive processes are involved in making such a judgement. In three experiments, we investigate likeness judgements for different types of images: natural images of film stars (Experiment 1), images of film stars from specific films (Experiment 2), and iconic images and face averages (Experiment 3). In all three experiments, participants rated images for likeness and completed speeded name verification tasks. We consistently show that participants are faster to identify images which they have previously rated asa good likeness compared to a poor likeness. We also consistently show that the more familiar we are with someone, the higher likeness rating we give to all images of them. A key finding is that our perception of likeness is idiosyncratic (Experiments 1 and 2), and can be tied to our specific experience of each individual (Experiment 2). We argue that likeness judgements require a comparison between the stimulus and our own representation of the person, and that this representation differs according to our prior experience with that individual. This has theoretical implications for our understanding of how we represent familiar people, and practical implications for how we go about selecting images for identity purposes such as photo-ID.

Expression Dependence in the Perception of Facial Identity

We recognise familiar faces irrespective of their expression. This ability, crucial for social interactions, is a fundamental feature of face perception. We ask whether this constancy of facial identity may be compromised by changes in expression. This, in turn, addresses the issue of whether facial identity and expression are processed separately or interact. Using an identification task, participants learned the identities of two actors from naturalistic (so-called ambient) face images taken from movies. Training was either with neutral images or their expressive counterparts, perceived expressiveness having been determined experimentally. Expressive training responses were slower and more erroneous than neutral training responses. When tested with novel images of the actors that varied in expressiveness, neutrally trained participants gave slower and less accurate responses to images of high compared with low expressiveness. These findings clearly demonstrate that facial expressions impede the processing and learning of facial identity. Because this expression dependence is consistent with a late bifurcation model of face processing, in which changeable facial aspects and identity are coded in a common framework, it suggests that expressions are a part of facial identity representation.

Can Gaze-Contingent Mirror-Feedback from Unfamiliar Faces alter Self-Recognition?

This study focuses on learning of the self, by examining how human observers update internal representations of their own face. For this purpose, we present a novel gaze-contingent paradigm, in which an onscreen face mimics observers’ own eye-gaze behaviour (in the congruent condition), moves its eyes in different directions to that of the observers (incongruent condition), or remains static and unresponsive (neutral condition). Across three experiments, the mimicry of the onscreen face did not affect observers’ perceptual self-representations. However, this paradigm influenced observers’ reports of their own face. This effect was such that observers felt the onscreen face to be their own and that, if the onscreen gaze had moved on its own accord, observers expected their own eyes to move too. The theoretical implications of these findings are discussed.

Ensemble perception of emotions in autistic and typical children and adolescents

Ensemble perception, the ability to assess automatically the summary of large amounts of information presented in visual scenes, is available early in typical development. This ability might be compromised in autistic children, who are thought to present limitations in maintaining summary statistics representations for the recent history of sensory input. Here we examined ensemble perception of facial emotional expressions in 35 autistic children, 30 age- and ability-matched typical children and 25 typical adults. Participants received three tasks: a) an 'ensemble' emotion discrimination task; b) a baseline (single-face) emotion discrimination task; and c) a facial expression identification task. Children performed worse than adults on all three tasks. Unexpectedly, autistic and typical children were, on average, indistinguishable in their precision and accuracy on all three tasks. Computational modelling suggested that, on average, autistic and typical children used ensemble-encoding strategies to a similar extent; but ensemble perception was related to non-verbal reasoning abilities in autistic but not in typical children. Eye-movement data also showed no group differences in the way children attended to the stimuli. Our combined findings suggest that the abilities of autistic and typical children for ensemble perception of emotions are comparable on average.

How does a newly encountered face become familiar? The effect of within-person variability on adults' and children's perception of identity

Adults and children aged 6years and older easily recognize multiple images of a familiar face, but often perceive two images of an unfamiliar face as belonging to different identities. Here we examined the process by which a newly encountered face becomes familiar, defined as accurate recognition of multiple images that capture natural within-person variability in appearance. In Experiment 1 we examined whether exposure to within-person variability in appearance helps children learn a new face. Children aged 6-13years watched a 10-min video of a woman reading a story; she was filmed on a single day (low variability) or over three days, across which her appearance and filming conditions (e.g., camera, lighting) varied (high variability). After familiarization, participants sorted a set of images comprising novel images of the target identity intermixed with distractors. Compared to participants who received no familiarization, children showed evidence of learning only in the high-variability condition, in contrast to adults who showed evidence of learning in both the low- and high-variability conditions. Experiment 2 highlighted the efficiency with which adults learn a new face; their accuracy was comparable across training conditions despite variability in duration (1 vs. 10min) and type (video vs. static images) of training. Collectively, our findings show that exposure to variability leads to the formation of a robust representation of facial identity, consistent with perceptual learning in other domains (e.g., language), and that the development of face learning is protracted throughout childhood. We discuss possible underlying mechanisms.

Variation in Photos of the Same Face Drives Improvements in Identity Verification

People are poor at matching the identity of unfamiliar faces, but very good at identifying familiar faces. Theoretical accounts suggest that representations derived from exposure to variation are instrumental in driving this familiarity based improvement. In support of this, recent work shows that providing multiple photographs of an unfamiliar face improves identity verification accuracy. Here, we test whether the extent of variation is critical to this improvement, by manipulating the degree of within-identity variation that participants are exposed to in a sequential matching test. Participants were more accurate and adopted more liberal response criteria, when matching high-variability pairs to probe images, compared with either low-variability pairs or single images. Importantly, benefits of variation are not explained by independent contributions of single images, suggesting that people extrapolate information across images to produce gains in identification accuracy. These results suggest that photo-ID can be improved by incorporating broader ranges of variation in facial appearance.