Extracting the mean size across the visual field in patients with mild, chronic unilateral neglect

Ensemble coding of multiple facial expressions is not affected by attentional load

Human observers can extract the mean emotion from multiple faces rapidly and precisely. However, whether attention is required in the ensemble coding of facial expressions remains debated. In this study, we examined the effect of attentional load on mean emotion processing with the dual-task paradigm. Individual emotion processing was also investigated as the control task. In the experiment, the letter string and a set of four happy or angry faces of various emotional intensities were shown. Participants had to complete the string task first, judging either the string color (low attention load) or the presence of the target letter (high attention load). Then a cue appeared indicating whether the secondary task was to evaluate the mean emotion of the faces or the emotion of the cued single face, and participants made their judgments on the visual analog scale. The results showed that compared with the color task, the letter task had a longer response time and lower accuracy, which verified the valid manipulation of the attention loads. More importantly, there was no significant difference in averaging performance between the low and high attention loads. By contrast, the individual face processing was impaired under the high attention load relative to the low attentional load. In addition, the advantage of extracting mean emotion over individual emotion was larger under the high attentional load. These results support the power of averaging and provide new evidence that a rather small amount of attention is needed in the ensemble coding of multiple facial expressions.

Holistic ensemble perception

In a glance, observers can evaluate gist characteristics from crowds of faces, such as the average emotional tenor or the average family resemblance. Prior research suggests that high-level ensemble percepts rely on holistic and viewpoint-invariant information. However, it is also possible that feature-based analysis was sufficient to yield successful ensemble percepts in many situations. To confirm that ensemble percepts can be extracted holistically, we asked observers to report the average emotional valence of Mooney face crowds. Mooney faces are two-tone, shadow-defined images that cannot be recognized in a part-based manner. To recognize features in a Mooney face, one must first recognize the image as a face by processing it holistically. Across experiments, we demonstrated that observers successfully extracted the average emotional valence from crowds that were spatially distributed or viewed in a rapid temporal sequence. In a subsequent set of experiments, we maximized holistic processing by including only those Mooney faces that were difficult to recognize when inverted. Under these conditions, participants remained highly sensitive to the average emotional valence of Mooney face crowds. Taken together, these experiments provide evidence that ensemble perception can operate selectively on holistic representations of human faces, even when feature-based information is not readily available.

Statistical stability and set size exert distinct influences on visual search

Despite continuous retinal chaos, we perceive the world as stable and complete. This illusion is sustained over consecutive glances by reliance on statistical redundancies inherent in the visual environment. For instance, repeating the average size of a collection of differently sized items speeds visual search for a randomly located target regardless of trial-to-trial changes in local element size (Corbett & Melcher, 2014b). Here, we manipulate set size to investigate the potential role attention may play in these facilitative effects of statistical stability on visual search. Observers discriminated the left or right tilt of a Gabor target defined by a unique conjunction of orientation and spatial frequency in displays of Gabors with a stable or unstable mean size over successive trials. When set size was manipulated over sequences of successive trials, but held constant within a given sequence in Experiment 1, we observed distinct effects of statistical stability and attention, such that participants made faster correct responses as a function of stability and slower correct responses as a function of increasing set size. Replicating these main effects in Experiment 2, when set size was always unstable, provided converging evidence for discrete influences of statistical stability and attentional contributions to visual search. Overall, results support the proposal that our stable impressions of the surrounding environment and our abilities to attend salient events within that environment are distinctively governed by inherent statistical context and attentional processing demands.

Quality of average representation can be enhanced by refined individual items

Ensemble perception is efficient because it summarizes redundant and complex information. However, it loses the fine details of individual items during the averaging process. Such characteristics of ensemble perception are similar to those of coarse processing. Here, we tested whether extracting an average of a set was similar to coarse processing. To manipulate coarse processing, we used the fast flicker adaptation known as suppressing coarse information processed by the magnocellular pathway. We hypothesized that if computing the average of a set relied on coarse processing, the precision of an averaging task should decrease after adaptation compared to baseline (no-adaptation). Across experiments with various features (orientation in Experiment 1, size in Experiment 2, and facial expression in Experiment 3), we found that suppressing coarse information did not disrupt the performance of the averaging tasks. Rather, adaptation increased the precision of mean representation. The precision of mean representation might have increased because fine information was relatively enhanced after adaptation. Our results suggest that the quality of ensemble representation relies on that of individual items.

Self-Construal Priming Modulates Ensemble Perception of Multiple-Face Identities

This study explored the modulatory role of independent/interdependent self-construal on ensemble perception. Two experiments were conducted to study the effect of self-construal on ensemble coding of multiple-face identities (Experiment 1) and dot size (Experiment 2) separately. Before the implicit ensemble perception task, participants in both experiments were either primed with independent or interdependent self-construal via a well-validated pronoun circle task, in which they were exposed to either singular ("I," "me," and "my") or plural ("We," "us," and "our") pronouns in essays. The results showed that interdependent self-construal (vs. independent self-construal) featured as global processing and emphasizing interconnectedness with others enhanced the ensemble coding of high-level features (e.g., identity in Experiment 1) but not of low-level features (e.g., size in Experiment 2). To the best of our knowledge, this study was the first to investigate the role of self-construal on ensemble representations. In sum, the results of the current study supported the domain-specific mechanism of ensemble perception on one hand, and extended the effect of self-construal on single face recognition to multiple face recognition on the other hand.

Ensemble coding remains accurate under object and spatial visual working memory load

A number of studies have provided evidence that the visual system statistically summarizes large amounts of information that would exceed the limitations of attention and working memory (ensemble coding). However the necessity of working memory resources for ensemble coding has not yet been tested directly. In the current study, we used a dual task design to test the effect of object and spatial visual working memory load on size averaging accuracy. In Experiment 1, we tested participants' accuracy in comparing the mean size of two sets under various levels of object visual working memory load. Although the accuracy of average size judgments depended on the difference in mean size between the two sets, we found no effect of working memory load. In Experiment 2, we tested the same average size judgment while participants were under spatial visual working memory load, again finding no effect of load on averaging accuracy. Overall our results reveal that ensemble coding can proceed unimpeded and highly accurately under both object and spatial visual working memory load, providing further evidence that ensemble coding reflects a basic perceptual process distinct from that of individual object processing.

Ensemble Perception

To understand visual consciousness, we must understand how the brain represents ensembles of objects at many levels of perceptual analysis. Ensemble perception refers to the visual system's ability to extract summary statistical information from groups of similar objects-often in a brief glance. It defines foundational limits on cognition, memory, and behavior. In this review, we provide an operational definition of ensemble perception and demonstrate that ensemble perception spans across multiple levels of visual analysis, incorporating both low-level visual features and high-level social information. Further, we investigate the functional usefulness of ensemble perception and its efficiency, and we consider possible physiological and cognitive mechanisms that underlie an individual's ability to make accurate and rapid assessments of crowds of objects.

Perceptual Averaging in Individuals with Autism Spectrum Disorder

There is mounting evidence that observers rely on statistical summaries of visual information to maintain stable and coherent perception. Sensitivity to the mean (or other prototypical value) of a visual feature (e.g., mean size) appears to be a pervasive process in human visual perception. Previous studies in individuals diagnosed with Autism Spectrum Disorder (ASD) have uncovered characteristic patterns of visual processing that suggest they may rely more on enhanced local representations of individual objects instead of computing such perceptual averages. To further explore the fundamental nature of abstract statistical representation in visual perception, we investigated perceptual averaging of mean size in a group of 12 high-functioning individuals diagnosed with ASD using simplified versions of two identification and adaptation tasks that elicited characteristic perceptual averaging effects in a control group of neurotypical participants. In Experiment 1, participants performed with above chance accuracy in recalling the mean size of a set of circles (mean task) despite poor accuracy in recalling individual circle sizes (member task). In Experiment 2, their judgments of single circle size were biased by mean size adaptation. Overall, these results suggest that individuals with ASD perceptually average information about sets of objects in the surrounding environment. Our results underscore the fundamental nature of perceptual averaging in vision, and further our understanding of how autistic individuals make sense of the external environment.

Computing an Average When Part of the Population Is Not Perceived

The syndrome of unilateral spatial neglect (USN) after right-hemisphere damage is characterized by failure of salient left-sided stimuli to activate an orienting response, attract attention, and gain access to conscious awareness. The explicit failure processing left-sided visual information is not uniform, however, and patients seem to be more successful performing certain visual tasks than others. The source of this difference is still not clear. We focus on processing of visual scene statistical properties, asking whether, in computing the average size of an array of objects, USN patients give appropriate weight to objects on the left; disregard left-side objects entirely; or assign them an intermediate, lower weight, in accord with their tendency to neglect these objects. The interest in testing this question stems from a series of studies in healthy individuals that led Chong and Treisman [Chong, S. C., & Treisman, A. Statistical processing: Computing the average size in perceptual groups. Vision Research, 45, 891–900, 2005a; Chong, S. C., & Treisman, A. Attentional spread in the statistical processing of visual displays. Perception & Psychophysics, 67, 1–13, 2005b] to propose that processing of statistical properties (like the average size of visual scene elements) is carried out in parallel, with no need for serial allocation of focal attention to the different scene elements. Our results corroborate this suggestion, showing that objects in the left (“neglected”) hemispace contribute to average size computation, despite a marked imbalance in spatial distribution of attention, which leads to a reduced weight of left-side elements in the averaging computation. This finding sheds light on the nature of the impairment in USN and on basic mechanisms underlying statistical processing in vision. We confirm that statistical processing depends mainly on spread-attention mechanisms, which are largely spared in USN.

Ensemble crowd perception: a viewpoint-invariant mechanism to represent average crowd identity

Individuals can rapidly and precisely judge the average of a set of similar items, including both low-level (Ariely, 2001) and high-level objects (Haberman & Whitney, 2007). However, to date, it is unclear whether ensemble perception is based on viewpoint-invariant object representations. Here, we tested this question by presenting participants with crowds of sequentially presented faces. The number of faces in each crowd and the viewpoint of each face varied from trial to trial. This design required participants to integrate information from multiple viewpoints into one ensemble percept. Participants reported the mean identity of crowds (e.g., family resemblance) using an adjustable, forward-oriented test face. Our results showed that participants accurately perceived the mean crowd identity even when required to incorporate information across multiple face orientations. Control experiments showed that the precision of ensemble coding was not solely dependent on the length of time participants viewed the crowd. Moreover, control analyses demonstrated that observers did not simply sample a subset of faces in the crowd but rather integrated many faces into their estimates of average crowd identity. These results demonstrate that ensemble perception can operate at the highest levels of object recognition after 3-D viewpoint-invariant faces are represented.

Characterizing ensemble statistics: mean size is represented across multiple frames of reference

The visual system represents the overall statistical, not individual, properties of sets. Here we tested the spatial nature of ensemble statistics. We used a mean-size adaptation paradigm (Corbett et al. in Visual Cognition, 20, 211-231, 2012) to examine whether average size is encoded in multiple reference frames. We adapted observers to patches of small- and large-sized dots in opposite regions of the display (left/right or top/bottom) and then tested their perceptions of the sizes of single test dots presented in regions that corresponded to retinotopic, spatiotopic, and hemispheric coordinates within the adapting displays. We observed retinotopic, spatiotopic, and hemispheric adaptation aftereffects, such that participants perceived a test dot as being larger when it was presented in the area adapted to the patch of small dots than when it was presented in the area adapted to large dots. This aftereffect also transferred between eyes. Our results demonstrate that mean size is represented across multiple spatial frames of reference, supporting the proposal that ensemble statistics play a fundamental role in maintaining perceptual stability.

Global statistical regularities modulate the speed of visual search in patients with focal attentional deficits

There is growing evidence that the statistical properties of ensembles of similar objects are processed in a qualitatively different manner than the characteristics of individual items. It has recently been proposed that these types of perceptual statistical representations are part of a strategy to complement focused attention in order to circumvent the visual system’s limited capacity to represent more than a few individual objects in detail. Previous studies have demonstrated that patients with attentional deficits are nonetheless sensitive to these sorts of statistical representations. Here, we examined how such global representations may function to aid patients in overcoming focal attentional limitations by manipulating the statistical regularity of a visual scene while patients performed a search task. Three patients previously diagnosed with visual neglect searched for a target Gabor tilted to the left or right of vertical in displays of horizontal distractor Gabors. Although the local sizes of the distractors changed on every trial, the mean size remained stable for several trials. Patients made faster correct responses to targets in neglected regions of the visual field when global statistics remained constant over several trials, similar to age-matched controls. Given neglect patients’ attentional deficits, these results suggest that stable perceptual representations of global statistics can establish a context to speed search without the need to represent individual elements in detail.