Efficiencies for the statistics of size discrimination

Variance aftereffect within and between sensory modalities for visual and auditory domains

We can grasp various features of the outside world using summary statistics efficiently. Among these statistics, variance is an index of information homogeneity or reliability. Previous research has shown that visual variance information in the context of spatial integration is encoded directly as a unique feature, and currently perceived variance can be distorted by that of the preceding stimuli. In this study, we focused on variance perception in temporal integration. We investigated whether any variance aftereffects occurred in visual size and auditory pitch. Furthermore, to examine the mechanism of cross-modal variance perception, we also investigated whether variance aftereffects occur between different modalities. Four experimental conditions (a combination of sensory modalities of adaptor and test: visual-to-visual, visual-to-auditory, auditory-to-auditory, and auditory-to-visual) were conducted. Participants observed a sequence of visual or auditory stimuli perturbed in size or pitch with certain variance and performed a variance classification task before and after the variance adaptation phase. We found that in visual size, within modality adaptation to small or large variance, resulted in a variance aftereffect, indicating that variance judgments are biased in the direction away from that of the adapting stimulus. In auditory pitch, within modality adaptation to small variance caused variance aftereffect. For cross-modal combinations, adaptation to small variance in visual size resulted in variance aftereffect. However, the effect was weak, and variance aftereffect did not occur in other conditions. These findings indicate that the variance information of sequentially presented stimuli is encoded independently in visual and auditory domains.

Mean orientation discrimination based on proximal stimuli

Ensemble perception refers to the ability to accurately and rapidly perceive summary statistical representations of specific features from a group of similar objects. However, the specific type of representation involved in this perception within a three-dimensional (3-D) environment remains unclear. In the context of perspective viewing with stereopsis, distal stimuli can be projected onto the retina as different forms of proximal stimuli based on their distances, despite sharing similar properties, such as object size and spatial frequency. This study aimed to investigate the effects of distal and proximal stimuli on the perception of summary statistical information related to orientation. In our experiment, we presented multiple Gabor patches in a stereoscopic environment, allowing us to measure the discrimination threshold of the mean orientation. The object size and spatial frequency were fixed for all patches regardless of depth. However, the physical angular size and absolute spatial frequency covaried with the depth. The results revealed the threshold elevation with depth expansion, especially when the patches formed two clusters at near and far distances, leading to large variations in their retinotopic representations. This finding indicates a minor contribution of similarity of the distal stimuli. Subsequent experiments demonstrated that the variability in physical angular size of the patches significantly influenced the threshold elevation in contrast to that of binocular disparity and absolute spatial frequency. These findings highlight the critical role of physical angular size variability in perceiving mean orientations within the 3-D space.

What is the basis of ensemble subset selection?

The visual system can rapidly calculate the ensemble statistics of a set of objects; for example, people can easily estimate an average size of apples on a tree. To accomplish this, it is not always useful to summarize all the visual information. If there are various types of objects, the visual system should select a relevant subset: only apples, not leaves and branches. Here, we ask what kind of visual information makes a "good" ensemble that can be selectively attended to provide an accurate summary estimate. We tested three candidate representations: basic features, preattentive object files, and full-fledged bound objects. In four experiments, we presented a target and several distractors' sets of differently colored objects. We found that conditions where a target ensemble had at least one unique color (basic feature) provided ensemble averaging performance comparable to the baseline displays without distractors. When the target subset was defined as a conjunction of two colors or color-shape partly shared with distractors (so that they could be differentiated only as preattentive object files), subset averaging was also possible but less accurate than in the baseline and feature conditions. Finally, performance was very poor when the target subset was defined by an exact feature relationship, such as in the spatial conjunction of two colors (spatially bound object). Overall, these results suggest that distinguishable features and, to a lesser degree, preattentive object files can serve as the representational basis of ensemble selection, while bound objects cannot.

Perceptual history biases in serial ensemble representation

Ensemble perception refers to the visual system's ability to efficiently represent groups of similar objects as a unified percept using their summary statistical information. Most studies focused on extraction of current trial averages, giving little attention to prior experience effects, although a few recent studies found that ensemble mean estimations contract toward previously presented stimuli, with most of these focusing on explicit perceptual averaging of simultaneously presented item ensembles. Yet, the time element is crucial in real dynamic environments, where we encounter ensemble items over time, aggregating information until reaching summary representations. Moreover, statistical information of objects and scenes is learned over time and often implicitly and then used for predictions that shape perception, promoting environmental stability. Therefore, we now focus on temporal aspects of ensemble statistics and test whether prior information, beyond the current trial, biases implicit perceptual decisions. We designed methods to separate current trial biases from those of previously seen trial ensembles. In each trial, six circles of different sizes were presented serially, followed by two test items. Participants were asked to choose which was present in the sequence. Participants unconsciously rely on ensemble statistics, choosing stimuli closer to the ensemble mean. To isolate the influence of earlier trials, the two test items were sometimes equidistant from the current trial mean. Results showed membership judgment biases toward current trial mean, when informative (largest effect). On equidistant trials, judgments were biased toward previously experienced stimulus statistics. Comparison of similar conditions with a shifted stimulus distribution ruled out a bias toward an earlier, presession, prototypical diameter. We conclude that ensemble perception, even for temporally experienced ensembles, is influenced not only by current trial mean but also by means of recently seen ensembles and that these influences are somewhat correlated on a participant-by-participant basis.

Variability of dot spread is overestimated

Previous research has demonstrated that individuals exhibit a tendency to overestimate the variability of both low-level features (e.g., color, orientation) and mid-level features (e.g., size) when items are presented dynamically in a sequential order, a finding we will refer to as the variability overestimation effect. Because previous research on this bias used sequential displays, an open question is whether the effect is due to a memory-related bias or a vision-related bias. To assess whether the bias would also be apparent with static, simultaneous displays, and to examine whether the bias generalizes to spatial properties, we tested participants' perception of the variability of a cluster of dots. Results showed a consistent overestimation bias: Participants judged the dots as being more spread than they actually were. The variability overestimation effect was observed when there were 10 or 20 dots but not when there were 50 dots. Taken together, the results of the current study contribute to the ensemble perception literature by providing evidence that simultaneously presented stimuli are also susceptible to the variability overestimation effect. The use of static displays further demonstrates that this bias is present in both dynamic and static contexts, suggesting an inherent bias existent in the human visual system. A potential theoretical account-boundary effect-is discussed as a potential underlying mechanism. Moreover, the present study has implications for common visual tasks carried out in real-world scenarios, such as a radiologist making judgments about distribution of calcification in breast cancer diagnoses.

Ensemble averaging: What can we learn from skewed feature distributions?

Many studies have shown that observers can accurately estimate the average feature of a group of objects. However, the way the visual system relies on the information from each individual item is still under debate. Some models suggest some or all items sampled and averaged arithmetically. Another strategy implies "robust averaging," when middle elements gain greater weight than outliers. One version of a robust averaging model was recently suggested by Teng et al. (2021), who studied motion direction averaging in skewed feature distributions and found systematic biases toward their modes. They interpreted these biases as evidence for robust averaging and suggested a probabilistic weighting model based on minimization of the virtual loss function. In four experiments, we replicated systematic skew-related biases in another feature domain, namely, orientation averaging. Importantly, we show that the magnitude of the bias is not determined by the locations of the mean or mode alone, but is substantially defined by the shape of the whole feature distribution. We test a model that accounts for such distribution-dependent biases and robust averaging in a biologically plausible way. The model is based on well-established mechanisms of spatial pooling and population encoding of local features by neurons with large receptive fields. Both the loss functions model and the population coding model with a winner-take-all decoding rule accurately predicted the observed patterns, suggesting that the pooled population response model can be considered a neural implementation of the computational algorithms of information sampling and robust averaging in ensemble perception.

Redundancy and Reducibility in the Formats of Spatial Representations

Mental representations are the essence of cognition. Yet to understand how the mind works, one must understand not just the content of mental representations (i.e., what information is stored) but also the format of those representations (i.e., how that information is stored). But what does it mean for representations to be formatted? How many formats are there? Is it possible that the mind represents some pieces of information in multiple formats at once? To address these questions, I discuss a "case study" of representational format: the representation of spatial location. I review work (a) across species and across development, (b) across spatial scales, and (c) across levels of analysis (e.g., high-level cognitive format vs. low-level neural format). Along the way, I discuss the possibility that the same information may be organized in multiple formats simultaneously (e.g., that locations may be represented in both Cartesian and polar coordinates). Ultimately, I argue that seemingly "redundant" formats may support the flexible spatial behavior observed in humans and that researchers should approach the study of all mental representations with this possibility in mind.

Ensemble size perception: Its neural signature and the role of global interaction over individual items

To efficiently process complex visual scenes, the visual system often summarizes statistical information across individual items and represents them as an ensemble. However, due to the lack of techniques to disentangle the representation of the ensemble from that of the individual items constituting the ensemble, whether there exists a specialized neural mechanism for ensemble processing and how ensemble perception is computed in the brain remain unknown. To address these issues, we used a frequency-tagging EEG approach to track brain responses to periodically updated ensemble sizes. Neural responses tracking the ensemble size were detected in parieto-occipital electrodes, revealing a global and specialized neural mechanism of ensemble size perception. We then used the temporal response function to isolate neural responses to the individual sizes and their interactions. Notably, while the individual sizes and their local and global interactions were encoded in the EEG signals, only the global interaction contributed directly to the ensemble size perception. Finally, distributed attention to the global stimulus pattern enhanced the neural signature of the ensemble size, mainly by modulating the neural representation of the global interaction between all individual sizes. These findings advocate a specialized, global neural mechanism of ensemble size perception and suggest that global interaction between individual items contributes to ensemble perception.

How are local orientation signals pooled?

Visual perception is capable of pooling multiple local orientation signals into a single more accurate summary orientation. However, there is still a lack of systematic inquiry into which summary statistics are implemented in that process. Here, the task was to recognize in which direction, clockwise or counter-clockwise, the mean orientation of a set of randomly distributed Gabor patches (N = 1, 2, 4, and 8) was rotated from the implicit vertical. The mean orientation discrimination accuracy did not improve with the increase of the number N of elements in proportion to the square-root-N, as could be expected if noisy internal representations were arithmetically averaged. The Proportion of Informative Elements (PIE), defined as the percentage of elements having an orientation different from the vertical, also affected the discrimination precision, violating the arithmetic averaging rules. The decrease in the orientation discrimination precision with the increase of the PIE would suggest that the orientation pooling could be more adequately described by a quadratic or higher power mean. Thus, we parameterized the averaging process for the power parameter of the generalized mean formula. The results indicate that different pooling rules in different trials may apply, for example, the arithmetic mean in some and the maximal deviation rule in others. It is concluded that pooling of orientation information is a relatively inaccurate process for which different perceptual cues and their combination rules can be used.

Modeling mean estimation tasks in within-trial and across-trial contexts

The mean estimation task, which explicitly asks observers to estimate the mean feature value of multiple stimuli, is a fundamental paradigm in research areas such as ensemble coding and cue integration. The current study uses computational models to formalize how observers summarize information in mean estimation tasks. We compare model predictions from our Fidelity-based Integration Model (FIM) and other models on their ability to simulate observed patterns in within-trial weight distribution, across-trial information integration, and set-size effects on mean estimation accuracy. Experiments show non-equal weighting within trials in both sequential and simultaneous mean estimation tasks. Observers implicitly overestimated trial means below the global mean and underestimated trial means above the global mean. Mean estimation performance declined and stabilized with increasing set sizes. FIM successfully simulated all observed patterns, while other models failed. FIM's information sampling structure provides a new way to interpret the capacity limit in visual working memory and sub-sampling strategies. As a model framework, FIM offers task-dependent modeling for various ensemble coding paradigms, facilitating research synthesis across different studies in the literature.

Perceived group size is determined by the centroids of the component elements

To accomplish the deceptively simple task of perceiving the size of objects in the visual scene, the visual system combines information about the retinal size of the object with several other cues, including perceived distance, relative size, and prior knowledge. When local component elements are perceptually grouped to form objects, the task is further complicated because a grouped object does not have a continuous contour from which retinal size can be estimated. Here, we investigate how the visual system solves this problem and makes it possible for observers to judge the size of perceptually grouped objects. We systematically vary the shape and orientation of the component elements in a two-alternative forced-choice task and find that the perceived size of the array of component objects can be almost perfectly predicted from the distance between the centroids of the component elements and the center of the array. This is true whether the global contour forms a circle or a square. When elements were positioned such that the centroids along the global contour were at different distances from the center, perceived size was based on the average distance. These results indicate that perceived size does not depend on the size of individual elements, and that smooth contours formed by the outer edges of the component elements are not used to estimate size. The current study adds to a growing literature highlighting the importance of centroids in visual perception and may have implications for how size is estimated for ensembles of different objects.

Testing temporal integration of feature probability distributions using role-reversal effects in visual search

The visual system is sensitive to statistical properties of complex scenes and can encode feature probability distributions in detail. But does the brain use these statistics to build probabilistic models of the ever-changing visual input? To investigate this, we examined how observers temporally integrate two different orientation distributions from sequentially presented visual search trials. If the encoded probabilistic information is used in a Bayesian optimal way, observers should weigh more reliable information more strongly, such as feature distributions with low variance. We therefore manipulated the variance of the two feature distributions. Participants performed sequential odd-one-out visual search for an oddly oriented line among distractors. During successive learning trials, the distractor orientations were sampled from two different Gaussian distributions on alternating trials. Then, observers performed a 'test trial' where the orientations of the target and distractors were switched, allowing us to assess observer's internal representation of distractor distributions based on changes in response times. In three experiments we observed that observer's search times on test trials depended mainly on the very last learning trial, indicating a strong recency effect. Since temporal integration has been previously observed with this method, we conclude that when the input is unreliable, the visual system relies more on the most recent stimulus. This indicates that the visual system prefers to utilize sensory history when the statistical properties of the environment are relatively stable.

The virtual loss function in the summary perception of motion and its limited adjustability

Humans can grasp the "average" feature of a visual ensemble quickly and effortlessly. However, it is largely unknown what is the exact form of the summary statistic humans perceive and it is even less known whether this form can be changed by feedback. Here we borrow the concept of loss function to characterize how the summary perception is related to the distribution of feature values in the ensemble, assuming that the summary statistic minimizes a virtual expected loss associated with its deviation from individual feature values. In two experiments, we investigated a random-dot motion estimation task to infer the virtual loss function implicit in ensemble perception and see whether it can be changed by feedback. On each trial, participants reported the average moving direction of an ensemble of moving dots whose distribution of moving directions was skewed. In Experiment 1, where no feedback was available, participants' estimates fell between the mean and the mode of the distribution and were closer to the mean. In particular, the deviation from the mean and toward the mode increased almost linearly with the mode-to-mean distance. The pattern was best modeled by an inverse Gaussian loss function, which punishes large errors less heavily than the quadratic loss function does. In Experiment 2, we tested whether this virtual loss function can be altered by feedback. Two groups of participants either received the mode or the mean as the correct answer. After extensive training up to five days, both groups' estimates moved slightly towards the mode. That is, feedback had no specific influence on participants' virtual loss function. To conclude, the virtual loss function in the summary perception of motion is close to inverse Gaussian, and it can hardly be changed by feedback.

Contributions of ensemble perception to outlier representation precision

It is known that the visual system can efficiently extract mean and variance information, facilitating the detection of outliers. However, no research to date has directly investigated whether ensemble perception mechanisms contribute to outlier representation precision. We specifically were interested in how the distinctiveness of outliers impacts their precision. Across two experiments, we compared how accurately viewers represented the orientation of spatial outliers that varied in distinctiveness and found that increased outlier distinctiveness resulted in greater precision. Based on comparisons of our data to simulations reflecting particular selective strategies, we eliminated the possibility that participants were selectively processing the outlier, at the expense of the ensemble. Thus, we argued that participants separately represented distinct outliers along with ensemble summaries of the remaining items in a display. We also found that outlier distinctiveness moderated the precision of how the remaining items were summarized. We discuss these findings in relation to computational capacity and constraints of ensemble perception mechanisms.

Perceived variability reflects the reliability of individual items

When confronted with many visual items, people can compute their variability accurately and rapidly, which facilitates efficient information processing and optimal decision making. However, how the visual system computes variability is still unclear. To investigate this, we implemented situations whereby estimates of variability based on several possible variability measures (e.g., range, standard deviation, and weighted standard deviation) could be differentiated, and then examined which best accounted for human variability perception. In three psychophysical experiments, participants watched two arrays of items with various orientations and judged which had more variable orientations. Results showed that perceived variability was most consistent with the weighted standard deviation based on the reliability of individual items. Specifically, participants gave less consideration to deviant orientations that were likely to be outliers, and greater consideration to salient orientations that were likely to be encoded precisely. This reliability-based weighted standard deviation suggests an efficient and flexible way of representing visual variability.

A method for detection of inattentional feature blindness

In ensemble displays, two principal factors determine the precision with which the mean value of some perceptual attribute, such as size and orientation, can be discriminated: inefficiency and representational noise of each element. Inefficiency is mainly caused by biased inference, or by inattentional (feature) blindness (i.e., some elements or their features are not processed). Here, we define inattentional feature blindness as an inability to perceive the value(s) of certain feature(s) of an object while the presence of the object itself may be registered. Separation of the effects of inattentional (feature) blindness and perceptual noise has escaped traditional analytic methods because of their trade-off effects on the slope of the psychometric discrimination function. Here, we propose a method that can separate the effects of inattentional feature blindness from that of the representational noise. The basic idea is to display a set of elements from which only one contains information relevant for solving the task, while all other elements are "dummies" carrying no useful information because they do not differ from the reference. If the single informative element goes unprocessed, the correct answer can only be given by a random guess. The guess rate can be modeled similarly to the lapse rate, traditionally represented by λ. As an illustration, we present evidence that the presence versus lack of inattentional feature blindness in orientation pooling depends on the feature types present in the display.

Size-distance rescaling in the ensemble representation of range: Study with binocular and monocular cues

According to numerous studies observers can rapidly and precisely evaluate mean or range of the set. Recent studies have shown that the mean size estimated based on sizes of objects rescaled to their distances (Tiurina & Utochkin, 2019). In the current study, we directly tested this rescaling mechanism on the perception of range using binocular and monocular cues. In Experiment 1, a sample set of circles with different angular sizes and in different apparent distances were stereoscopically presented. Participants had to adjust the range of the test set to match the range of the sample set. The main manipulation was the size-distance correlation for sample and test sets: in negative size-distance correlation, the apparent range had to decrease, while in positive correlation - increase. We found the highest underestimation in the condition with the negative sample correlation and positive test correlation, which could be explained only if ensemble summary statistics were estimated after the item's rescaling. In Experiment 2, we used Ponzo-like illusion and spatial positions as a depth cue. Sets were presented with positive, negative or without size-distance correlation on a grey background or the background with Ponzo-like illusion. We found that the range was underestimated in negative correlation and overestimated in positive correlation. Thus, items of ensemble could be automatically rescaled according to their distance, based on both binocular and monocular cues, and ensemble summary statistics estimation is based on perceived sizes.

Synergy between research on ensemble perception, data visualization, and statistics education: A tutorial review

In the age of big data, we are constantly inventing new data visualizations to consolidate massive amounts of numerical information into smaller and more digestible visual formats. These data visualizations use various visual features to convey quantitative information, such as spatial position in scatter plots, color saturation in heat maps, and area in dot maps. These data visualizations are typically composed of ensembles, or groups of related objects, that together convey information about a data set. Ensemble perception, or one's ability to perceive summary statistics from an ensemble, such as the mean, has been used as a foundation for understanding and explaining the effectiveness of certain data visualizations. However, research in data visualization has revealed some perceptual biases and conceptual difficulties people face when trying to utilize the information in these graphs. In this tutorial review, we will provide a broad overview of research conducted in ensemble perception, discuss how principles of ensemble encoding have been applied to the research in data visualization, and showcase the barriers graphs can pose to learning statistical concepts, using histograms as a specific example. The goal of this tutorial review is to highlight possible connections between three areas of research-ensemble perception, data visualization, and statistics education-and to encourage research in the practical applications of ensemble perception in solving real-world problems in statistics education.

Ensemble coding of crowd speed using biological motion

The accurate perception of human crowds is integral to social understanding and interaction. Previous studies have shown that observers are sensitive to several crowd characteristics such as average facial expression, gender, identity, joint attention, and heading direction. In two experiments, we examined ensemble perception of crowd speed using standard point-light walkers (PLW). Participants were asked to estimate the average speed of a crowd consisting of 12 figures moving at different speeds. In Experiment 1, trials of intact PLWs alternated with trials of scrambled PLWs with a viewing duration of 3 seconds. We found that ensemble processing of crowd speed could rely on local motion alone, although a globally intact configuration enhanced performance. In Experiment 2, observers estimated the average speed of intact-PLW crowds that were displayed at reduced viewing durations across five blocks of trials (between 2500 ms and 500 ms). Estimation of fast crowds was precise and accurate regardless of viewing duration, and we estimated that three to four walkers could still be integrated at 500 ms. For slow crowds, we found a systematic deterioration in performance as viewing time reduced, and performance at 500 ms could not be distinguished from a single-walker response strategy. Overall, our results suggest that rapid and accurate ensemble perception of crowd speed is possible, although sensitive to the precise speed range examined.

Task-driven and flexible mean judgment for heterogeneous luminance ensembles

Spatial averaging of luminances over a variegated region has been assumed in visual processes such as light adaptation, texture segmentation, and lightness scaling. Despite the importance of these processes, how mean brightness can be computed remains largely unknown. We investigated how accurately and precisely mean brightness can be compared for two briefly presented heterogeneous luminance arrays composed of different numbers of disks. The results demonstrated that mean brightness judgments can be made in a task-dependent and flexible fashion. Mean brightness judgments measured via the point of subjective equality (PSE) exhibited a consistent bias, suggesting that observers relied strongly on a subset of the disks (e.g., the highest- or lowest-luminance disks) in making their judgments. Moreover, the direction of the bias flexibly changed with the task requirements, even when the stimuli were completely the same. When asked to choose the brighter array, observers relied more on the highest-luminance disks. However, when asked to choose the darker array, observers relied more on the lowest-luminance disks. In contrast, when the task was the same, observers' judgments were almost immune to substantial changes in apparent contrast caused by changing the background luminance. Despite the bias in PSE, the mean brightness judgments were precise. The just-noticeable differences measured for multiple disks were similar to or even smaller than those for single disks, which suggested a benefit of averaging. These findings implicated flexible weighted averaging; that is, mean brightness can be judged efficiently by flexibly relying more on a few items that are relevant to the task.

Perception of means, sums, and areas

In this age of data visualization, it is important to understand our perception of the symbols that are used. For example, does the perceived size of a disc correspond most closely to its area, diameter, circumference, or some other measure? When multiple items are present, this becomes a question of ensemble perception. Here, we compare observers' performance across three different tasks: judgments of (i) the mean diameter, (ii) the total diameter, or (iii) the total area of (N = 1, 2, 3, or 7) test circles compared with a single reference circle. We draw a parallel between Anne Treisman's feature integration theory and Daniel Kahneman's cognitive systems, comparing the preattentive stage to System 1, and the focused attention stage to System 2. In accordance with Kahneman's prediction, average size (diameter) of the geometric figures can be judged with considerable accuracy, but the total diameter of the same figures cannot. Like the total length, the cumulative area covered by circles was also judged considerably less accurately than the mean diameter. Differences in efficiency between these three tasks illustrate powerful constraints upon visual processing: The visual system is well adapted for the perception of the mean size while there are no analogous mechanisms for the accurate perception of the total length or cumulative area. Thus, in visualizing data, using bubble charts proportional to area may be misleading as our visual system seems better adapted to perceive disc size by the radius rather than the area.

Quality of average representation can be enhanced by refined individual items

Ensemble perception is efficient because it summarizes redundant and complex information. However, it loses the fine details of individual items during the averaging process. Such characteristics of ensemble perception are similar to those of coarse processing. Here, we tested whether extracting an average of a set was similar to coarse processing. To manipulate coarse processing, we used the fast flicker adaptation known as suppressing coarse information processed by the magnocellular pathway. We hypothesized that if computing the average of a set relied on coarse processing, the precision of an averaging task should decrease after adaptation compared to baseline (no-adaptation). Across experiments with various features (orientation in Experiment 1, size in Experiment 2, and facial expression in Experiment 3), we found that suppressing coarse information did not disrupt the performance of the averaging tasks. Rather, adaptation increased the precision of mean representation. The precision of mean representation might have increased because fine information was relatively enhanced after adaptation. Our results suggest that the quality of ensemble representation relies on that of individual items.

Roles of saliency and set size in ensemble averaging

Ensemble statistics are often thought of as a reliable impression of numerous items despite limited capacities to consciously represent each individual. However, whether all items equally contribute to ensemble summaries (e.g., mean) and whether they might be affected by known limited-capacity processes, such as focused attention, is still debated. We addressed these questions via a recently described "amplification effect," a systematic bias of perceived mean (e.g., average size) towards the more salient "tail" of a feature distribution (e.g., larger items). In our experiments, observers adjusted the mean orientation of sets of items varying in set size. We made some of the items more salient or less salient by changing their size. While the whole orientation distribution was fixed, the more salient subset could be shifted relative to the set mean or differ in range. We measured the bias away from the set mean and the standard deviation (SD) of errors, as it is known to reflect the physical range from which ensemble information is sampled. We found that bias and SD changes followed the shifts and range changes in salient subsets, providing evidence for amplification. However, these changes were weaker than those expected from sampling only salient items, suggesting that less salient items were also sampled. Importantly, the SD decreased as a function of set size, which is only possible if the number of sampled elements increased with set size. Overall, we conclude that orientation summary statistics are sampled from an entire ensemble and modulated by the amplification effect of attention.

Ensemble perception and focused attention: Two different modes of visual processing to cope with limited capacity

The visual system has a limited capacity for dealing with complex and redundant information in a scene. Here, we propose that a distributed attention mode of processing is necessary for coping with this limit, together with a focused attention mode of processing. The distributed attention mode provides a statistical summary of a scene, whereas the focused attention mode provides relevant information for object recognition. In this paper, we claim that a distributed mode of processing is necessary because (1) averaging performance improves with increased set-sizes, (2) even unselected items are likely to contribute to averaging, and (3) the assumption of variable capacity limits in averaging over different set-sizes is not plausible. We then propose how the averaging process can access multiple items over the capacity limit of focused attention. The visual system can represent multiple items as population responses and read out relevant information using the two modes of attention. It can summarize population responses with a broad application of a Gaussian profile (i.e., distributed attention) and represent its peak as the mean. It can focus on relevant population responses with a narrow application of a Gaussian profile (i.e., focused attention) and select important information for object recognition. The two attention modes of processing provide a framework for incorporating two seemingly opposing fields of study (ensemble perception and selective attention) and a unified theory of a coping strategy with our limited capacity.

Independent and parallel visual processing of ensemble statistics: Evidence from dual tasks

The visual system can represent multiple objects in a compressed form of ensemble summary statistics (such as object numerosity, mean, and feature variance/range). Yet the relationships between the different types of visual statistics remain relatively unclear. Here, we tested whether two summaries (mean and numerosity, or mean and range) are calculated independently from each other and in parallel. Our participants performed dual tasks requiring a report about two summaries in each trial, and single tasks requiring a report about one of the summaries. We estimated trial-by-trial correlations between the precision of reports as well as correlations across observers. Both analyses showed the absence of correlations between different types of ensemble statistics, suggesting their independence. We also found no decrement (except that related to the order of report explained by memory retrieval) in performance in dual compared to single tasks, which suggests that two statistics of one ensemble can be processed in parallel.

Adaptation to mean and variance: Interrelationships between mean and variance representations in orientation perception

When there are many visual items, the visual system could represent their summary statistics (e.g., mean, variance) to process them efficiently. Although many previous studies have investigated the mean or variance representation itself, a relationship between these two ensemble representations has not been investigated much. In this study, we tested the potential interaction between mean and variance representations by using a visual adaptation method. We reasoned that if mean and variance representations interact with each other, an adaptation aftereffect to either mean or variance would influence the perception of the other. Participants watched a sequence of orientation arrays containing a specific statistical property during the adaptation period. To produce an adaptation aftereffect specific to variance or mean, one property of the adaptor arrays (variance or mean) had a fixed value while the other property was randomly varied. After the adaptation, participants were asked to discriminate the property of the test array that was randomly varied during the adaptation. We found that the adaptation aftereffect of orientation variance influenced the sensitivity of mean orientation discrimination (Experiment 1), and that the adaptation aftereffect of mean orientation influenced the bias of orientation variance discrimination (Experiment 2). These results suggest that mean and variance representations do closely interact with each other. Considering that mean and variance reflect the representative value and dispersion of multiple items respectively, the interactions between mean and variance representations may reflect their complementary roles to summarize complex visual information effectively.

Distinguishing target biases and strategic guesses in visual working memory

Previous studies have shown that when there are statistical regularities in the items stored in visual working memory, the responses are biased toward the ensemble average. This statistical-regularity-induced bias could happen in two ways: (1) a target bias, where the individual memory representations are pulled toward the ensemble average; or (2) a strategic guess, for items that are not memorized, other information in the ensemble (e.g., another item) is reported as a substitute. Here, these two mechanisms are distinguished on the basis of a three-part model (target responses + swap responses + random guesses; e.g., Bays, Catalao, & Husain, 2009, Journal of Vision, 9, 7). The strategic guess is operationalized as swap responses, whereas the target bias is reflected by a bias parameter in the target responses. This model was applied on 8 data sets (22 observers each). In this model, contributions of target biases and strategic guesses can be clearly distinguished from each other because they lead to distinctive patterns in the distribution of responses. In the present results, strategic guesses always contributed substantially to the statistical-regularity-induced biases, whereas target biases were limited to specific conditions. All in all, the Bayesian inference in visual working memory is much more limited than what is previously advocated.

The gist of Anne Treisman's revolution

Anne Treisman investigated many aspects of perception, and in particular the roles of different forms of attention. Four aspects of her work are reviewed here, including visual search, set mean perception, perception in special populations, and binocular rivalry. The importance of the breakthrough in each case is demonstrated. Search is easy or slow depending on whether it depends on the application of global or focused attention. Mean perception depends on global attention and affords simultaneous representation of the means of at least two sets of elements, and then of comparing them. Deficits exhibited in Balint's or unilateral neglect patients identify basic sensory system mechanisms. And, the ability to integrate binocular information for stereopsis despite simultaneous binocular rivalry for color, demonstrates the division of labor underlying visual system computations. All these studies are related to an appreciation of the difference between perceiving the gist of a scene, its elements or objects, versus perceiving the details of the scene and its components. This relationship between Anne Treisman's revolutionary discoveries and the concept of gist perception is the core of the current review.

Distributed attention model of perceptual averaging

The visual system efficiently processes complex and redundant information in a scene despite its limited capacity. One strategy for coping with the complexity and redundancy of a scene is to summarize it by using average information. However, despite its importance, the mechanism of averaging is not well understood. Here, a distributed attention model of averaging is proposed. Human percept for an object can be disturbed by various sources of internal noise, which can occur either before (early noise) or after (late noise) forming an ensemble perception. The model assumes these noises and reflects noise cancellation by averaging multiple items. The model predicts increased precision for more items with decelerated increments for large set-sizes resulting from late noise. Importantly, the model incorporates mechanisms of attention, which modulate each item's contribution to the averaging process. The attention in the model also results in saturation of performance increments for small set-sizes because the amount of attention allocated to each item is greater for small set-sizes than for large set-sizes. To evaluate the proposed model, a psychophysical experiment was conducted in which observers' ability to discriminate average sizes of two displays was measured. The observers' averaging performance increased at a decreasing rate with small set-sizes and it approached an asymptote for large set-sizes. The model accurately predicted the observed pattern of data. It provides a theoretical framework for interpreting behavioral data and leads to an understanding of the characteristics of ensemble perception.

The Additive-Area Heuristic: An Efficient but Illusory Means of Visual Area Approximation

How do we determine how much of something is present? A large body of research has investigated the mechanisms and consequences of number estimation, yet surprisingly little work has investigated area estimation. Indeed, area is often treated as a pesky confound in the study of number. Here, we describe the additive-area heuristic, a means of rapidly estimating visual area that results in substantial distortions of perceived area in many contexts, visible even in simple demonstrations. We show that when we controlled for additive area, observers were unable to discriminate on the basis of true area, per se, and that these results could not be explained by other spatial dimensions. These findings reflect a powerful perceptual illusion in their own right but also have implications for other work, namely, that which relies on area controls to support claims about number estimation. We discuss several areas of research potentially affected by these findings.

Coding of low-level position and orientation information in human naturalistic vision

Orientation and position of small image segments are considered to be two fundamental low-level attributes in early visual processing, yet their encoding in complex natural stimuli is underexplored. By measuring the just-noticeable differences in noise perturbation, we investigated how orientation and position information of a large number of local elements (Gabors) were encoded separately or jointly. Importantly, the Gabors composed various classes of naturalistic stimuli that were equated by all low-level attributes and differed only in their higher-order configural complexity and familiarity. Although unable to consciously tell apart the type of perturbation, observers detected orientation and position noise significantly differently. Furthermore, when the Gabors were perturbed by both types of noise simultaneously, performance adhered to a reliability-based optimal probabilistic combination of individual attribute noises. Our results suggest that orientation and position are independently coded and probabilistically combined for naturalistic stimuli at the earliest stage of visual processing.

Effects of the Simultaneous Presentation of Corresponding Auditory and Visual Stimuli on Size Variance Perception

To overcome limitations in perceptual bandwidth, humans condense various features of the environment into summary statistics. Variance constitutes indices that represent diversity within categories and also the reliability of the information regarding that diversity. Studies have shown that humans can efficiently perceive variance for visual stimuli; however, to enhance perception of environments, information about the external world can be obtained from multisensory modalities and integrated. Consequently, this study investigates, through two experiments, whether the precision of variance perception improves when visual information (size) and corresponding auditory information (pitch) are integrated. In Experiment 1, we measured the correspondence between visual size and auditory pitch for each participant by using adjustment measurements. The results showed a linear relationship between size and pitch-that is, the higher the pitch, the smaller the corresponding circle. In Experiment 2, sequences of visual stimuli were presented both with and without linked auditory tones, and the precision of perceived variance in size was measured. We consequently found that synchronized presentation of audio and visual stimuli that have the same variance improves the precision of perceived variance in size when compared with visual-only presentation. This suggests that audiovisual information may be automatically integrated in variance perception.

Ensemble statistics accessed through proxies: Range heuristic and dependence on low-level properties in variability discrimination

People can quickly and accurately compute not only the mean size of a set of items but also the size variability of the items. However, it remains unknown how these statistics are estimated. Here we show that neither parallel access to all items nor random subsampling of just a few items is sufficient to explain participants' estimations of size variability. In three experiments, we had participants compare two arrays of circles with different variability in their sizes. In the first two experiments, we manipulated the congruency of the range and variance of the arrays. The arrays with congruent range and variability information were judged more accurately, indicating the use of range as a proxy for variability. Experiments 2B and 3 showed that people also are not invariant to low- or mid-level visual information in the arrays, as comparing arrays with different low-level characteristics (filled vs. outlined circles) led to systematic biases. Together, these experiments indicate that range and low- or mid-level properties are both utilized as proxies for variability discrimination, and people are flexible in adopting these strategies. These strategies are at odds with the claim of parallel extraction of ensemble statistics per se and random subsampling strategies previously proposed in the literature.

Ensemble perception of facial attractiveness

Ensemble perception, the extraction of a statistical summary of multiple instances of a feature, enables efficient processing of information. Here we investigated whether ensemble representations can be formed for facial attractiveness, a socially important complex feature. After verifying that our face stimuli produced by geometric morphing represented a valid continuum of attractiveness (Experiment 1), we asked participants to compare the average attractiveness of four faces with a single probe face. Whether the four faces were homogeneous or heterogeneous resulted in highly similar performance levels, suggesting the visual system could extract an ensemble representation of the attractiveness of a heterogeneous group of faces. Statistical simulations with human-level bias and noise indicated participants did not rely on subsampling one random face or the most/least attractive face from the array (Experiment 2). Ensemble perception of facial attractiveness was not affected by variance in the stimulus array (Experiment 3), did not depend on memory of individual faces in the array (Experiment 4), and could be extended to larger arrays with faces asymmetrically distributed around the set mean (Experiment 5). Our findings give further evidence to the prevalence of perception of statistical regularities in vision.

Calculation Efficiencies for Mean Numerosity

Relative numerosity is traditionally studied using texture pairs. Observers must decide which member of each pair has the greater total number of texture elements. In the present experiment, textures were segregated into nonoverlapping "sectors" containing between zero and four elements, and our observers were asked to select the texture containing the greater average number of texture elements (per sector). If observers were more sensitive to total numerosity than average numerosity, their performance (quantified by the just-noticeable Weber fraction) should have been better when the two textures occupied the same number of sectors than when they occupied unequal numbers of sectors. However, we recorded Weber fractions between 11% and 13% for all observers in all conditions. This performance was comparable with an otherwise-ideal observer whose decisions were based on between three and five sectors in each texture. We conjecture that traditional numerosity discriminations are based on similarly small numbers of element clusters.

The Effect of Size Statistics of the Background Texture on Perceived Target Size

We investigated the effect of the size distribution statistics of background elements on the perceived size of a target. We manipulated the first, second, and third order statistics (i.e., mean, variance, and skewness) of the background element size distribution. We used a two-interval forced-choice paradigm to measure perceived target size at different background size distributions. In each trial, a standard disk, or target, was presented in one interval with a textured background and a comparison disk, on a blank background, in the other. The task for the observers was to determine which interval contained a larger disk. We measured the point of subjective equality for the perceived target size with a staircase procedure. The perceived target size decreased with the increment of mean background disk size. The variance and skewness of the background element size did not affect the perceived target size. Our results showed that only the first order statistics of the background modulated the perceived target size, not the higher order statistics. A computational model, in which the visual system extracts size information by averaging the responses of different size channels, whose response is modulated by the size of the background elements, can account for the results.

Visual similarity modulates visual size contrast

Perception is relational: object properties are perceived in comparison to the spatiotemporal context rather than absolutely. This principle predicts well known contrast effects: For instance, the same sphere will feel smaller after feeling a larger sphere and larger after feeling a smaller sphere (the Uznadze effect). In a series of experiments, we used a visual version of the Uznadze effect to test whether such contrast effects can be modulated by organizational factors, such as the similarity between the contrasting inducer stimulus and the contrasted induced stimulus. We report that this is indeed the case: size contrast is attenuated for inducer-inducing pairs having different 3D shapes, orientations, and even - surprisingly - color and lightness, in comparison to equivalent conditions where these features are the same. These findings complement related work in revealing basic mechanisms for fine-tuning local interactions in space-time in accord to the global stimulus context.

The development of perceptual averaging: learning what to do, not just how to do it

The mature visual system condenses complex scenes into simple summary statistics (e.g., average size, location, orientation, etc.). However, children, often perform poorly on perceptual averaging tasks. Children's difficulties are typically thought to represent the suboptimal implementation of an adult-like strategy. This paper examines another possibility: that children actually make decisions in a qualitatively different way to adults (optimal implementation of a non-ideal strategy). Ninety children (6-7, 8-9, 10-11 years) and 30 adults were asked to locate the middle of randomly generated dot-clouds. Nine plausible decision strategies were formulated, and each was fitted to observers' trial-by-trial response data (Reverse Correlation). When the number of visual elements was low (N < 6), children used a qualitatively different decision strategy from adults: appearing to "join up the dots" and locate the gravitational center of the enclosing shape. Given denser displays, both children and adults used an ideal strategy of arithmetically averaging individual points. Accounting for this difference in decision strategy explained 29% of children's lower precision. These findings suggest that children are not simply suboptimal at performing adult-like computations, but may at times use sensible, but qualitatively different strategies to make perceptual judgments. Learning which strategy is best in which circumstance might be an important driving factor of perceptual development.

Capacity limitations to extract the mean emotion from multiple facial expressions depend on emotion variance

We examined the processing capacity and the role of emotion variance in ensemble representation for multiple facial expressions shown concurrently. A standard set size manipulation was used, whereby the sets consisted of 4, 8, or 16 morphed faces each uniquely varying along a happy-angry continuum (Experiment 1) or a neutral-happy/angry continuum (Experiments 2 & 3). Across the three experiments, we reduced the amount of emotion variance in the sets to explore the boundaries of this process. Participants judged the perceived average emotion from each set on a continuous scale. We computed and compared objective and subjective difference scores, using the morph units and post-experiment ratings, respectively. Results of the subjective scores were more consistent than the objective ones across the first two experiments where the variance was relatively large, and revealed each time that increasing set size led to a poorer averaging ability, suggesting capacity limitations in establishing ensemble representations for multiple facial expressions. However, when the emotion variance in the sets was reduced in Experiment 3, both subjective and objective scores remained unaffected by set size, suggesting that the emotion averaging process was unlimited in these conditions. Collectively, these results suggest that extracting mean emotion from a set composed of multiple faces depends on both structural (attentional) and stimulus-related effects.

Is There a Common Summary Statistical Process for Representing the Mean and Variance? A Study Using Illustrations of Familiar Items

A number of studies revealed that our visual system can extract different types of summary statistics, such as the mean and variance, from sets of items. Although the extraction of such summary statistics has been studied well in isolation, the relationship between these statistics remains unclear. In this study, we explored this issue using an individual differences approach. Observers viewed illustrations of strawberries and lollypops varying in size or orientation and performed four tasks in a within-subject design, namely mean and variance discrimination tasks with size and orientation domains. We found that the performances in the mean and variance discrimination tasks were not correlated with each other and demonstrated that extractions of the mean and variance are mediated by different representation mechanisms. In addition, we tested the relationship between performances in size and orientation domains for each summary statistic (i.e. mean and variance) and examined whether each summary statistic has distinct processes across perceptual domains. The results illustrated that statistical summary representations of size and orientation may share a common mechanism for representing the mean and possibly for representing variance. Introspections for each observer performing the tasks were also examined and discussed.

Size discrimination in barn owls as compared to humans

We tested how well barn owls can discriminate objects of different sizes. This ability may be important for the owls when catching prey. We performed a quantitative experiment in the laboratory and trained owls in a task in which the owls had to discriminate whether two rhombi presented simultaneously on a computer monitor were of the same or of different sizes. We obtained full data sets with two experienced owls and one data point with a third owl. For objects being sufficiently larger than the spatial resolution of the barn owl, the angular threshold was related to object size, implying that the discrimination followed Weber's law. The range of Weber fractions we determined was between 0.026 and 0.09. For object sizes close to the spatial resolution, performance degraded. We conducted similar experiments with human subjects. Human thresholds showed the same dependence on object size, albeit down to smaller object sizes. Human performance resulted in a range of Weber fractions extending from 0.025 to 0.036. The differences between owls and humans could be explained by the much higher spatial acuity of humans compared with owls.

A Perceptual-Like Population-Coding Mechanism of Approximate Numerical Averaging

Humans possess a remarkable ability to rapidly form coarse estimations of numerical averages. This ability is important for making decisions that are based on streams of numerical or value-based information, as well as for preference formation. Nonetheless, the mechanism underlying rapid approximate numerical averaging remains unknown, and several competing mechanism may account for it. Here, we tested the hypothesis that approximate numerical averaging relies on perceptual-like processes, instantiated by population coding. Participants were presented with rapid sequences of numerical values (four items per second) and were asked to convey the sequence average. We manipulated the sequences' length, variance, and mean magnitude and found that similar to perceptual averaging, the precision of the estimations improves with the length and deteriorates with (higher) variance or (higher) magnitude. To account for the results, we developed a biologically plausible population-coding model and showed that it is mathematically equivalent to a population vector. Using both quantitative and qualitative model comparison methods, we compared the population-coding model to several competing models, such as a step-by-step running average (based on leaky integration) and a midrange model. We found that the data support the population-coding model. We conclude that humans' ability to rapidly form estimations of numerical averages has many properties of the perceptual (intuitive) system rather than the arithmetic, linguistic-based (analytic) system and that population coding is likely to be its underlying mechanism.

Ensemble Perception

To understand visual consciousness, we must understand how the brain represents ensembles of objects at many levels of perceptual analysis. Ensemble perception refers to the visual system's ability to extract summary statistical information from groups of similar objects-often in a brief glance. It defines foundational limits on cognition, memory, and behavior. In this review, we provide an operational definition of ensemble perception and demonstrate that ensemble perception spans across multiple levels of visual analysis, incorporating both low-level visual features and high-level social information. Further, we investigate the functional usefulness of ensemble perception and its efficiency, and we consider possible physiological and cognitive mechanisms that underlie an individual's ability to make accurate and rapid assessments of crowds of objects.