Limited memory for ensemble statistics in visual change detection

How general is ensemble perception?

People can summarize features of groups of objects (e.g., the mean size of apples). Claims of dissociations or common mechanisms supporting such ensemble perception (EP) judgments have generally been made on the basis of correlations between pairs of tasks. These correlations can be inflated because they use the same stimuli, summary statistics and/or task format. Performance on EP tasks also correlates with that on object recognition (OR) tasks. Here, we seek evidence for a general EP ability that is also distinct from OR ability. Two-hundred participants completed three tasks that did not overlap in stimuli, summary statistic or task format. Participants performed a diversity comparison for arrays of nonsense blobs, a mean identity judgment with ensembles of Transformer toys, and the novel object memory task with novel objects (NOMT-Greeble). We hypothesized that EP contributes to the first two of these tasks, while OR contributes only to the last two. Performance on the two tasks suggested to tap an EP ability were correlated after controlling for the third task. Confirmatory factor analysis was used to test our predictions without the confound of measurement error. Correlations between factors assumed to share influence from EP or from OR were higher than that between the factors that we expect did not share these influences. The results provide the first clear evidence for a domain-general EP ability distinct from OR. We argue that understanding such a general ability will require a change in designs and analytical approaches in the study if individual differences in EP.

A signal-detection account of item-based and ensemble-based visual change detection: A reply to Harrison, McMaster, and Bays

Growing empirical evidence shows that ensemble information (e.g., the average feature or feature variance of a set of objects) affects visual working memory for individual items. Recently, Harrison, McMaster, and Bays (2021) used a change detection task to test whether observers explicitly rely on ensemble representations to improve their memory for individual objects. They found that sensitivity to simultaneous changes in all memorized items (which also globally changed set summary statistics) rarely exceeded a level predicted by the so-called optimal summation model within the signal-detection framework. This model implies simple integration of evidence for change from all individual items and no additional evidence coming from ensemble. Here, we argue that performance at the level of optimal summation does not rule out the use of ensemble information. First, in two experiments, we show that, even if evidence from only one item is available at test, the statistics of the whole memory set affect performance. Second, we argue that optimal summation itself can be conceptually interpreted as one of the strategies of holistic, ensemble-based decision. We also redefine the reference level for the item-based strategy as the so-called "minimum rule," which predicts performance far below the optimum. We found that that both our and Harrison et al. (2021)'s observers consistently outperformed this level. We conclude that observers can rely on ensemble information when performing visual change detection. Overall, our work clarifies and refines the use of signal-detection analysis in measuring and modeling working memory.

Ensemble coding of facial identity is robust, but may not contribute to face learning

Ensemble coding - the rapid extraction of a perceptual average - has been proposed as a potential mechanism underlying face learning. We tested this proposal across five pre-registered experiments in which four ambient images of an identity were presented in the study phase. In Experiments 1 and 2a-c, participants were asked whether a test image was in the study array; these experiments examined the robustness of ensemble coding. Experiment 1 replicated ensemble coding in an online sample; participants recognize images from the study array and the average of those images. Experiments 2a-c provide evidence that ensemble coding meets several criteria of a possible learning mechanism: It is robust to changes in head orientation (± 60o), survives a short (30s) delay, and persists when images of two identities are interleaved during the study phase. Experiment 3 examined whether ensemble coding is sufficient for face learning (i.e., facilitates recognition of novel images of a target identity). Each study array comprised four ambient images (variability + average), a single image, or an average of four images (average only). Participants were asked whether a novel test image showed the identity from a study array. Performance was best in the four-image condition, with no difference between the single-image and average-only conditions. We conclude that ensemble coding of facial identity is robust but that the perceptual average per se is not sufficient for face learning.

The role of ensemble average differs in working memory for depth and planar information

The representation of individual planar locations and features stored in working memory can be affected by the average representation. However, less is known about how the average representation affects the short-term storage of depth information. To evaluate the possible different roles of the ensemble average in working memory for planar and depth information, we used mathematical models to fit the data collected from one study on working memory for depth and 12 studies on working memory for planar information. The pattern of recalled depth was well captured by models assuming that there was a probability of reporting the average depth instead of the individual depth, compressing the recalled front-back distance of the stimulus ensemble compared to the perceived distance. However, when modeling the recalled planar information, we found that participants tended to report individual nontarget features when the target was not memorized, and the assumption of reporting average information improves the data fitting only in very few studies. These results provide evidence for our hypothesis that average depth information can be used as a substitution for individual depth information stored in working memory, but for planar visual features, the substitution of target with the average works under a constraint that the average of to-be-remembered features is readily accessible.