Crowding results from optimal integration of visual targets with contextual information

Color crowding considered as adaptive spatial integration

Crowding is the inability to recognize an object in clutter, classically considered a fundamental low-level bottleneck to object recognition. Recently, however, it has been suggested that crowding, like predictive phenomena such as serial dependence, may result from optimizing strategies that exploit redundancies in natural scenes. This notion leads to several testable predictions, such as crowding being greater for nonsalient targets and, counterintuitively, that flanker interference should be associated with higher precision in judgements, leading to a lower overall error rate. Here we measured color discrimination for targets flanked by stimuli of variable color. The results verified both predictions, showing that although crowding can affect object recognition, it may be better understood not as a processing bottleneck, but rather as a consequence of mechanisms evolved to efficiently exploit the spatial redundancies of the natural world. Analyses of reaction times of judgments shows that the integration occurs at sensory rather than decisional levels.

Oculomotor Contributions to Foveal Crowding

Crowding, the phenomenon of impaired visual discrimination due to nearby objects, has been extensively studied and linked to cortical mechanisms. Traditionally, crowding has been studied extrafoveally; its underlying mechanisms in the central fovea, where acuity is highest, remain debated. While low-level oculomotor factors are not thought to play a role in crowding, this study shows that they are key factors in defining foveal crowding. Here, we investigate the influence of fixational behavior on foveal crowding and provide a comprehensive assessment of the magnitude and extent of this phenomenon (N = 13 human participants, four males). Leveraging on a unique blend of tools for high-precision eyetracking and retinal stabilization, we show that removing the retinal motion introduced by oculomotor behavior with retinal stabilization, diminishes the negative effects of crowding. Ultimately, these results indicate that ocular drift contributes to foveal crowding resulting in the same pooling region being stimulated both by the target and nearby objects over the course of time, not just in space. The temporal aspect of this phenomenon is peculiar to crowding at this scale and indicates that the mechanisms contributing to foveal and extrafoveal crowding differ.

Seeing in crowds: Averaging first, then max

Crowding, a fundamental limit in object recognition, is believed to result from excessive integration of nearby items in peripheral vision. To understand its pooling mechanisms, we measured subjects' internal response distributions in an orientation crowding task. Contrary to the prediction of an averaging model, we observed a pattern suggesting that the perceptual judgement is made based on choosing the largest response across the noise-perturbed items. A model featuring first-stage averaging and second-stage signed-max operation predicts the diverse errors made by human observers under various signal strength levels. These findings suggest that different rules operate to resolve the bottleneck at early and high-level stages of visual processing, implementing a combination of linear and nonlinear pooling strategies.

Super recognizers: Increased sensitivity or reduced biases? Insights from serial dependence

Super recognizers (SRs) are people that exhibit a naturally occurring superiority for processing facial identity. Despite the increase of SR research, the mechanisms underlying their exceptional abilities remain unclear. Here, we investigated whether the enhanced facial identity processing of SRs could be attributed to the lack of sequential effects, such as serial dependence. In serial dependence, perception of stimulus features is assimilated toward stimuli presented in previous trials. This constant error in visual perception has been proposed as a mechanism that promotes perceptual stability in everyday life. We hypothesized that an absence of this constant source of error in SRs could account for their superior processing-potentially in a domain-general fashion. We tested SRs (n = 17) identified via a recently proposed diagnostic framework (Ramon, 2021) and age-matched controls (n = 20) with two experiments probing serial dependence in the face and shape domains. In each experiment, observers were presented with randomly morphed face identities or shapes and were asked to adjust a face's identity or a shape to match the stimulus they saw. We found serial dependence in controls and SRs alike, with no difference in its magnitude across groups. Interestingly, we found that serial dependence impacted the performance of SRs more than that of controls. Taken together, our results show that enhanced face identity processing skills in SRs cannot be attributed to the lack of serial dependence. Rather, serial dependence, a beneficial nested error in our visual system, may in fact further stabilize the perception of SRs and thus enhance their visual processing proficiency.

Probing the Bottleneck of Awareness Formed by Foveal Crowding: A Neurophysiological Study

Crowding occurs when an easily identified isolated stimulus is surrounded by stimuli with similar properties, making it very difficult to identify. Crowding is suggested as a mechanism that creates a bottleneck in object recognition and awareness. Recently, we showed that brief presentation times at the fovea resulted in a significant crowding effect on target identification, impaired the target's color awareness, and resulted in a slower reaction time. However, when tagging the target with a red letter, the crowding effect is abolished. Crowding is widely considered a grouping; hence, it is pre-attentive. An event-related potential (ERP) study that investigated the spatial-temporal properties of crowding suggested the involvement of higher-level visual processing. Here, we investigated whether ERP's components may be affected by crowding and tagging, and whether the temporal advantage of ERP can be utilized to gain further information about the crowding mechanism. The participants reported target identification using our standard foveal crowing paradigm. It is assumed that crowding occurs due to a suppressive effect; thus, it can be probed by changes in perceptual (N1, ~160 ms) and attentive (P3 ~300-400 ms) components. We found a suppression effect (less negative ERP magnitude) in N1 under foveal crowding, which was recovered under tagging conditions. ERP's amplitude components (N1 and P3) and the behavioral proportion correct are highly correlated. These findings suggest that crowding is an early grouping mechanism that may be combined with later processing involving the segmentation mechanism.

Serial Dependence in Perception

Much evidence has shown that perception is biased towards previously presented similar stimuli, an effect recently termed serial dependence. Serial dependence affects nearly every aspect of perception, often causing gross perceptual distortions, especially for weak and ambiguous stimuli. Despite unwanted side-effects, empirical evidence and Bayesian modeling show that serial dependence acts to improve efficiency and is generally beneficial to the system. Consistent with models of predictive coding, the Bayesian priors of serial dependence are generated at high levels of cortical analysis, incorporating much perceptual experience, but feed back to lower sensory areas. These feedback loops may drive oscillations in the alpha range, linked strongly with serial dependence. The discovery of top-down predictive perceptual processes is not new, but the new, more quantitative approach characterizing serial dependence promises to lead to a deeper understanding of predictive perceptual processes and their underlying neural mechanisms.

Serial dependence improves performance and biases confidence-based decisions

Perception depends on both the current sensory input and on the preceding stimuli history, a mechanism referred to as serial dependence (SD). One interesting, and somewhat controversial, question is whether serial dependence originates at the perceptual stage, which should lead to a sensory improvement, or at a subsequent decisional stage, causing solely a bias. Here, we studied the effects of SD in a novel manner by leveraging on the human capacity to spontaneously assess the quality of sensory information. Two noisy-oriented Gabor stimuli were simultaneously presented along with two bars of the same orientation as the Gabor stimuli. Participants were asked to choose which Gabor stimulus to judge and then make a forced-choice judgment of its orientation by selecting the appropriate response bar. On all trials, one of the Gabor stimuli had the same orientation as the Gabor in the same position on the previous trial. We explored whether continuity in orientation and position affected choice and accuracy. Results show that continuity of orientation leads to a persistent (up to four back) accuracy advantage and a higher preference in the selection of stimuli with the same orientation, and this advantage accumulates over trials. In contrast, analysis of the continuity of the selected position indicated that participants had a strong tendency to choose stimuli in the same position, but this behavior did not lead to an improvement in accuracy.

Modeling facial perception in group context from a serial perception perspective

By utilizing statistical properties and summary statistics, the visual system can efficiently integrate perception of spatially and temporally adjacent stimuli into perception of a given target. For instance, perception of a target face can either be biased positively toward previous faces (e.g. the serial dependence effect) or be biased negatively by surrounding faces in the same trial/space (e.g. spatial ensemble averaging). However, both aspects were investigated separately. As spatial and temporal processing share the same purpose to reduce redundancy in visual processing, if one statistical processing occurs, would the statistical processing in the other domain still exist or be discarded? We investigated this question by exploring whether serial dependence of face perception (of attractiveness and averageness) survives when the changed face perception in the group context occurs. The results of Markov Chain modeling and conventional methods suggested that serial dependence (the temporal aspect) co-occurs with changed face perception in the group context (the spatial aspect). We also utilized the Hidden Markov modeling, as a new mathematical method, to model statistical processing from both domains. The results confirmed the co-occurrence of temporal effect and changed face perception in the group context for both attractiveness and averageness, suggesting potentially different spatial and temporal compression mechanisms in high-level vision. Further modeling and cluster analysis further revealed that the detailed computation of spatially and temporally adjacent faces in the attractiveness and averageness processing were similar yet different among different individuals. This work builds a bridge to understanding mathematical principles underlying changed face perception in the group context from the serial perspective.

Unlocking crowding by ensemble statistics

In crowding,^{1,2,3,4,5,6,7} objects that can be easily recognized in isolation appear jumbled when surrounded by other elements.⁸ Traditionally, crowding is explained by local pooling mechanisms,^{3,6,9,10,11,12,13,14,15} but many findings have shown that the global configuration of the entire stimulus display, rather than local aspects, determines crowding.^{8,16,17,18,19,20,21,22,23,24,25,26,27,28} However, understanding global configurations is challenging because even slight changes can lead from crowding to uncrowding and vice versa.^23,25,28,29 Unfortunately, the number of configurations to explore is virtually infinite. Here, we show that one does not need to know the specific configuration of flankers to determine crowding strength but only their ensemble statistics, which allow for the rapid computation of groups within the stimulus display.^{30,31,32,33,34,35,36,37} To investigate the role of ensemble statistics in (un)crowding, we used a classic vernier offset discrimination task in which the vernier was flanked by multiple squares. We manipulated the orientation statistics of the squares based on the following rationale: a central square with an orientation different from the mean orientation of the other squares stands out from the rest and groups with the vernier, causing strong crowding. If, on the other hand, all squares group together, the vernier is the only element that stands out, and crowding is weak. These effects should depend exclusively on the perceived ensemble statistics, i.e., on the mean orientation of the squares and not on their individual orientations. In two experiments, we confirmed these predictions.