Efficient sensory encoding predicts robust averaging

The tilt illusion arises from an efficient reallocation of neural coding resources at the contextual boundary

The tilt illusion-a bias in the perceived orientation of a center stimulus induced by an oriented surround-illustrates how context shapes visual perception. Although extensively studied for decades, we still lack a comprehensive account of the illusion that connects its behavioral and neural characteristics. Here, we demonstrate that the tilt illusion originates from dynamic changes in neural coding precision induced by the surround context. We simultaneously obtained psychophysical and functional MRI responses from human subjects while they viewed gratings in the absence and presence of an oriented surround and independently extracted sensory encoding precision from their behavioral and neural data. Both measures show that in the absence of an oriented surround, encoding reflects the natural scene statistics of orientation. However, with an oriented surround, encoding precision is significantly increased for stimuli similar to the surround orientation. This local change in encoding is sufficient to predict the behavioral characteristics of the tilt illusion using a Bayesian observer model. The effect of surround modulation increases along the ventral stream and is localized to the portion of the visual cortex with receptive fields at the center-surround boundary. The pattern of change in coding accuracy reflects the surround-conditioned orientation statistics in natural scenes, but cannot be explained by local stimulus configuration. Our results suggest that the tilt illusion naturally emerges from an adaptive coding strategy that efficiently reallocates neural coding resources based on the current stimulus context.

Time Course of Orientation Ensemble Representation in the Human Brain

Natural scenes are filled with groups of similar items. Humans employ ensemble coding to extract the summary statistical information of the environment, thereby enhancing the efficiency of information processing, something particularly useful when observing natural scenes. However, the neural mechanisms underlying the representation of ensemble information in the brain remain elusive. In particular, whether ensemble representation results from the mere summation of individual item representations or it engages other specific processes remains unclear. In this study, we utilized a set of orientation ensembles wherein none of the individual item orientations were the same as the ensemble orientation. We recorded magnetoencephalography (MEG) signals from human participants (both sexes) when they performed an ensemble orientation discrimination task. Time-resolved multivariate pattern analysis (MVPA) and the inverted encoding model (IEM) were employed to unravel the neural mechanisms of the ensemble orientation representation and track its time course. First, we achieved successful decoding of the ensemble orientation, with a high correlation between the decoding and behavioral accuracies. Second, the IEM analysis demonstrated that the representation of the ensemble orientation differed from the sum of the representations of individual item orientations, suggesting that ensemble coding could further modulate orientation representation in the brain. Moreover, using source reconstruction, we showed that the representation of ensemble orientation manifested in early visual areas. Taken together, our findings reveal the emergence of the ensemble representation in the human visual cortex and advance the understanding of how the brain captures and represents ensemble information.

The tilt illusion arises from an efficient reallocation of neural coding resources at the contextual boundary

The tilt illusion - a bias in the perceived orientation of a center stimulus induced by an oriented surround - illustrates how context shapes visual perception. While the tilt illusion has been the subject of quantitative study for over 85 years, we still lack a comprehensive account of the phenomenon that connects its neural and behavioral characteristics. Here, we demonstrate that the tilt illusion originates from a dynamic change in neural coding precision induced by the surround context. We simultaneously obtained psychophysical and fMRI responses from human subjects while they viewed gratings in the absence and presence of an oriented surround, and extracted sensory encoding precision from their behavioral and neural data. Both measures show that in the absence of a surround, encoding reflects the natural scene statistics of orientation. However, in the presence of an oriented surround, encoding precision is significantly increased for stimuli similar to the surround orientation. This local change in encoding is sufficient to accurately predict the behavioral characteristics of the tilt illusion using a Bayesian observer model. The effect of surround modulation increases along the ventral stream, and is localized to the portion of the visual cortex with receptive fields at the center-surround boundary. The pattern of change in coding accuracy reflects the surround-conditioned orientation statistics in natural scenes, but cannot be explained by local stimulus configuration. Our results suggest that the tilt illusion naturally emerges from a dynamic coding strategy that efficiently reallocates neural coding resources based on the current stimulus context.

Mean orientation discrimination based on proximal stimuli

Ensemble perception refers to the ability to accurately and rapidly perceive summary statistical representations of specific features from a group of similar objects. However, the specific type of representation involved in this perception within a three-dimensional (3-D) environment remains unclear. In the context of perspective viewing with stereopsis, distal stimuli can be projected onto the retina as different forms of proximal stimuli based on their distances, despite sharing similar properties, such as object size and spatial frequency. This study aimed to investigate the effects of distal and proximal stimuli on the perception of summary statistical information related to orientation. In our experiment, we presented multiple Gabor patches in a stereoscopic environment, allowing us to measure the discrimination threshold of the mean orientation. The object size and spatial frequency were fixed for all patches regardless of depth. However, the physical angular size and absolute spatial frequency covaried with the depth. The results revealed the threshold elevation with depth expansion, especially when the patches formed two clusters at near and far distances, leading to large variations in their retinotopic representations. This finding indicates a minor contribution of similarity of the distal stimuli. Subsequent experiments demonstrated that the variability in physical angular size of the patches significantly influenced the threshold elevation in contrast to that of binocular disparity and absolute spatial frequency. These findings highlight the critical role of physical angular size variability in perceiving mean orientations within the 3-D space.

An efficient Bayesian observer model of attractive and repulsive temporal context effects when perceiving multistable dot lattices

In multistable dot lattices, the orientation we perceive is attracted toward the orientation we perceived in the immediately preceding stimulus and repelled from the orientation for which most evidence was present previously (Van Geert, Moors, Haaf, & Wagemans, 2022). Theoretically-inspired models have been proposed to explain the co-occurrence of attractive and repulsive context effects in multistable dot lattice tasks, but these models artificially induced an influence of the previous trial on the current one without detailing the process underlying such an influence (Gepshtein & Kubovy, 2005; Schwiedrzik et al., 2014). We conducted a simulation study to test whether the observed attractive and repulsive context effects could be explained with an efficient Bayesian observer model (Wei & Stocker, 2015). This model assumes variable encoding precision of orientations in line with their frequency of occurrence (i.e., efficient encoding) and takes the dissimilarity between stimulus space and sensory space into account. An efficient Bayesian observer model including both a stimulus and a perceptual level was needed to explain the co-occurrence of both attractive and repulsive temporal context effects. Furthermore, this model could reproduce the empirically observed strong positive correlation between individuals' attractive and repulsive effects (Van Geert et al., 2022), by assuming a positive correlation between temporal integration constants at the stimulus and the perceptual level. To conclude, the study brings evidence that efficient encoding and likelihood repulsion on the stimulus level can explain the repulsive context effect, whereas perceptual prior attraction can explain the attractive temporal context effect when perceiving multistable dot lattices.