Changing the Real Viewing Distance Reveals the Temporal Evolution of Size Constancy in Visual Cortex

The contribution of semantic distance knowledge to size constancy in perception and grasping when visual cues are limited

To achieve a stable perception of object size in spite of variations in viewing distance, our visual system needs to combine retinal image information and distance cues. Previous research has shown that, not only retinal cues, but also extraretinal sensory signals can provide reliable information about depth and that different neural networks (perception versus action) can exhibit preferences in the use of these different sources of information during size-distance computations. Semantic knowledge of distance, a purely cognitive signal, can also provide distance information. Do the perception and action systems show differences in their ability to use this information in calculating object size and distance? To address this question, we presented 'glow-in-the-dark' objects of different physical sizes at different real distances in a completely dark room. Participants viewed the objects monocularly through a 1-mm pinhole. They either estimated the size and distance of the objects or attempted to grasp them. Semantic knowledge was manipulated by providing an auditory cue about the actual distance of the object: "20 cm", "30 cm", and "40 cm". We found that semantic knowledge of distance contributed to some extent to size constancy operations during perceptual estimation and grasping, but size constancy was never fully restored. Importantly, the contribution of knowledge about distance to size constancy was equivalent between perception and action. Overall, our study reveals similarities and differences between the perception and action systems in the use of semantic distance knowledge and suggests that this cognitive signal is useful but not a reliable depth cue for size constancy under restricted viewing conditions.

Internal representations of the canonical real-world distance of objects

In the real world, every object has its canonical distance from observers. For example, airplanes are usually far away from us, whereas eyeglasses are close to us. Do we have an internal representation of the canonical real-world distance of objects in our cognitive system? If we do, does the canonical distance influence the perceived size of an object? Here, we conducted two experiments to address these questions. In Experiment 1, we first asked participants to rate the canonical distance of objects. Participants gave consistent ratings to each object. Then, pairs of object images were presented one by one in a trial, and participants were asked to rate the distance of the second object (i.e., a priming paradigm). We found that the rating of the perceived distance of the target object was modulated by the canonical real-world distance of the prime. In Experiment 2, participants were asked to judge the perceived size of canonically near or far objects that were presented at the converging end (i.e., far location) or the opening end (i.e., near location) of a background image with converging lines. We found that regardless of the presentation location, participants perceived the canonically near object as smaller than the canonically far object even though their retinal and real-world sizes were matched. In all, our results suggest that we have an internal representation of the canonical real-world distance of objects, which affects the perceived distance of subsequent objects and the perceived size of the objects themselves.

Temporal features of size constancy for perception and action in a real-world setting: A combined EEG-kinematics study

A stable representation of object size, in spite of continuous variations in retinal input due to changes in viewing distance, is critical for perceiving and acting in a real 3D world. In fact, our perceptual and visuo-motor systems exhibit size and grip constancies in order to compensate for the natural shrinkage of the retinal image with increased distance. The neural basis of this size-distance scaling remains largely unknown, although multiple lines of evidence suggest that size-constancy operations might take place remarkably early, already at the level of the primary visual cortex. In this study, we examined for the first time the temporal dynamics of size constancy during perception and action by using a combined measurement of event-related potentials (ERPs) and kinematics. Participants were asked to maintain their gaze steadily on a fixation point and perform either a manual estimation or a grasping task towards disks of different sizes placed at different distances. Importantly, the physical size of the target was scaled with distance to yield a constant retinal angle. Meanwhile, we recorded EEG data from 64 scalp electrodes and hand movements with a motion capture system. We focused on the first positive-going visual evoked component peaking at approximately 90 ms after stimulus onset. We found earlier latencies and greater amplitudes in response to bigger than smaller disks of matched retinal size, regardless of the task. In line with the ERP results, manual estimates and peak grip apertures were larger for the bigger targets. We also found task-related differences at later stages of processing from a cluster of central electrodes, whereby the mean amplitude of the P2 component was greater for manual estimation than grasping. Taken together, these findings provide novel evidence that size constancy for real objects at real distances occurs at the earliest cortical stages and that early visual processing does not change as a function of task demands.

Behavioral examination of the role of the primary visual cortex in the perceived size representation

Previous research has shown that neural activity in the primary visual cortex (V1) and V1 surface area may be linked with subjective experience of size illusions. Here, we behaviorally measured the hallway illusion with experimental manipulations as a proxy of V1's influence on size perception. We first tested whether the hallway illusion can persist without further recurrent processing by using backward masking. Next, we examined relations among the hallway illusion magnitude and other perceptual measures that have been suggested to be correlated with V1 surface area. In Experiment 1, the magnitude of the hallway illusion was not affected by the stimulus duration and visual masking when the hallway context was previewed (i.e., complex depth information is already processed). It suggests that V1 activity could support the size illusion to some extent even when recurrent processing between V1 and higher areas is disturbed. In Experiment 2, the hallway illusion magnitude was correlated with the Vernier acuity threshold, but not with physical size discriminability. Our results provide converging evidence with the previous findings in that neural activity in V1 may contribute to size illusions and that V1 surface area is not the sole factor that mediates size perception and visual precision.

The spatial properties of adaptation-induced distance compression

Exposure to a dynamic texture reduces the perceived separation between objects, altering the mapping between physical relations in the environment and their neural representations. Here we investigated the spatial tuning and spatial frame of reference of this aftereffect to understand the stage(s) of processing where adaptation-induced changes occur. In Experiment 1, we measured apparent separation at different positions relative to the adapted area, revealing a strong but tightly tuned compression effect. We next tested the spatial frame of reference of the effect, either by introducing a gaze shift between adaptation and test phase (Experiment 2) or by decoupling the spatial selectivity of adaptation in retinotopic and world-centered coordinates (Experiment 3). Results across the two experiments indicated that both retinotopic and world-centered adaptation effects can occur independently. Spatial attention to the location of the adaptor alone could not account for the world-centered transfer we observed, and retinotopic adaptation did not transfer to world-centered coordinates after a saccade (Experiment 4). Finally, we found that aftereffects in different reference frames have a similar, narrow spatial tuning profile (Experiment 5). Together, our results suggest that the neural representation of local separation resides early in the visual cortex, but it can also be modulated by activity in higher visual areas.

Developmental Trajectories of Size Constancy as Implicitly Examined by Simple Reaction Times

It is still unclear whether size constancy is an innate ability or whether it develops with age. As many developmental studies are limited to the child's comprehension of the task instructions, here, an implicit measure of perceived size, namely, simple manual reaction time (RT), was opted for based on the assumption that perceptually bigger objects generate faster detection times. We examined size constancy in children (from 5 to 14 years of age) and adults using a simple RT approach. Participants were presented with pictures of tennis balls on a screen that was physically moved to two viewing distances. Visual stimuli were adjusted in physical size in order to subtend the same visual angle across distances, determining two conditions: a small-near tennis ball vs. a big-far tennis ball. Thanks to size constancy, the two tennis balls were perceived as different even though they were of equal size on the retina. Stimuli were also matched in terms of luminance. Participants were asked to react as fast as possible to the onset of the stimuli. The results show that the RTs reflected the perceived rather than the retinal size of the stimuli across the different age groups, such that participants responded faster to stimuli that were perceived as bigger than those perceived as smaller. Hence, these findings are consistent with the idea that size constancy is already present in early childhood, at least from the age of five, and does not require extensive visual learning.

V1 as an egocentric cognitive map

We typically distinguish between V1 as an egocentric perceptual map and the hippocampus as an allocentric cognitive map. In this article, we argue that V1 also functions as a post-perceptual egocentric cognitive map. We argue that three well-documented functions of V1, namely (i) the estimation of distance, (ii) the estimation of size, and (iii) multisensory integration, are better understood as post-perceptual cognitive inferences. This argument has two important implications. First, we argue that V1 must function as the neural correlates of the visual perception/cognition distinction and suggest how this can be accommodated by V1's laminar structure. Second, we use this insight to propose a low-level account of visual consciousness in contrast to mid-level accounts (recurrent processing theory; integrated information theory) and higher-level accounts (higher-order thought; global workspace theory). Detection thresholds have been traditionally used to rule out such an approach, but we explain why it is a mistake to equate visibility (and therefore the presence/absence of visual experience) with detection thresholds.

Pupillary Responses Obey Emmert's Law and Co-vary with Autistic Traits

We measured the pupil response to a light stimulus subject to a size illusion and found that stimuli perceived as larger evoke a stronger pupillary response. The size illusion depends on combining retinal signals with contextual 3D information; contextual processing is thought to vary across individuals, being weaker in individuals with stronger autistic traits. Consistent with this theory, autistic traits correlated negatively with the magnitude of pupil modulations in our sample of neurotypical adults; however, psychophysical measurements of the illusion did not correlate with autistic traits, or with the pupil modulations. This shows that pupillometry provides an accurate objective index of complex perceptual processes, particularly useful for quantifying interindividual differences, and potentially more informative than standard psychophysical measures.

Does Vergence Affect Perceived Size?

Since Kepler (1604) and Descartes (1637), it has been suggested that 'vergence' (the angular rotation of the eyes) plays a key role in size constancy. However, this has never been tested divorced from confounding cues such as changes in the retinal image. In our experiment, participants viewed a target which grew or shrank in size over 5 s. At the same time, the fixation distance specified by vergence was reduced from 50 to 25 cm. The question was whether this change in vergence affected the participants' judgements of whether the target grew or shrank in size? We found no evidence of any effect, and therefore no evidence that eye movements affect perceived size. If this is correct, then our finding has three implications. First, perceived size is much more reliant on cognitive influences than previously thought. This is consistent with the argument that visual scale is purely cognitive in nature (Linton, 2017; 2018). Second, it leads us to question whether the vergence modulation of V1 contributes to size constancy. Third, given the interaction between vergence, proprioception, and the retinal image in the Taylor illusion, it leads us to ask whether this cognitive approach could also be applied to multisensory integration.

The conceptual understanding of depth rather than the low-level processing of spatial frequencies drives the corridor illusion

Our objective was to determine how different spatial frequencies affect the perceptual size rescaling of stimuli in the corridor illusion. Two experiments were performed using the method of constant stimuli. In experiment 1, the task required participants to compare the size of comparison and standard rings displayed over the same background image. ANOVA on the points of subject equality (PSEs) revealed that the perceived size of the top and bottom standard rings changed as a function of the availability of the high, medium, and low spatial frequency information. In experiment 2, the task required participants to compare the size of a comparison ring presented outside of the background image with a standard ring presented inside it. ANOVA on the PSEs revealed that the apparent size of the top and not the bottom standard ring changed depending on the availability of medium spatial frequency information. Eye-tracking revealed that the spatial frequency range of the background image in the periphery affected participants' eye positioning, which may explain why the effects of different spatial frequencies fluctuated across experiments. Nonetheless, when we consider these findings together, we propose that the conceptual understanding of depth plays a more important role in explaining the corridor illusion than the low-level processing of depth information extracted from different spatial frequencies along separate channels.

Opposite effects of motion dynamics on the Ebbinghaus and corridor illusions

We recently showed that motion dynamics greatly enhance the magnitude of certain size contrast illusions, such as the Ebbinghaus and Delboeuf illusions. Here, we extend our study of the effect of motion dynamics on size illusions through a novel dynamic corridor illusion, in which a single target translates along a corridor background. Across three psychophysical experiments, we quantify the effects of stimulus dynamics on the Ebbinghaus and corridor illusions across different viewing conditions. The results revealed that stimulus dynamics had opposite effects on these different classes of size illusions. Whereas dynamic motion enhanced the magnitude of the Ebbinghaus illusion, it attenuated the magnitude the corridor illusion. Our results highlight precision-driven weighting of visual cues by neural circuits computing perceived object size. This hypothesis is consistent with observations beyond size perception and may represent a more general principle of cue integration in the visual system.

Resting EEG in alpha band predicts individual differences in visual size perception

Human visual size perception results from an interaction of external sensory information and internal state. The cognitive mechanisms involved in the processing of context-dependent visual size perception have been found to be innate in nature to some extent, suggesting that visual size perception might correlate with human intrinsic brain activity. Here we recorded human resting alpha activity (8-12 Hz), which is an inverse indicator of sustained alertness. Moreover, we measured an object's perceived size in a two-alternative forced-choice manner and the Ebbinghaus illusion magnitude which is a classic illustration of context-dependent visual size perception. The results showed that alpha activity along the ventral visual pathway, including left V1, right LOC and bilateral inferior temporal gyrus, negatively correlated with an object's perceived size. Moreover, alpha activity in the left superior temporal gyrus positively correlated with size discrimination threshold and size illusion magnitude. The findings provide clear evidence that human visual size perception scales as a function of intrinsic alertness, with higher alertness linking to larger perceived size of objects and better performance in size discrimination and size illusion tasks, and suggest that individual variation in resting-state brain activity provides a neural explanation for individual variation in cognitive performance of normal participants.

Size aftereffect is non-local

It is well known that prolonged exposure to a certain size stimulus alters the perceived size of a subsequently presented stimulus at the same location. How the rest of the visual space is affected by this size adaptation, however, has not been systematically studied before. Here, to fill this gap in literature, we tested size adaptation at the adapter location as well as the rest of the visual space. We used peripherally presented solid discs (Experiment 1) and rings (Experiment 2) as adapter and target (test) stimuli. Observers adapted to a mid-sized stimulus and judged the size of the subsequently presented smaller or larger target stimuli. Results showed that the perceived sizes of target stimuli were repelled away from the adapter size, not only at the adapter location but also at other locations. These findings demonstrate that size adaptation causes widespread distortion of the visual space and alters perceived size. We discuss possible computational models that may underpin the perceptual effect.

Grip Constancy but Not Perceptual Size Constancy Survives Lesions of Early Visual Cortex

Object constancies are central constructs in theories of visual phenomenology. A powerful example is "size constancy," in which the perceived size of an object remains stable despite changes in viewing distance [1-4]. Evidence from neuropsychology [5], neuroimaging [6-11], transcranial magnetic stimulation [12, 13], single-unit and lesion studies in monkey [14-20], and computational modeling [21] suggests that re-entrant processes involving reciprocal interactions between primary visual cortex (V1) and extrastriate visual areas [22-26] play an essential role in mediating size constancy. It is seldom appreciated, however, that object constancies must also operate for the visual guidance of goal-directed action. For example, when reaching out to pick up an object, the hand's in-flight aperture scales with size of the goal object [27-30] and is refractory to the decrease in retinal-image size with increased viewing distance [31-41] (Figure 1), a phenomenon we call "grip constancy." Does grip constancy, like perceptual constancy, depend on V1 or can it be mediated by pathways that bypass it altogether? We tested these possibilities in an individual, M.C., who has bilateral lesions encompassing V1 and much of the ventral visual stream. We show that her perceptual estimates of object size co-vary with retinal-image size rather than real-world size as viewing distance varies. In contrast, M.C. shows near-normal scaling of in-flight grasp aperture to object size despite changes in viewing distance. Thus, although early visual cortex is necessary for perceptual object constancy, it is unnecessary for grip constancy, which is mediated instead by separate visual inputs to dorsal-stream visuomotor areas [42-48].

Visual Size Processing in Early Visual Cortex Follows Lateral Occipital Cortex Involvement

Neural activation in the early visual cortex (EVC) reflects the perceived rather than retinal size of stimuli, suggesting that feedback possibly from extrastriate regions modulates retinal size information in EVC. Meanwhile, the lateral occipital cortex (LOC) has been suggested to be critically involved in object size processing. To test for the potential contributions of feedback modulations on size representations in EVC, we investigated the dynamics of relevant processes using transcranial magnetic stimulation (TMS). Specifically, we briefly disrupted the neural activity of EVC and LOC at early, intermediate, and late time windows while participants performed size judgment tasks in either an illusory or neutral context. TMS over EVC and LOC allowed determining whether these two brain regions are relevant for generating phenomenological size impressions. Furthermore, the temporal order of TMS effects allowed inferences on the dynamics of information exchange between the two areas. Particularly, if feedback signals from LOC to EVC are crucial for generating altered size representations in EVC, then TMS effects over EVC should be observed simultaneously or later than the effects following LOC stimulation. The data from 20 humans (13 females) revealed that TMS over both EVC and LOC impaired illusory size perception. However, the strongest effects of TMS applied over EVC occurred later than those of LOC, supporting a functionally relevant feedback modulation from LOC to EVC for scaling size information. Our results suggest that context integration and the concomitant change of perceived size require LOC and result in modulating representations in EVC via recurrent processing.SIGNIFICANCE STATEMENT How we perceive an object's size is not entirely determined by its physical size or the size of its retinal representation but also the spatial context. Using transcranial magnetic stimulation, we investigated the role of the early visual cortex (EVC) and the higher-level visual area, lateral occipital cortex (LOC), known to be critically involved in object processing, in transforming an initial retinal representation into one that reflects perceived size. Transcranial magnetic stimulation altered size perception earlier over LOC compared with EVC, suggesting that context integration and the concomitant change in perceived size representations in EVC rely on feedback from LOC.

Scenes Modulate Object Processing Before Interacting With Memory Templates

When searching for relevant objects in our environment (say, an apple), we create a memory template (a red sphere), which causes our visual system to favor template-matching visual input (applelike objects) at the expense of template-mismatching visual input (e.g., leaves). Although this principle seems straightforward in a lab setting, it poses a problem in naturalistic viewing: Two objects that have the same size on the retina will differ in real-world size if one is nearby and the other is far away. Using the Ponzo illusion to manipulate perceived size while keeping retinal size constant, we demonstrated across 71 participants that visual objects attract attention when their perceived size matches a memory template, compared with mismatching objects that have the same size on the retina. This shows that memory templates affect visual selection after object representations are modulated by scene context, thus providing a working mechanism for template-based search in naturalistic vision.