Does vision extract absolute distance from vergence?

Obstacle avoidance of physical, stereoscopic, and pictorial objects

Simulated environments, e.g., virtual or augmented reality environments, are becoming increasingly popular for the investigation and training of motor actions. Yet, so far it remains unclear if results of research and training in those environments transfer in the expected way to natural environments. Here, we investigated the types of visual cues that are required to ensure naturalistic hand movements in simulated environments. We compared obstacle avoidance of physical objects with obstacle avoidance of closely matched 2D and 3D images of the physical objects. Participants were asked to reach towards a target position without colliding with obstacles of varying height that were placed in the movement path. Using a pre-test post-test design, we tested obstacle avoidance for 2D and 3D images of obstacles both before and after exposure to the physical obstacles. Consistent with previous findings, we found that participants initially underestimated the magnitude differences between the obstacles, but after exposure to the physical obstacles avoidance performance for the 3D images became similar to performance for the physical obstacles. No such change was found for 2D images. Our findings highlight the importance of disparity cues for naturalistic motor actions in personal space. Furthermore, they suggest that the observed change in obstacle avoidance for 3D images resulted from a calibration of the disparity cues in the 3D images using an accurate estimate of the egocentric distance to the obstacles gained from the interaction with the physical obstacles.

Depth constancy and the absolute vergence anomaly

Binocular disparity provides information about the depth structure of objects and surfaces in our environment. Since disparity depends on the distance to objects as well as the depth separation of points, information about distance is required to estimate depth from disparity. Our perception of size and shape is biased, such that far objects appear too small and flattened in depth, and near objects too big and stretched in depth. The current study assessed the extent to which the failure of depth constancy can be accounted for by the uncertainty of distance information provided by vergence. We measured individual differences in vergence noise using a nonius line task, and the degree of depth constancy using a task in which observers judged the magnitude of a depth interval relative to the vertical distance between two targets in the image plane. We found no correlation between the two measures, and show that depth constancy was much poorer than would be expected from vergence noise measured in this way. This limited ability to take account of vergence in the perception of depth is, however, consistent with our poor sensitivity to absolute disparity differences. This absolute disparity anomaly thus also applies to our poor ability to make use of vergence information for absolute distance judgements.

Effects of Vergence Eye Movement Planning on Size Perception and Early Visual Processing

Our perception of objects depends on non-oculomotor depth cues, such as pictorial distance cues and binocular disparity, and oculomotor depth cues, such as vergence and accommodation. Although vergence eye movements are always involved in perceiving real distance, previous studies have mainly focused on the effect of oculomotor state via "proprioception" on distance and size perception. It remains unclear whether the oculomotor command of vergence eye movement would also influence visual processing. To address this question, we placed a light at 28.5 cm and a screen for stimulus presentation at 57 cm from the participants. In the NoDivergence condition, participants were asked to maintain fixation on the light regardless of stimulus presentation throughout the trial. In the WithDivergence condition, participants were instructed to initially maintain fixation on the near light and then turn their two eyes outward to look at the stimulus on the far screen. The stimulus was presented for 100 msec, entirely within the preparation stage of the divergence eye movement. We found that participants perceived the stimulus as larger but were less sensitive to stimulus sizes in the WithDivergence condition than in the NoDivergence condition. The earliest visual evoked component C1 (peak latency 80 msec), which varied with stimulus size in the NoDivergence condition, showed similar amplitudes for larger and smaller stimuli in the WithDivergence condition. These results show that vergence eye movement planning affects the earliest visual processing and size perception, and demonstrate an example of the effect of motor command on sensory processing.

Factors Affecting Stimulus Duration Threshold for Depth Discrimination of Asynchronous Targets in the Intermediate Distance Range

Purpose

Binocular depth discrimination in the near distance range (< 2 m) improves with stimulus duration. However, whether the same response-pattern holds in the intermediate distance range (approximately 2-25 m) remains unknown because the spatial coding mechanisms are thought to be different.

Methods

We used the two-interval forced choice procedure to measure absolute depth discrimination of paired asynchronous targets (3, 6, or 16 arc min). The paired targets (0.2 degrees) were located over a distance and height range, respectively, of 4.5 to 7.0 m and 0.15 to 0.7 m. Experiment 1 estimated duration thresholds for binocular depth discrimination at varying target durations (40-1610 ms), in the presence of a 2 × 6 array of parallel texture-elements spanning 1.5 × 5.83 m on the floor. The texture-elements provided a visible background in the light-tight room (9 × 3 m). Experiment 2 used a similar setup to control for viewing conditions: binocular versus monocular and with versus without texture background. Experiment 3 compared binocular depth discrimination between brief (40, 80, and 125 ms) and continuous texture background presentation.

Results

Stimulus duration threshold for depth discrimination decreased with increasing disparity in experiment 1. Experiment 2 revealed depth discrimination performance with texture background was near chance level with monocular viewing. Performance with binocular viewing degraded without texture background. Experiment 3 showed continuous texture background presentation enhances binocular depth discrimination.

Conclusions

Absolute depth discrimination improves with target duration, binocular viewing, and texture background. Performance further improved with longer background duration underscoring the role of ground surface representation in spatial coding.

The contribution of semantic distance knowledge to size constancy in perception and grasping when visual cues are limited

To achieve a stable perception of object size in spite of variations in viewing distance, our visual system needs to combine retinal image information and distance cues. Previous research has shown that, not only retinal cues, but also extraretinal sensory signals can provide reliable information about depth and that different neural networks (perception versus action) can exhibit preferences in the use of these different sources of information during size-distance computations. Semantic knowledge of distance, a purely cognitive signal, can also provide distance information. Do the perception and action systems show differences in their ability to use this information in calculating object size and distance? To address this question, we presented 'glow-in-the-dark' objects of different physical sizes at different real distances in a completely dark room. Participants viewed the objects monocularly through a 1-mm pinhole. They either estimated the size and distance of the objects or attempted to grasp them. Semantic knowledge was manipulated by providing an auditory cue about the actual distance of the object: "20 cm", "30 cm", and "40 cm". We found that semantic knowledge of distance contributed to some extent to size constancy operations during perceptual estimation and grasping, but size constancy was never fully restored. Importantly, the contribution of knowledge about distance to size constancy was equivalent between perception and action. Overall, our study reveals similarities and differences between the perception and action systems in the use of semantic distance knowledge and suggests that this cognitive signal is useful but not a reliable depth cue for size constancy under restricted viewing conditions.

A comparative study of stereopsis measurements: analyzing natural conditions versus dichoptic presentation using smartphones and ultraviolet printer technology

Background

Accurate differentiation between stereopsis assessments in the natural and dichoptic presentation states has proven challenging with commercial stereopsis measurement tools. This study proposes a novel method to delineate these differences more precisely.

Methods

We instituted two stereopsis test systems predicated on a pair of 4K smartphones and a modified Frisby Near Stereotest (FNS) version. Stereoacuity was evaluated both in the natural environment state (via the modified FNS) and the dichoptic state (via smartphones). Thirty subjects aged 20 to 28 years participated in the study with the best-corrected visual acuity (VA) of each eye no less than 0 logMAR and stereoauity of no worse than 40″. Varying degrees of monocular VA loss were induced using the fogging method, while this study does not explore conditions where the VA of both eyes is worse than 0 logMAR.

Results

When the VA difference between the two eyes did not exceed 0.2 logMAR, the modified FNS produced lower stereoacuity values compared to the 4K smartphones (Wilcoxon signed-rank test: difference = 0 logMAR, Z = -3.879, P < 0.001; difference = 0.1 logMAR, Z = -3.478, P = 0.001; difference = 0.2 logMAR, Z = -3.977, P < 0.001). Conversely, no significant differences were observed when the binocular vision difference exceeded 0.2 logMAR (difference = 0.3 logMAR, Z = -1.880, P = 0.060; difference = 0.4 logMAR, Z = -1.784, P = 0.074; difference = 0.5 logMAR, Z = -1.812, P = 0.070).

Conclusion

The findings suggest that stereoacuity values measurements taken in the natural environment state surpass those derived from the dichoptic presentation. However, the observed difference diminishes as stereopsis decreases, corresponding to an increase in induced anisometropia.

Linear perspective cues have a greater effect on the perceptual rescaling of distant stimuli than textures in the virtual environment

The presence of pictorial depth cues in virtual environments is important for minimising distortions driven by unnatural viewing conditions (e.g., vergence-accommodation conflict). Our aim was to determine how different pictorial depth cues affect size constancy in virtual environments under binocular and monocular viewing conditions. We systematically removed linear perspective cues and textures of a hallway in a virtual environment. The experiment was performed using the method of constant stimuli. The task required participants to compare the size of 'far' (10 m) and 'near' (5 m) circles displayed inside a virtual environment with one or both or none of the pictorial depth cues. Participants performed the experiment under binocular and monocular viewing conditions while wearing a virtual reality headset. ANOVA revealed that size constancy was greater for both the far and the near circles in the virtual environment with pictorial depth cues compared to the one without cues. However, the effect of linear perspective cues was stronger than textures, especially for the far circle. We found no difference between the binocular and monocular viewing conditions across the different virtual environments. We conclude that linear perspective cues exert a stronger effect than textures on the perceptual rescaling of far stimuli placed in the virtual environment, and that this effect does not vary between binocular and monocular viewing conditions.

A Framework for the Continuous Evaluation of 3D Motion Perception in Virtual Reality

OBJECTIVE

We present a novel framework for the detection and continuous evaluation of 3D motion perception by deploying a virtual reality environment with built-in eye tracking.

METHODS

We created a biologically-motivated virtual scene that involved a ball moving in a restricted Gaussian random walk against a background of 1/f noise. Sixteen visually healthy participants were asked to follow the moving ball while their eye movements were monitored binocularly using the eye tracker. We calculated the convergence positions of their gaze in 3D using their fronto-parallel coordinates and linear least-squares optimization. Subsequently, to quantify 3D pursuit performance, we employed a first-order linear kernel analysis known as the Eye Movement Correlogram technique to separately analyze the horizontal, vertical and depth components of the eye movements. Finally, we checked the robustness of our method by adding systematic and variable noise to the gaze directions and re-evaluating 3D pursuit performance.

RESULTS

We found that the pursuit performance in the motion-through depth component was reduced significantly compared to that for fronto-parallel motion components. We found that our technique was robust in evaluating 3D motion perception, even when systematic and variable noise was added to the gaze directions.

CONCLUSION

The proposed framework enables the assessment of 3D Motion perception by evaluating continuous pursuit performance through eye-tracking.

SIGNIFICANCE

Our framework paves the way for a rapid, standardized and intuitive assessment of 3D motion perception in patients with various eye disorders.

Active Vision in Binocular Depth Estimation: A Top-Down Perspective

Depth estimation is an ill-posed problem; objects of different shapes or dimensions, even if at different distances, may project to the same image on the retina. Our brain uses several cues for depth estimation, including monocular cues such as motion parallax and binocular cues such as diplopia. However, it remains unclear how the computations required for depth estimation are implemented in biologically plausible ways. State-of-the-art approaches to depth estimation based on deep neural networks implicitly describe the brain as a hierarchical feature detector. Instead, in this paper we propose an alternative approach that casts depth estimation as a problem of active inference. We show that depth can be inferred by inverting a hierarchical generative model that simultaneously predicts the eyes' projections from a 2D belief over an object. Model inversion consists of a series of biologically plausible homogeneous transformations based on Predictive Coding principles. Under the plausible assumption of a nonuniform fovea resolution, depth estimation favors an active vision strategy that fixates the object with the eyes, rendering the depth belief more accurate. This strategy is not realized by first fixating on a target and then estimating the depth; instead, it combines the two processes through action-perception cycles, with a similar mechanism of the saccades during object recognition. The proposed approach requires only local (top-down and bottom-up) message passing, which can be implemented in biologically plausible neural circuits.

Stereopsis without correspondence

Stereopsis has traditionally been considered a complex visual ability, restricted to large-brained animals. The discovery in the 1980s that insects, too, have stereopsis, therefore, challenged theories of stereopsis. How can such simple brains see in three dimensions? A likely answer is that insect stereopsis has evolved to produce simple behaviour, such as orienting towards the closer of two objects or triggering a strike when prey comes within range. Scientific thinking about stereopsis has been unduly anthropomorphic, for example assuming that stereopsis must require binocular fusion or a solution of the stereo correspondence problem. In fact, useful behaviour can be produced with very basic stereoscopic algorithms which make no attempt to achieve fusion or correspondence, or to produce even a coarse map of depth across the visual field. This may explain why some aspects of insect stereopsis seem poorly designed from an engineering point of view: for example, paying no attention to whether interocular contrast or velocities match. Such algorithms demonstrably work well enough in practice for their species, and may prove useful in particular autonomous applications. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Familiar size affects perception differently in virtual reality and the real world

The promise of virtual reality (VR) as a tool for perceptual and cognitive research rests on the assumption that perception in virtual environments generalizes to the real world. Here, we conducted two experiments to compare size and distance perception between VR and physical reality (Maltz et al. 2021 J. Vis. 21, 1-18). In experiment 1, we used VR to present dice and Rubik's cubes at their typical sizes or reversed sizes at distances that maintained a constant visual angle. After viewing the stimuli binocularly (to provide vergence and disparity information) or monocularly, participants manually estimated perceived size and distance. Unlike physical reality, where participants relied less on familiar size and more on presented size during binocular versus monocular viewing, in VR participants relied heavily on familiar size regardless of the availability of binocular cues. In experiment 2, we demonstrated that the effects in VR generalized to other stimuli and to a higher quality VR headset. These results suggest that the use of binocular cues and familiar size differs substantially between virtual and physical reality. A deeper understanding of perceptual differences is necessary before assuming that research outcomes from VR will generalize to the real world. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Minimal theory of 3D vision: new approach to visual scale and visual shape

Since Kepler and Descartes in the early-1600s, vision science has been committed to a triangulation model of stereo vision. But in the early-1800s, we realized that disparities are responsible for stereo vision. And we have spent the past 200 years trying to shoe-horn disparities back into the triangulation account. The first part of this article argues that this is a mistake, and that stereo vision is a solution to a different problem: the eradication of rivalry between the two retinal images, rather than the triangulation of objects in space. This leads to a 'minimal theory of 3D vision', where 3D vision is no longer tied to estimating the scale, shape, and direction of objects in the world. The second part of this article then asks whether the other aspects of 3D vision, which go beyond stereo vision, really operate at the same level of visual experience as stereo vision? I argue they do not. Whilst we want a theory of real-world 3D vision, the literature risks giving us a theory of picture perception instead. And I argue for a two-stage theory, where our purely internal 'minimal' 3D percept (from stereo vision) is linked to the world through cognition. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

From pictures to reality: modelling the phenomenology and psychophysics of 3D perception

The dominant inferential approach to human 3D perception assumes a model of spatial encoding based on a physical description of objects and space. Prevailing models based on this physicalist approach assume that the visual system infers an objective, unitary and mostly veridical representation of the external world. However, careful consideration of the phenomenology of 3D perception challenges these assumptions. I review important aspects of phenomenology, psychophysics and neurophysiology which suggest that human visual perception of 3D  objects and space is underwritten by distinct and dissociated spatial encodings that are optimized for specific regions of space. Specifically, I argue that 3D perception is underwritten by at least three distinct encodings for (1) egocentric distance perception at the ambulatory scale, (2) exocentric distance (scaled depth) perception optimized for near space, and (3) perception of object shape and layout (unscaled depth). This tripartite division can more satisfactorily account for the phenomenology, psychophysics and adaptive logic of human 3D perception. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Stereoscopic depth constancy for physical objects and their virtual counterparts

Stereopsis plays an important role in depth perception; if so, disparity-defined depth should not vary with distance. However, studies of stereoscopic depth constancy often report systematic distortions in depth judgments over distance, particularly for virtual stimuli. Our aim was to understand how depth estimation is impacted by viewing distance and display-based cue conflicts by replicating physical objects in virtual counterparts. To this end, we measured perceived depth using virtual textured half-cylinders and identical three-dimensional (3D) printed versions at two viewing distances under monocular and binocular conditions. Virtual stimuli were viewed using a mirror stereoscope and an Oculus Rift head-mounted display (HMD), while physical stimuli were viewed in a controlled test environment. Depth judgments were similar in both virtual apparatuses, which suggests that variations in the viewing geometry and optics of the HMD have little impact on perceived depth. When viewing physical stimuli binocularly, judgments were accurate and exhibited stereoscopic depth constancy. However, in all cases, depth was underestimated for virtual stimuli and failed to achieve depth constancy. It is clear that depth constancy is only complete for cue-rich physical stimuli and that the failure of constancy in virtual stimuli is due to the presence of the vergence-accommodation conflict. Further, our post hoc analysis revealed that prior experience with virtual and physical environments had a strong effect on depth judgments. That is, performance in virtual environments was enhanced by limited exposure to a related task using physical objects.

Developmental Trajectories of Size Constancy as Implicitly Examined by Simple Reaction Times

It is still unclear whether size constancy is an innate ability or whether it develops with age. As many developmental studies are limited to the child's comprehension of the task instructions, here, an implicit measure of perceived size, namely, simple manual reaction time (RT), was opted for based on the assumption that perceptually bigger objects generate faster detection times. We examined size constancy in children (from 5 to 14 years of age) and adults using a simple RT approach. Participants were presented with pictures of tennis balls on a screen that was physically moved to two viewing distances. Visual stimuli were adjusted in physical size in order to subtend the same visual angle across distances, determining two conditions: a small-near tennis ball vs. a big-far tennis ball. Thanks to size constancy, the two tennis balls were perceived as different even though they were of equal size on the retina. Stimuli were also matched in terms of luminance. Participants were asked to react as fast as possible to the onset of the stimuli. The results show that the RTs reflected the perceived rather than the retinal size of the stimuli across the different age groups, such that participants responded faster to stimuli that were perceived as bigger than those perceived as smaller. Hence, these findings are consistent with the idea that size constancy is already present in early childhood, at least from the age of five, and does not require extensive visual learning.

V1 as an egocentric cognitive map

We typically distinguish between V1 as an egocentric perceptual map and the hippocampus as an allocentric cognitive map. In this article, we argue that V1 also functions as a post-perceptual egocentric cognitive map. We argue that three well-documented functions of V1, namely (i) the estimation of distance, (ii) the estimation of size, and (iii) multisensory integration, are better understood as post-perceptual cognitive inferences. This argument has two important implications. First, we argue that V1 must function as the neural correlates of the visual perception/cognition distinction and suggest how this can be accommodated by V1's laminar structure. Second, we use this insight to propose a low-level account of visual consciousness in contrast to mid-level accounts (recurrent processing theory; integrated information theory) and higher-level accounts (higher-order thought; global workspace theory). Detection thresholds have been traditionally used to rule out such an approach, but we explain why it is a mistake to equate visibility (and therefore the presence/absence of visual experience) with detection thresholds.

When virtual and real worlds coexist: Visualization and visual system affect spatial performance in augmented reality

New visualization approaches are being actively developed aiming to mitigate the effect of vergence-accommodation conflict in stereoscopic augmented reality; however, high interindividual variability in spatial performance makes it difficult to predict user gain. To address this issue, we investigated the effects of consistent and inconsistent binocular and focus cues on perceptual matching in the stereoscopic environment of augmented reality using a head-mounted display that was driven in multifocal and single focal plane modes. Participants matched the distance of a real object with images projected at three viewing distances, concordant with the display focal planes when driven in the multifocal mode. As a result, consistency of depth cues facilitated faster perceptual judgments on spatial relations. Moreover, the individuals with mild binocular and accommodative disorders benefited from the visualization of information on the focal planes corresponding to image planes more than individuals with normal vision, which was reflected in performance accuracy. Because symptoms and complaints may be absent when the functionality of the sensorimotor system is reduced, the results indicate the need for a detailed assessment of visual functions in research on spatial performance. This study highlights that the development of a visualization system that reduces visual stress and improves user performance should be a priority for the successful implementation of augmented reality displays.

Binocular Vision and Stereopsis Across the Animal Kingdom

Most animals have at least some binocular overlap, i.e., a region of space that is viewed by both eyes. This reduces the overall visual field and raises the problem of combining two views of the world, seen from different vantage points, into a coherent whole. However, binocular vision also offers many potential advantages, including increased ability to see around obstacles and increased contrast sensitivity. One particularly interesting use for binocular vision is comparing information from both eyes to derive information about depth. There are many different ways in which this might be done, but in this review, I refer to them all under the general heading of stereopsis. This review examines the different possible uses of binocular vision and stereopsis and compares what is currently known about the neural basis of stereopsis in different taxa. Studying different animals helps us break free of preconceptions stemming from the way that stereopsis operates in human vision and provides new insights into the different possible forms of stereopsis. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Does Vergence Affect Perceived Size?

Since Kepler (1604) and Descartes (1637), it has been suggested that 'vergence' (the angular rotation of the eyes) plays a key role in size constancy. However, this has never been tested divorced from confounding cues such as changes in the retinal image. In our experiment, participants viewed a target which grew or shrank in size over 5 s. At the same time, the fixation distance specified by vergence was reduced from 50 to 25 cm. The question was whether this change in vergence affected the participants' judgements of whether the target grew or shrank in size? We found no evidence of any effect, and therefore no evidence that eye movements affect perceived size. If this is correct, then our finding has three implications. First, perceived size is much more reliant on cognitive influences than previously thought. This is consistent with the argument that visual scale is purely cognitive in nature (Linton, 2017; 2018). Second, it leads us to question whether the vergence modulation of V1 contributes to size constancy. Third, given the interaction between vergence, proprioception, and the retinal image in the Taylor illusion, it leads us to ask whether this cognitive approach could also be applied to multisensory integration.