Stereoscopic depth constancy for physical objects and their virtual counterparts

An equivalent illuminant analysis of lightness constancy with physical objects and in virtual reality

Several previous studies have found significant differences between visual perception in real and virtual environments. Given the increasing use of virtual reality (VR) in performance-critical applications such as medical training and vision research, it is important to understand these differences. Here, we compared lightness constancy in physical and VR environments using a task where viewers matched the reflectance of a fronto-parallel match patch to the reflectance of a reference patch at a range of 3D orientations relative to a light source. We used a custom-built physical apparatus and four VR conditions: (1) All-Cue (replicating the physical apparatus), (2) Reduced-Depth (no disparity or parallax), (3) Shadowless (no cast shadows), and (4) Reduced-Context (no surrounding objects). Lightness constancy was markedly better in the physical condition than in all four VR conditions. Surprisingly, viewers achieved a degree of lightness constancy even in the Reduced-Context condition, despite the absence of lighting cues. In a follow-up experiment, we re-tested the All-Cue and Reduced-Context conditions in VR with new observers, each participating in only one condition. Here, we found lower levels of constancy than in the first experiment, suggesting that experience across multiple experimental settings and possibly exposure to the physical apparatus during instructions had enhanced performance. We conclude that even when robust lighting and shape cues are available, lightness constancy is substantially better in real environments than in virtual environments. We consider possible explanations for this finding, such as the imperfect models of materials and lighting that are used for rendering in real-time VR.

Obstacle avoidance of physical, stereoscopic, and pictorial objects

Simulated environments, e.g., virtual or augmented reality environments, are becoming increasingly popular for the investigation and training of motor actions. Yet, so far it remains unclear if results of research and training in those environments transfer in the expected way to natural environments. Here, we investigated the types of visual cues that are required to ensure naturalistic hand movements in simulated environments. We compared obstacle avoidance of physical objects with obstacle avoidance of closely matched 2D and 3D images of the physical objects. Participants were asked to reach towards a target position without colliding with obstacles of varying height that were placed in the movement path. Using a pre-test post-test design, we tested obstacle avoidance for 2D and 3D images of obstacles both before and after exposure to the physical obstacles. Consistent with previous findings, we found that participants initially underestimated the magnitude differences between the obstacles, but after exposure to the physical obstacles avoidance performance for the 3D images became similar to performance for the physical obstacles. No such change was found for 2D images. Our findings highlight the importance of disparity cues for naturalistic motor actions in personal space. Furthermore, they suggest that the observed change in obstacle avoidance for 3D images resulted from a calibration of the disparity cues in the 3D images using an accurate estimate of the egocentric distance to the obstacles gained from the interaction with the physical obstacles.

Lightness constancy in reality, in virtual reality, and on flat-panel displays

Virtual reality (VR) displays are being used in an increasingly wide range of applications. However, previous work shows that viewers often perceive scene properties very differently in real and virtual environments and so realistic perception of virtual stimuli should always be a carefully tested conclusion, not an assumption. One important property for realistic scene perception is surface color. To evaluate how well virtual platforms support realistic perception of achromatic surface color, we assessed lightness constancy in a physical apparatus with real lights and surfaces, in a commercial VR headset, and on a traditional flat-panel display. We found that lightness constancy was good in all three environments, though significantly better in the real environment than on the flat-panel display. We also found that variability across observers was significantly greater in VR and on the flat-panel display than in the physical environment. We conclude that these discrepancies should be taken into account in applications where realistic perception is critical but also that in many cases VR can be used as a flexible alternative to flat-panel displays and a reasonable proxy for real environments.

Applications and implications for extended reality to improve binocular vision and stereopsis

Extended reality (XR) devices, including virtual reality (VR), augmented reality (AR), and mixed reality (MR) devices, are immersive technologies that can swap or merge the natural environment with virtual content (e.g., videogames, movies, or other content). Although these devices are widely used for playing videogames and other applications, they have one distinct feature that makes them potentially very useful for the measurement and treatment of binocular vision anomalies-they can deliver different content to the two eyes simultaneously. Indeed, horizontally shifting the images in the two eyes (thereby creating binocular disparity) can provide the user with a compelling percept of depth through stereopsis. Because these devices are stereoscopic, they can also be used as high-tech synoptophores, in which the images to the two eyes differ in contrast, luminance, size, position, and content for measuring and treating binocular anomalies. The inclusion of eye tracking in VR adds an additional dimension to its utility in measuring and treating binocular vision anomalies, as well as other conditions. This paper describes the essential requirements for testing and treating binocular anomalies and reviews current studies in which XR devices have been used to measure and treat binocular vision anomalies.