Temporal presentation protocols in stereoscopic displays: Flicker visibility, perceived motion, and perceived depth

BiPMAP: a toolbox for predicting perceived motion artifacts on modern displays

Viewers of digital displays often experience motion artifacts (e.g., flicker, judder, edge banding, motion blur, color breakup, depth distortion) when presented with dynamic scenes. We developed an interactive software tool for display designers that predicts how a viewer perceives motion artifacts for a variety of stimulus, display, and viewing parameters: the Binocular Perceived Motion Artifact Predictor (BiPMAP). The tool enables the user to specify numerous stimulus, display, and viewing parameters. It implements a model of human spatiotemporal contrast sensitivity in order to determine which artifacts will be seen by a viewer and which will not. The tool visualizes the perceptual effects of discrete space-time sampling on the display by presenting side by side the expected perception when the stimulus is continuous compared to when the same stimulus is presented with the spatial and temporal parameters of a prototype display.

Neuropsychological and Neurophysiological Mechanisms behind Flickering Light Stimulus Processing

The aim of this review is to summarise current knowledge about flickering light and the underlying processes that occur during its processing in the brain. Despite the growing interest in the topic of flickering light, its clinical applications are still not well understood. Studies using EEG indicate an appearing synchronisation of brain wave frequencies with the frequency of flickering light, and hopefully, it could be used in memory therapy, among other applications. Some researchers have focused on using the flicker test as an indicator of arousal, which may be useful in clinical studies if the background for such a relationship is described. Since flicker testing has a risk of inducing epileptic seizures, however, every effort must be made to avoid high-risk combinations, which include, for example, red-blue light flashing at 15 Hz. Future research should focus on the usage of neuroimaging methods to describe the specific neuropsychological and neurophysiological processes occurring in the brain during the processing of flickering light so that its clinical utility can be preliminarily determined and randomised clinical trials can be initiated to test existing reports.

Enhancing Static Charts With Data-Driven Animations

Static visual attributes such as color and shape are used with great success in visual charts designed to be displayed in static, hard-copy form. However, nowadays digital displays become ubiquitous in the visualization of any form of data, lifting the confines of static presentations. In this article, we propose incorporating data-driven animations to bring static charts to life, with the purpose of encoding and emphasizing certain attributes of the data. We lay out a design space for data-driven animated effects and experiment with three versatile effects, marching ants, geometry deformation and gradual appearance. For each, we provide practical details regarding their mode of operation and extent of interaction with existing visual encodings. We examine the impact and effectiveness of our enhancements through an empirical user study to assess preference as well as gauge the influence of animated effects on human perception in terms of speed and accuracy of visual understanding.

Visual Perception: Monovision Can Bias the Apparent Depth of Moving Objects

'Monovision' - using one eye for near work and one for distance - is a common alternative to reading glasses. New work shows that monovision can cause the distance of moving objects to be misestimated, with potentially serious consequences.

Temporal Resolution Multiplexing: Exploiting the limitations of spatio-temporal vision for more efficient VR rendering

Rendering in virtual reality (VR) requires substantial computational power to generate 90 frames per second at high resolution with good-quality antialiasing. The video data sent to a VR headset requires high bandwidth, achievable only on dedicated links. In this paper we explain how rendering requirements and transmission bandwidth can be reduced using a conceptually simple technique that integrates well with existing rendering pipelines. Every even-numbered frame is rendered at a lower resolution, and every odd-numbered frame is kept at high resolution but is modified in order to compensate for the previous loss of high spatial frequencies. When the frames are seen at a high frame rate, they are fused and perceived as high-resolution and high-frame-rate animation. The technique relies on the limited ability of the visual system to perceive high spatio-temporal frequencies. Despite its conceptual simplicity, correct execution of the technique requires a number of non-trivial steps: display photometric temporal response must be modeled, flicker and motion artifacts must be avoided, and the generated signal must not exceed the dynamic range of the display. Our experiments, performed on a high-frame-rate LCD monitor and OLED-based VR headsets, explore the parameter space of the proposed technique and demonstrate that its perceived quality is indistinguishable from full-resolution rendering. The technique is an attractive alternative to reprojection and resolution reduction of all frames.

Avoiding monocular artifacts in clinical stereotests presented on column-interleaved digital stereoscopic displays

New forms of stereoscopic 3-D technology offer vision scientists new opportunities for research, but also come with distinct problems. Here we consider autostereo displays where the two eyes' images are spatially interleaved in alternating columns of pixels and no glasses or special optics are required. Column-interleaved displays produce an excellent stereoscopic effect, but subtle changes in the angle of view can increase cross talk or even interchange the left and right eyes' images. This creates several challenges to the presentation of cyclopean stereograms (containing structure which is only detectable by binocular vision). We discuss the potential artifacts, including one that is unique to column-interleaved displays, whereby scene elements such as dots in a random-dot stereogram appear wider or narrower depending on the sign of their disparity. We derive an algorithm for creating stimuli which are free from this artifact. We show that this and other artifacts can be avoided by (a) using a task which is robust to disparity-sign inversion—for example, a disparity-detection rather than discrimination task—(b) using our proposed algorithm to ensure that parallax is applied symmetrically on the column-interleaved display, and (c) using a dynamic stimulus to avoid monocular artifacts from motion parallax. In order to test our recommendations, we performed two experiments using a stereoacuity task implemented with a parallax-barrier tablet. Our results confirm that these recommendations eliminate the artifacts. We believe that these recommendations will be useful to vision scientists interested in running stereo psychophysics experiments using parallax-barrier and other column-interleaved digital displays.

3D Displays

Creating realistic three-dimensional (3D) experiences has been a very active area of research and development, and this article describes progress and what remains to be solved. A very active area of technical development has been to build displays that create the correct relationship between viewing parameters and triangulation depth cues: stereo, motion, and focus. Several disciplines are involved in the design, construction, evaluation, and use of 3D displays, but an understanding of human vision is crucial to this enterprise because in the end, the goal is to provide the desired perceptual experience for the viewer. In this article, we review research and development concerning displays that create 3D experiences. And we highlight areas in which further research and development is needed.

The Conflicting Evidence of Three-dimensional Displays in Laparoscopy: A Review of Systems Old and New

OBJECTIVE

To describe studies evaluating 3 generations of three-dimensional (3D) displays over the course of 20 years.

SUMMARY BACKGROUND DATA

Most previous studies have analyzed performance differences during 3D and two-dimensional (2D) laparoscopy without using appropriate controls that equated conditions in all respects except for 3D or 2D viewing.

METHODS

Databases search consisted of MEDLINE and PubMed. The reference lists for all relevant articles were also reviewed for additional articles. The search strategy employed the use of keywords "3D," "Laparoscopic," "Laparoscopy," "Performance," "Education," "Learning," and "Surgery" in appropriate combinations.

RESULTS

Our current understanding of the performance metrics between 3D and 2D laparoscopy is mostly from the research with flawed study designs. This review has been written in a qualitative style to explain in detail how prior research has underestimated the potential benefit of 3D displays and the improvements that must be made in future experiments comparing 3D and 2D displays to better determine any advantage of using one display or the other.

CONCLUSIONS

Individual laparoscopic performance in 3D may be affected by a multitude of factors. It is crucial for studies to measure participant stereoscopic ability, control for system crosstalk, and use validated measures of performance.

Temporal Video Filtering and Exposure Control for Perceptual Motion Blur

We propose the computation of a perceptual motion blur in videos. Our technique takes the predicted eye motion into account when watching the video. Compared to traditional motion blur recorded by a video camera our approach results in a perceptual blur that is closer to reality. This postprocess can also be used to simulate different shutter effects or for other artistic purposes. It handles real and artificial video input, is easy to compute and has a low additional cost for rendered content. We illustrate its advantages in a user study using eye tracking.

Stereoscopic 3D display technique using spatiotemporal interlacing has improved spatial and temporal properties

Stereoscopic 3D (S3D) displays use spatial or temporal interlacing to send different images to the two eyes. Temporal interlacing delivers images to the left and right eyes alternately in time; it has high effective spatial resolution but is prone to temporal artifacts. Spatial interlacing delivers even pixel rows to one eye and odd rows to the other eye simultaneously; it is subject to spatial limitations such as reduced spatial resolution. We propose a spatiotemporal-interlacing protocol that interlaces the left- and right-eye views spatially, but with the rows being delivered to each eye alternating with each frame. We performed psychophysical experiments and found that flicker, motion artifacts, and depth distortion are substantially reduced relative to the temporal-interlacing protocol, and spatial resolution is better than in the spatial-interlacing protocol. Thus, the spatiotemporal-interlacing protocol retains the benefits of spatial and temporal interlacing while minimizing or even eliminating the drawbacks.

Humans perceive flicker artifacts at 500 Hz

Humans perceive a stable average intensity image without flicker artifacts when a television or monitor updates at a sufficiently fast rate. This rate, known as the critical flicker fusion rate, has been studied for both spatially uniform lights, and spatio-temporal displays. These studies have included both stabilized and unstablized retinal images, and report the maximum observable rate as 50–90 Hz. A separate line of research has reported that fast eye movements known as saccades allow simple modulated LEDs to be observed at very high rates. Here we show that humans perceive visual flicker artifacts at rates over 500 Hz when a display includes high frequency spatial edges. This rate is many times higher than previously reported. As a result, modern display designs which use complex spatio-temporal coding need to update much faster than conventional TVs, which traditionally presented a simple sequence of natural images.

Stereoscopic 3D display with color interlacing improves perceived depth

Temporal interlacing is a method for presenting stereoscopic 3D content whereby the two eyes' views are presented at different times and optical filtering selectively delivers the appropriate view to each eye. This approach is prone to distortions in perceived depth because the visual system can interpret the temporal delay between binocular views as spatial disparity. We propose a novel color-interlacing display protocol that reverses the order of binocular presentation for the green primary but maintains the order for the red and blue primaries: During the first sub-frame, the left eye sees the green component of the left-eye view and the right eye sees the red and blue components of the right-eye view, and vice versa during the second sub-frame. The proposed method distributes the luminance of each eye's view more evenly over time. Because disparity estimation is based primarily on luminance information, a more even distribution of luminance over time should reduce depth distortion. We conducted a psychophysical experiment to test these expectations and indeed found that less depth distortion occurs with color interlacing than temporal interlacing.

The visibility of color breakup and a means to reduce it

Color breakup is an artifact seen on displays that present colors sequentially. When the eye tracks a moving object on such a display, different colors land on different places on the retina, and this gives rise to visible color fringes at the object's leading and trailing edges. Interestingly, color breakup is also observed when the eye is stationary and an object moves by. Using a novel psychophysical procedure, we measured breakup both when viewers tracked and did not track a moving object. Breakup was somewhat more visible in the tracking than in the non-tracking condition. The video frames contained three subframes, one each for red, green, and blue. We spatially offset the green and blue stimuli in the second and third subframes, respectively, to find the values that minimized breakup. In the tracking and non-tracking conditions, spatial offsets of Δx/3 in the second subframe (where Δx is the displacement of the object in one frame) and 2Δx/3 in the third eliminated breakup. Thus, this method offers a way to minimize or even eliminate breakup whether the viewer is tracking or not. We suggest ways to implement the method with real video content. We also developed a color-breakup model based on spatiotemporal filtering in color-opponent pathways in early vision. We found close agreement between the model's predictions and the experimental results. The model can be used to predict breakup for a wide variety of conditions.

User experience while viewing stereoscopic 3D television

3D display technologies have been linked to visual discomfort and fatigue. In a lab-based study with a between-subjects design, 433 viewers aged from 4 to 82 years watched the same movie in either 2D or stereo 3D (S3D), and subjectively reported on a range of aspects of their viewing experience. Our results suggest that a minority of viewers, around 14%, experience adverse effects due to viewing S3D, mainly headache and eyestrain. A control experiment where participants viewed 2D content through 3D glasses suggests that around 8% may report adverse effects which are not due directly to viewing S3D, but instead are due to the glasses or to negative preconceptions about S3D (the 'nocebo effect'). Women were slightly more likely than men to report adverse effects with S3D. We could not detect any link between pre-existing eye conditions or low stereoacuity and the likelihood of experiencing adverse effects with S3D.

Effect of interlacing methods of stereoscopic displays on perceived image quality

To provide two-eyed views with one device, stereoscopic 3D (S3D) displays interlace the two views either temporally or spatially: temporal interlacing (TI) alternates the two views in time with full resolution, while spatial interlacing (SI) presents the two views simultaneously but with half resolution for each eye. We investigate the effect of interlacing methods on image quality through a psychophysical experiment. We compared four experimental conditions: three S3D interlacing methods (TI, SI with raw sampling, and SI with vertical interpolation), and one nonconventional interlacing method (vertical interpolation). The stimuli were 10 natural stereo images presented at nine levels of pixel sizes (0.64, 0.78, 0.89, 1.00, 1.28, 1.55, 1.78, 2.00, and 2.56 arcmin). To test the effect of interlacing methods per se, we provided all the experimental conditions to the subjects using a single experimental setup: a mirror stereoscope. The results show that TI does not degrade the image quality for any pixel size. SI degrades the image quality when the pixel size is relatively large, but the effect of the two SI methods does not differ significantly. Comparison of SI methods against the vertical interpolation method implies that the primary cause of the degradation in image quality for SI methods is the visibility of the interlacing pattern rather than the loss of high-frequency information.

Stereoscopy and the Human Visual System

Stereoscopic displays have become important for many applications, including operation of remote devices, medical imaging, surgery, scientific visualization, and computer-assisted design. But the most significant and exciting development is the incorporation of stereo technology into entertainment: specifically, cinema, television, and video games. In these applications for stereo, three-dimensional (3D) imagery should create a faithful impression of the 3D structure of the scene being portrayed. In addition, the viewer should be comfortable and not leave the experience with eye fatigue or a headache. Finally, the presentation of the stereo images should not create temporal artifacts like flicker or motion judder. This paper reviews current research on stereo human vision and how it informs us about how best to create and present stereo 3D imagery. The paper is divided into four parts: (1) getting the geometry right, (2) depth cue interactions in stereo 3D media, (3) focusing and fixating on stereo images, and (4) how temporal presentation protocols affect flicker, motion artifacts, and depth distortion.

Visual Discomfort with Stereo 3D Displays when the Head is Not Upright

Properly constructed stereoscopic images are aligned vertically on the display screen, so on-screen binocular disparities are strictly horizontal. If the viewer's inter-ocular axis is also horizontal, he/she makes horizontal vergence eye movements to fuse the stereoscopic image. However, if the viewer's head is rolled to the side, the on-screen disparities now have horizontal and vertical components at the eyes. Thus, the viewer must make horizontal and vertical vergence movements to binocularly fuse the two images. Vertical vergence movements occur naturally, but they are usually quite small. Much larger movements are required when viewing stereoscopic images with the head rotated to the side. We asked whether the vertical vergence eye movements required to fuse stereoscopic images when the head is rolled cause visual discomfort. We also asked whether the ability to see stereoscopic depth is compromised with head roll. To answer these questions, we conducted behavioral experiments in which we simulated head roll by rotating the stereo display clockwise or counter-clockwise while the viewer's head remained upright relative to gravity. While viewing the stimulus, subjects performed a psychophysical task. Visual discomfort increased significantly with the amount of stimulus roll and with the magnitude of on-screen horizontal disparity. The ability to perceive stereoscopic depth also declined with increasing roll and on-screen disparity. The magnitude of both effects was proportional to the magnitude of the induced vertical disparity. We conclude that head roll is a significant cause of viewer discomfort and that it also adversely affects the perception of depth from stereoscopic displays.