Foveated light-field display and real-time rendering for virtual reality

Near eye display based on multiplexed retinal projections for robust compensation of eye pupil variance

Among various specifications of near eye display (NED) devices, a compact formfactor is essential for comfortable user experience but also the hardest one to accomplish due to the slowest progresses. A pinhole/pinlight array based light-field (LF) technique is considered as one of the candidates to achieve that goal without thicker and heavier refractive optics. Despite those promising advantages, however, there are critical issues, such as dark spots and contrast distortion, which degrade the image quality because of the vulnerability of the LF retinal image when the observer's eye pupil size changes. Regardless of previous attempts to overcome those artifacts, it was impossible to resolve both issues due to their trade-off relation. In this paper, in order to resolve them simultaneously, we propose a concept of multiplexed retinal projections to integrate the LF retinal image through rotating transitions of refined and modulated elemental images for robust compensation of eye pupil variance with improved conservation of contrast distribution. Experimental demonstrations and quantitative analysis are also provided to verify the principle.

High-brightness hybrid compressive light field display with improved image quality

Previous LCD-based multiplicative compressive light field (CLF) display has the trade-off between the brightness and the depth of field (DOF). In this paper, we propose a hybrid CLF display using a reflective polarizer and RGB mini-LED panel. By the polarization-multiplexing and the reflector dam (RD) designed on the mini-LED panel, the proposed system can preserve high brightness while enhancing the DOF. Then, a decomposition algorithm is proposed to improve the image quality by depth segmentation and limiting the motion parallax. Compared to the conventional hybrid CLF display, the brightness of the proposed system reaches 348 nits and the reconstruction quality achieves structural similarity index measure (SSIM) improvement by 0.12. The experiments also demonstrate that the proposed method could achieve a higher brightness, larger depth of field, and higher image quality.

Perceptually-guided Dual-mode Virtual Reality System For Motion-adaptive Display

An ideal Virtual reality (VR) device should simultaneously provide retina-level resolution, wide field-of-view (FOV), and high refresh rate display, thereby bringing users into a deeply immersive virtual world. However, directly providing such high-quality display poses great challenges for display panel fabrication, real-time rendering, and data transfer. To address this issue, we introduce a dual-mode virtual reality system based on the spatio-temporal perception characteristics of human vision. The proposed VR system has a novel optical architecture. It can switch display modes according to the user's perceptual requirements for different display scenes to adaptively adjust the display spatial and temporal resolution based on a given display budget, thus providing users with the optimal visual perception quality. In this work, a complete design pipeline for the dual-mode VR optical system is proposed, and a bench-top prototype is built with only off-the-shelf hardware and components to verify its capability. Compared to the conventional VR system, our proposed scheme is more efficient and flexible in utilizing the display budget, and this work is expected to facilitate the development of the VR device based on the human visual system.

Super multi-view display based on near-eye pinholes

Vergence-accommodation-conflict-free super multi-view (SMV) display based on more than one near-eye pinhole group for each pupil of the viewer is developed in this paper. Two-dimensionally arranged pinholes of a group correspond to different subscreens of the display screen, with perspective views projected by each subscreen through a corresponding pinhole splicing into an image of enlarged field of view (FOV). By sequentially switching on/off different pinhole groups, more than one mosaic image is projected to each pupil of the viewer. Adjacent pinholes of a group are endowed with different timing-polarizing characteristics to generate an effective noise-free region for each pupil. In the experiment, four groups of 3×3 pinholes are configured for a proof-of-concept SMV display on a 240 Hz display screen, with a diagonal FOV of 55 deg and a depth of field reaching 1.2 m.

Super multi-view near-eye virtual reality with directional backlights from wave-guides

Directional backlights have often been employed for generating multiple view-zones in three-dimensional (3D) display, with each backlight converging into a corresponding view-zone. By designing the view-zone interval for each pupil smaller than the pupil's diameter, super multi-view (SMV) can get implemented for a VAC-free 3D display. However, expanding the backlight from a light-source to cover the corresponding display panel often needs an extra thickness, which results in a thicker structure and is unwanted by a near-eye display. In this paper, two wave-guides are introduced into a near-eye virtual reality (NEVR) system, for sequentially guiding more than one directional backlight to each display panel for SMV display without bringing obvious extra thickness. A prototype SMV NEVR gets demonstrated, with two backlights from each wave-guide converging into two view-zones for a corresponding pupil. Although the additional configured light-sources are positioned far from the corresponding wave-guide in our proof-of-concept prototype, multiple light-sources can be attached to the corresponding wave-guide compactly if necessary. As proof, a 3D scene with defocus-blur effects gets displayed. The design range of the backlights' total reflection angles in the wave-guide is also discussed.

Large-scale metagrating complex-based light field 3D display with space-variant resolution for non-uniform distribution of information and energy

Glasses-free three-dimensional (3D) display has attracted wide interest for providing stereoscopic virtual contents with depth cues. However, how to achieve high spatial and angular resolution while keeping ultrawide field of view (FOV) remains a significant challenge in 3D display. Here, we propose a light field 3D display with space-variant resolution for non-uniform distribution of information and energy. The spatial resolution of each view is modulated according to watching habit. A large-scale combination of pixelated 1D and 2D metagratings is used to manipulate dot and horizontal line views. With the joint modulation of pixel density and view arrangement, the information density and illuminance of high-demand views are at most 5.6 times and 16 times that of low-demand views, respectively. Furthermore, a full-color and video rate light field 3D display with non-uniform information distribution is demonstrated. The prototype provides 3D images with a high spatial resolution of 119.6 pixels per inch and a high angular resolution of 0.25 views per degree in the high-demand views. An ultrawide viewing angle of 140° is also provided. The proposed light field 3D display does not require ultrahigh-resolution display panels and has form factors of thin and light. Thus, it has the potential to be used in portable electronics, window display, exhibition display, as well as tabletop display.