Depth-layer weighted prediction method for a full-color polygon-based holographic system with real objects

End-to-end real-time holographic display based on real-time capture of real scenes

Holographic display is considered as a promising three-dimensional (3D) display technology and has been widely studied. However, to date, the real-time holographic display for real scenes is still far from being incorporated in our life. The speed and quality of information extraction and holographic computing need to be further improved. In this paper, we propose an end-to-end real-time holographic display based on real-time capture of real scenes, where the parallax images are collected from the scene and a convolutional neural network (CNN) builds the mapping from the parallax images to the hologram. Parallax images are acquired in real time by a binocular camera, and contain depth information and amplitude information needed for 3D hologram calculation. The CNN, which can transform parallax images into 3D holograms, is trained by datasets consisting of parallax images and high-quality 3D holograms. The static colorful reconstruction and speckle-free real-time holographic display based on real-time capture of real scenes have been verified by the optical experiments. With simple system composition and affordable hardware requirements, the proposed technique will break the dilemma of the existing real-scene holographic display, and open up a new direction for the application of real-scene holographic 3D display such as holographic live video and solving vergence-accommodation conflict (VAC) problems for head-mounted display devices.

Implementation of a full-color holographic system using RGB-D salient object detection and divided point cloud gridding

At present, a real objects-based full-color holographic system usually uses a digital single-lens reflex (DSLR) camera array or depth camera to collect data. It then relies on a spatial light modulator to modulate the input light source for the reconstruction of the 3-D scene of the real objects. However, the main challenges the high-quality holographic 3-D display faced were the limitation of generation speed and the low accuracy of the computer-generated holograms. This research generates more effective and accurate point cloud data by developing an RGB-D salient object detection model in the acquisition unit. In addition, a divided point cloud gridding method is proposed to enhance the computing speed of hologram generation. In the RGB channels, we categorized each object point into depth grids with identical depth values. The depth girds are divided into M × N parts, and only the effective parts will be calculated. Compared with traditional methods, the calculation time is dramatically reduced. The feasibility of our proposed approach is established through experiments.

Binocular full-color holographic three-dimensional near eye display using a single SLM

A binocular full-color holographic three-dimensional near eye display system using a single spatial light modulator (SLM) is proposed. In the display system, the frequency spectrum shifting operation and color spectrum shifting operation are adopted to realize the frequency division multiplexing (FDM) and frequency superposition multiplexing (FSM) by manipulating the frequency spectrums of each color- and view-channel sub-holograms. The FDM combined with polarization multiplexing will be used to implement binocular display using a single SLM, and the FSM working with a bandpass filter for each view-channel will be used to achieve full-color display from single frame hologram. The optical analysis and experiments with 3D color objects confirm the feasibility of the proposed system in the practical application.

Three-dimensional computer holography enabled from a single 2D image

To compute a high-quality computer-generated hologram (CGH) for true 3D real scenes, a huge amount of 3D data must be physically acquired and provided depending on specific devices or 3D rendering techniques. Here, we propose a computational framework for generating a CGH from a single image based on the idea of 2D-to-3D wavefront conversion. We devise a deep view synthesis neural network to synthesize light-field contents from a single image and convert the light-field data to the diffractive wavefront of the hologram using a ray-wave algorithm. The method is able to achieve extremely straightforward 3D CGH generation from hand-accessible 2D image content and outperforms existing real-world-based CGH computation, which inevitably relies on a high-cost depth camera and cumbersome 3D data rendering. We experimentally demonstrate 3D reconstructions of indoor and outdoor scenes from a single image enabled phase-only CGH.

Color curved hologram calculation method based on angle multiplexing

In this paper, a method of color curved hologram calculation based on angle multiplexing is proposed. The relationship between the wavelength, center angle and sampling interval of the curved holograms is analyzed for the first time by analyzing the reconstruction process of the curved holograms with different wavelengths. Based on this relationship, the color curved holograms are calculated by compensating phase to the complex amplitude distribution of the planar holograms. To eliminate the chromatic aberration, the curved holograms of different objects with the same color channel are respectively used for angle multiplexing and phase compensation, and then the color composed curved hologram is generated. Different color objects without chromatic aberration can be reconstructed by bending the composed curved hologram into different central angles. The experimental results verify the feasibility of the proposed method. Besides, the application of the proposed method in augmented reality display is also shown.

Fast calculation of high-definition depth-added computer-generated holographic stereogram by spectrum-domain look-up table [Invited]

High-definition depth-added computer-generated holographic stereogram (DA-CGHS) is superior in its high quality, easy realization, and auto-shading effect. However, its computing cost is extremely high because numerous scenes together with depth information must be calculated. Here, we proposed a fast calculation scheme of DA-CGHS by the spectrum-domain look-up table (SDLUT) method. In SDLUT, diffraction fields on the hogel plane of selected reference points in the object space are calculated. Subsequently, the fields are Fourier transformed to the spectrum domain. Because the signal energy always concentrates in a small spectrum region, these regions are cropped as the elemental tables. In the computing of the hogels, the field superposition is conducted in the spectrum domain by using the elemental tables. In our demonstration, the table size of SDLUT is only 0.44% that of the look-up table (LUT). Because the table size is very small, the computing time of SDLUT method can be nearly 80 times faster than that of conventional LUTs in the spatial domain, while the image quality is comparable.

Implementation of full-color holographic system using non-uniformly sampled 2D images and compressed point cloud gridding

A multiple-camera holographic system using non-uniformly sampled 2D images and compressed point cloud gridding (C-PCG) is suggested. High-quality, digital single-lens reflex cameras are used to acquire the depth and color information from real scenes; these are then virtually reconstructed by the uniform point cloud using a non-uniform sampling method. The C-PCG method is proposed to generate efficient depth grids by classifying groups of object points with the same depth values in the red, green, and blue channels. Holograms are obtained by applying fast Fourier transform diffraction calculations to the grids. Compared to wave-front recording plane methods, the quality of the reconstructed images is substantially better, and the computational complexity is dramatically reduced. The feasibility of our method is confirmed both numerically and optically.

Progress in virtual reality and augmented reality based on holographic display

The past, present, and future industry prospects of virtual reality (VR) and augmented reality (AR) are presented. The future of VR/AR technology based on holographic display is predicted by analogy with the VR/AR based on binocular vision display and light field display. The investigations on holographic display that can be used in VR/AR are reviewed. The breakthroughs of holographic display are promising in VR/AR with high resolution. The challenges faced by VR/AR based on holographic display are analyzed.

3D Imaging Based on Depth Measurement Technologies

Three-dimensional (3D) imaging has attracted more and more interest because of its widespread applications, especially in information and life science. These techniques can be broadly divided into two types: ray-based and wavefront-based 3D imaging. Issues such as imaging quality and system complexity of these techniques limit the applications significantly, and therefore many investigations have focused on 3D imaging from depth measurements. This paper presents an overview of 3D imaging from depth measurements, and provides a summary of the connection between the ray-based and wavefront-based 3D imaging techniques.

Quality enhancement and GPU acceleration for a full-color holographic system using a relocated point cloud gridding method

The calculation of realistic full-color holographic displays is hindered by the high computational cost. Previously, we suggested a point cloud gridding (PCG) method to calculate monochrome holograms of real objects. In this research, a relocated point cloud gridding (R-PCG) method is proposed to enhance the reconstruction quality and accelerate the calculation speed in GPU for a full-color holographic system. We use a depth camera to acquire depth and color information from the real scene then reconstruct the point cloud model virtually. The R-PCG method allows us to classify groups of object points with the same depth values into grids in the red, green, and blue (RGB) channels. Computer-generated holograms (CGHs) are obtained by applying a fast Fourier transform (FFT) diffraction calculation to the grids. The feasibility of the R-PCG method is confirmed by numerical and optical reconstruction.