Parallel multi-view polygon rasterization for 3D light field display

Real-time light-field generation based on the visual hull for the 3D light-field display with free-viewpoint texture mapping

Real-time dense view synthesis based on three-dimensional (3D) reconstruction of real scenes is still a challenge for 3D light-field display. It's time-consuming to reconstruct an entire model, and then the target views are synthesized afterward based on volume rendering. To address this issue, Light-field Visual Hull (LVH) is presented with free-viewpoint texture mapping for 3D light-field display, which can directly produce synthetic images with the 3D reconstruction of real scenes in real-time based on forty free-viewpoint RGB cameras. An end-to-end subpixel calculation procedure of the synthetic image is demonstrated, which defines a rendering ray for each subpixel based on light-field image coding. In the ray propagation process, only the essential spatial point of the target model is located for the corresponding subpixel by projecting the frontmost point of the ray to all the free-viewpoints, and the color of each subpixel is identified in one pass. A dynamic free-viewpoint texture mapping method is proposed to solve the correct graphic texture considering the free-viewpoint cameras. To improve the efficiency, only the visible 3D position and texture that contributes to the synthetic image are calculated based on backward ray tracing rather than computing the entire 3D model and generating all elemental images. In addition, an incremental calibration method by dividing camera groups is proposed to satisfy the accuracy. Experimental results show the validity of our method. All the rendered views are analyzed for justifying the texture mapping method, and the PSNR is improved by an average of 11.88dB. Finally, LVH can achieve a natural and smooth viewing effect at 4K resolution and the frame rate of 25 ∼ 30fps with a large viewing angle.

Real-time dense-view imaging for three-dimensional light-field display based on image color calibration and self-supervised view synthesis

Three-Dimensional (3D) light-field display has achieved promising improvement in recent years. However, since the dense-view images cannot be collected fast in real-world 3D scenes, the real-time 3D light-field display is still challenging to achieve in real scenes, especially at the high-resolution 3D display. Here, a real-time 3D light-field display method with dense-view is proposed based on image color correction and self-supervised optical flow estimation, and a high-quality and high frame rate of 3D light-field display can be realized simultaneously. A sparse camera array is firstly used to capture sparse-view images in the proposed method. To eliminate the color deviation of the sparse views, the imaging process of the camera is analyzed, and a practical multi-layer perception (MLP) network is proposed to perform color calibration. Given sparse views with consistent color, the optical flow can be estimated by a lightweight convolutional neural network (CNN) at high speed, which uses the input image pairs to learn the optical flow in a self-supervised manner. With inverse warp operation, dense-view images can be synthesized in the end. Quantitative and qualitative experiments are performed to evaluate the feasibility of the proposed method. Experimental results show that over 60 dense-view images at a resolution of 1024 × 512 can be generated with 11 input views at a frame rate over 20 fps, which is 4× faster than previous optical flow estimation methods PWC-Net and LiteFlowNet3. Finally, large viewing angles and high-quality 3D light-field display at 3840 × 2160 resolution can be achieved in real-time.

Research on Estimation Method of Geometric Features of Structured Negative Obstacle Based on Single-Frame 3D Laser Point Cloud

A single VLP-16 LiDAR estimation method based on a single-frame 3D laser point cloud is proposed to address the problem of estimating negative obstacles’ geometrical features in structured environments. Firstly, a distance measurement method is developed to determine the estimation range of the negative obstacle, which can be used to verify the accuracy of distance estimation. Secondly, the 3D point cloud of a negative obstacle is transformed into a 2D elevation raster image, making the detection and estimation of negative obstacles more intuitive and accurate. Thirdly, we compare the effects of a StatisticalOutlierRemoval filter, RadiusOutlier removal, and Conditional removal on 3D point clouds, and the effects of a Gauss filter, Median filter, and Aver filter on 2D image denoising, and design a flowchart for point cloud and image noise reduction and denoising. Finally, a geometrical feature estimation method is proposed based on the elevation raster image. The negative obstacle image in the raster is used as an auxiliary line, and the number of pixels is derived from the OpenCV-based Progressive Probabilistic Hough Transform to estimate the geometrical features of the negative obstacle based on the raster size. The experimental results show that the algorithm has high accuracy in estimating the geometric characteristics of negative obstacles on structured roads and has a practical application value for LiDAR environment perception research.

Focus issue introduction: 3D image acquisition and display: technology, perception, and applications

This feature issue of Optics Express is organized in conjunction with the 2020 OSA conference on 3D image acquisition and display: technology, perception and applications which was held virtually in Vancouver from 22 to 26, June 2020 as part of the imaging and sensing congress 2020. This feature issue presents 29 articles based on the topics and scope of the 3D conference. This review provides a summary of these articles.