High-resolution far-field integral-imaging camera by double snapshot

Dual convolutional neural network for aberration pre-correction and image quality enhancement in integral imaging display

This paper proposes a method that utilizes a dual neural network model to address the challenges posed by aberration in the integral imaging microlens array (MLA) and the degradation of 3D image quality. The approach involves a cascaded dual convolutional neural network (CNN) model designed to handle aberration pre-correction and image quality restoration tasks. By training these models end-to-end, the MLA aberration is corrected effectively and the image quality of integral imaging is enhanced. The feasibility of the proposed method is validated through simulations and optical experiments, using an optimized, high-quality pre-corrected element image array (EIA) as the image source for 3D display. The proposed method achieves high-quality integral imaging 3D display by alleviating the contradiction between MLA aberration and 3D image resolution reduction caused by system noise without introducing additional complexity to the display system.

Neural network based 3D tracking with a graphene transparent focal stack imaging system

Recent years have seen the rapid growth of new approaches to optical imaging, with an emphasis on extracting three-dimensional (3D) information from what is normally a two-dimensional (2D) image capture. Perhaps most importantly, the rise of computational imaging enables both new physical layouts of optical components and new algorithms to be implemented. This paper concerns the convergence of two advances: the development of a transparent focal stack imaging system using graphene photodetector arrays, and the rapid expansion of the capabilities of machine learning including the development of powerful neural networks. This paper demonstrates 3D tracking of point-like objects with multilayer feedforward neural networks and the extension to tracking positions of multi-point objects. Computer simulations further demonstrate how this optical system can track extended objects in 3D, highlighting the promise of combining nanophotonic devices, new optical system designs, and machine learning for new frontiers in 3D imaging.

Transparent photodetectors based on graphene stacked vertically along the optical axis have shown promising potential for light field reconstruction. Here, the authors develop transparent photodetector arrays and implement a neural network for real-time 3D optical imaging and object tracking.

Enhanced resolution of holographic stereograms by moving or diffusing a virtual pinhole array

In this paper, two different display modes, the "pinhole mode" and the "lens mode" of the pinhole-type integral imaging (PII) based hologram are demonstrated by proper use of random phase. The performances of resolution, fill factor and image depth, of the two display modes are analyzed. Two different methods, the moving array lenslet technique (MALT) and the high-resolution elemental image array (EIA) encoding are introduced for the spatial resolution enhancement of the two display modes, respectively. Both methods enhance the spatial resolution without increasing the total pixel number or the space-bandwidth product (SBP) of the hologram. Both simulation and optical experiments verify that the proposed methods enhance the spatial resolution of PII-based hologram at a very low cost.

Integral imaging-based 2D/3D convertible display system by using holographic optical element and polymer dispersed liquid crystal

An integral imaging-based 2D/3D convertible display system is proposed by using a lens-array holographic optical element (LAHOE), a polymer dispersed liquid crystal (PDLC) film, and a projector. The LAHOE is closely attached to the PDLC film to constitute a projection screen. The LAHOE is used to realize integral imaging 3D display. When the PDLC film with an applied voltage is in the transparent state, the projector projects a Bragg matched 3D image, and the display system works in 3D mode. When the PDLC film without an applied voltage is in the scattering state, the projector projects a 2D image, and the display system works in 2D mode. A prototype of the integral imaging-based 2D/3D convertible display is developed, and it provides 2D/3D convertible images properly.

Three-dimensional imaging system with both improved lateral resolution and depth of field considering non-uniform system parameters

In this paper, we propose a new 3D passive image sensing and visualization technique to improve lateral resolution and depth of field (DoF) of integral imaging simultaneously. There is a resolution trade-off between lateral resolution and DoF in integral imaging. To overcome this issue, a large aperture and a small aperture can be used to record the elemental images to reduce the diffraction effect and extend the DoF, respectively. Therefore, in this paper, we utilize these two pickup concepts with a non-uniform camera array. To show the feasibility of our proposed method, we implement an optical experiment. For comparison in details, we calculate the peak signal-to-noise ratio (PSNR) as the performance metric.

Unconventional methods of imaging: computational microscopy and compact implementations

In the past two decades or so, there has been a renaissance of optical microscopy research and development. Much work has been done in an effort to improve the resolution and sensitivity of microscopes, while at the same time to introduce new imaging modalities, and make existing imaging systems more efficient and more accessible. In this review, we look at two particular aspects of this renaissance: computational imaging techniques and compact imaging platforms. In many cases, these aspects go hand-in-hand because the use of computational techniques can simplify the demands placed on optical hardware in obtaining a desired imaging performance. In the first main section, we cover lens-based computational imaging, in particular, light-field microscopy, structured illumination, synthetic aperture, Fourier ptychography, and compressive imaging. In the second main section, we review lensfree holographic on-chip imaging, including how images are reconstructed, phase recovery techniques, and integration with smart substrates for more advanced imaging tasks. In the third main section we describe how these and other microscopy modalities have been implemented in compact and field-portable devices, often based around smartphones. Finally, we conclude with some comments about opportunities and demand for better results, and where we believe the field is heading.

Computational photography with plenoptic camera and light field capture: tutorial

Photography is a cornerstone of imaging. Ever since cameras became consumer products more than a century ago, we have witnessed great technological progress in optics and recording mediums, with digital sensors replacing photographic films in most instances. The latest revolution is computational photography, which seeks to make image reconstruction computation an integral part of the image formation process; in this way, there can be new capabilities or better performance in the overall imaging system. A leading effort in this area is called the plenoptic camera, which aims at capturing the light field of an object; proper reconstruction algorithms can then adjust the focus after the image capture. In this tutorial paper, we first illustrate the concept of plenoptic function and light field from the perspective of geometric optics. This is followed by a discussion on early attempts and recent advances in the construction of the plenoptic camera. We will then describe the imaging model and computational algorithms that can reconstruct images at different focus points, using mathematical tools from ray optics and Fourier optics. Last, but not least, we will consider the trade-off in spatial resolution and highlight some research work to increase the spatial resolution of the resulting images.

Resolution enhancement in integral microscopy by physical interpolation

Integral-imaging technology has demonstrated its capability for computing depth images from the microimages recorded after a single shot. This capability has been shown in macroscopic imaging and also in microscopy. Despite the possibility of refocusing different planes from one snap-shot is crucial for the study of some biological processes, the main drawback in integral imaging is the substantial reduction of the spatial resolution. In this contribution we report a technique, which permits to increase the two-dimensional spatial resolution of the computed depth images in integral microscopy by a factor of √2. This is made by a double-shot approach, carried out by means of a rotating glass plate, which shifts the microimages in the sensor plane. We experimentally validate the resolution enhancement as well as we show the benefit of applying the technique to biological specimens.

Superresolution of 3-D computational integral imaging based on moving least square method

In this paper, we propose an edge directive moving least square (ED-MLS) based superresolution method for computational integral imaging reconstruction(CIIR). Due to the low resolution of the elemental images and the alignment error of the microlenses, it is not easy to obtain an accurate registration result in integral imaging, which makes it difficult to apply superresolution to the CIIR application. To overcome this problem, we propose the edge directive moving least square (ED-MLS) based superresolution method which utilizes the properties of the moving least square. The proposed ED-MLS based superresolution takes the direction of the edge into account in the moving least square reconstruction to deal with the abrupt brightness changes in the edge regions, and is less sensitive to the registration error. Furthermore, we propose a framework which shows how the data have to be collected for the superresolution problem in the CIIR application. Experimental results verify that the resolution of the elemental images is enhanced, and that a high resolution reconstructed 3-D image can be obtained with the proposed method.

Light ray field capture using focal plane sweeping and its optical reconstruction using 3D displays

We propose a method to capture light ray field of three-dimensional scene using focal plane sweeping. Multiple images are captured using a usual camera at different focal distances, spanning the three-dimensional scene. The captured images are then back-projected to four-dimensional spatio-angular space to obtain the light ray field. The obtained light ray field can be visualized either using digital processing or optical reconstruction using various three-dimensional display techniques including integral imaging, layered display, and holography.

Feasibility study for pseudoscopic problem in integral imaging using negative refractive index materials

To solve the pseudoscopic problem, we propose a one-step integral imaging system with negative refractive index materials, which can avoid the deterioration in resolution inherent to the optical or digital two-step processes. Specifically, the proposed method is based on the novel feature of negative refractive index materials, bending light to a negative angle relative to the surface normal. The pseudoscopic imaging property of the negative refractive index material slab is theoretically investigated. For formation of orthoscopic reconstructed images, the matching condition of the negative index lens array and the positive index lens array is deduced. Two types of conceptual prototypes of integral imaging system with negative refractive index materials are designed. Experimental results show the validity of the proposed method. To the best of our knowledge, this is the first time to explore the application of negative index materials in eliminating the pseudoscopic effect in integral imaging.

Three-dimensional display by smart pseudoscopic-to-orthoscopic conversion with tunable focus

The original aim of the integral-imaging concept, reported by Gabriel Lippmann more than a century ago, is the capture of images of 3D scenes for their projection onto an autostereoscopic display. In this paper we report a new algorithm for the efficient generation of microimages for their direct projection onto an integral-imaging monitor. Like our previous algorithm, the smart pseudoscopic-to-orthoscopic conversion (SPOC) algorithm, this algorithm produces microimages ready to produce 3D display with full parallax. However, this new algorithm is much simpler than the previous one, produces microimages free of black pixels, and permits fixing at will, between certain limits, the reference plane and the field of view of the displayed 3D scene. Proofs of concept are illustrated with 3D capture and 3D display experiments.

Analysis of the depth of field of integral imaging displays based on wave optics

In this paper, we analyze the depth of field (DOF) of integral imaging displays based on wave optics. With considering the diffraction effect, we analyze the intensity distribution of light with multiple micro-lenses and derive a DOF calculation formula for integral imaging display system. We study the variations of DOF values with different system parameters. Experimental results are provided to verify the accuracy of the theoretical analysis. The analyses and experimental results presented in this paper could be beneficial for better understanding and designing of integral imaging displays.

Hologram synthesis of three-dimensional real objects using portable integral imaging camera

We propose a portable hologram capture system based on integral imaging. An integral imaging camera with an integrated micro lens array captures spatio-angular light ray distribution of the three-dimensional scene under incoherent illumination. The captured light ray distribution is then processed to synthesize corresponding hologram. Experimental results show that the synthesized hologram is optically reconstructed successfully, demonstrating accommodation and motion parallax of the reconstructed three-dimensional scene.

Axially moving a lenslet array for high-resolution 3D images in computational integral imaging

This paper presents a new high-resolution computational integral imaging system employing a pickup with the axial movement of a lenslet array and a computation reconstruction algorithm with pixel-to-pixel mapping. In the proposed method, a lenslet array and its image sensor are moved together along the z-axis direction (or axial direction) and a series of elemental image arrays are obtained while moving. The elemental image arrays are then applied to pixel-to-pixel mapping without interpolation for the reconstruction of 3D slice images. Also, an analysis of the proposed reconstruction method is provided. To show the usefulness of the proposed method, experiments are conducted. The results indicate that the proposed method is superior to the existing method such as MALT in terms of image quality.