Review of computer-generated hologram algorithms for color dynamic holographic three-dimensional display

Solution to the issue of high-order diffraction images for cylindrical computer-generated holograms

The cylindrical computer-generated hologram (CCGH), featuring a 360° viewing zone, has garnered widespread attention. However, the issue of high-order diffraction images due to pixelated structure in CCGH has not been previously reported and solved. For a cylindrical model offering a 360° viewing zone in the horizontal direction, the high-order diffraction images always overlap with the reconstruction image, leading to quality degradation. Furthermore, the 4f system is commonly used to eliminate high-order diffraction images in planar CGH, but its implementation is predictably complex for a cylindrical model. In this paper, we propose a solution to the issue of high-order diffraction images for CCGH. We derive the cylindrical diffraction formula from the outer hologram surface to the inner object surface in the spectral domain, and based on this, we subsequently analyze the effects brought by the pixel structure and propose the high-order diffraction model. Based on the proposed high-order diffraction model, we use the gradient descent method to optimize CCGH accounting for all diffraction orders simultaneously. Furthermore, we discuss the issue of circular convolution due to the periodicity of the Fast Fourier Transform (FFT) in cylindrical diffraction. The correctness of the proposed high-order diffraction model and the effectiveness of the proposed optimization method are demonstrated by numerical simulation. To our knowledge, this is the first time that the issue of high-order diffraction images in CCGH has been proposed, and we believe our solution can offer valuable guidance to practitioners in the field.

Computer-generated holography with ordinary display

We propose a method of computer-generated holography (CGH) using incoherent light emitted from a mobile phone screen. In this method, we suppose a cascade of holograms in which the first hologram is a color image displayed on the mobile phone screen. The hologram cascade is synthesized by solving an inverse problem with respect to the propagation of incoherent light. We demonstrate a three-dimensional color image reproduction using a two-layered hologram cascade composed of an iPhone and a spatial light modulator.

Speckle reduction for single sideband-encoded computer-generated holograms by using an optimized carrier wave

Computer-generated hologram (CGH) is an evolving field that facilitates three-dimensional displays, with speckle noise reduction being a pivotal challenge. In hologram synthesis, complex data with random phase distributions are typically employed as carrier waves for wide viewing angles and a shallow depth of focus (DOF). However, these carrier waves are a source of speckle noise, which can significantly degrade image quality. In this paper, we propose a novel technique for speckle reduction for single sideband (SSB)-encoded holograms, applicable to any arbitrary 3D object. The proposed method focuses on optimizing the random carrier wave used in the hologram synthesis to achieve a uniform amplitude distribution at the object's location. This optimization results in a carrier wave that consistently exhibits uniform amplitude at specific depth planes, leading to a significant reduction of the speckle occurring from the carrier wave. The proposed method has been validated through simulations and optical experiments.

Vision transformer empowered physics-driven deep learning for omnidirectional three-dimensional holography

The inter-plane crosstalk and limited axial resolution are two key points that hinder the performance of three-dimensional (3D) holograms. The state-of-the-art methods rely on increasing the orthogonality of the cross-sections of a 3D object at different depths to lower the impact of inter-plane crosstalk. Such strategy either produces unidirectional 3D hologram or induces speckle noise. Recently, learning-based methods provide a new way to solve this problem. However, most related works rely on convolution neural networks and the reconstructed 3D holograms have limited axial resolution and display quality. In this work, we propose a vision transformer (ViT) empowered physics-driven deep neural network which can realize the generation of omnidirectional 3D holograms. Owing to the global attention mechanism of ViT, our 3D CGH has small inter-plane crosstalk and high axial resolution. We believe our work not only promotes high-quality 3D holographic display, but also opens a new avenue for complex inverse design in photonics.

Adaptive layer-based computer-generated holograms

We propose a novel, to the best of our knowledge, and fast adaptive layer-based (ALB) method for generating a computer-generated hologram (CGH) with accurate depth information. A complex three-dimensional (3D) object is adaptively divided into layers along the depth direction according to its own non-uniformly distributed depth coordinates, which reduces the depth error caused by the conventional layer-based method. Each adaptive layer generates a single-layer hologram using the angular spectrum method for diffraction, and the final hologram of a complex three-dimensional object is obtained by superimposing all the adaptive layer holograms. A hologram derived with the proposed method is referred to as an adaptive layer-based hologram (ALBH). Our demonstration shows that the desired reconstruction can be achieved with 52 adaptive layers in 8.7 s, whereas the conventional method requires 397 layers in 74.9 s.

High quality holographic 3D display with enhanced focus cues based on multiple directional light reconstruction

Holographic display faces the trade-off between image quality and focus cues, resulting from the specific choice of phase distribution. In this Letter, we propose a speckle-free holographic display with enhanced focus cues by multiple directional light reconstruction. The uniform phase hologram is first generated by the gradient descent optimization algorithm. The blazed grating phase is used to steer the object light to a specific direction. Multiple sub-holograms with different blazed gratings are refreshed fast to reconstruct the images from different directions. Thus, the defocus blur is improved due to the separation of multiple spots on the defocus plane. The multi-plane reconstruction is also realized by pre-processing the depth images to eliminate image separation. The proposed method provides apparent focus cues while maintaining high image qualities, which is expected to realize comfortable holographic near-eye display in the near future.

Multiplicative-noise-multiplexing holography with ultrahigh capacity and low cross talk

Optical multiplexing technologies, by utilizing various dimensions of light, can effectively expand the information capacity and density for holography but may also lead to multiplexing cross talk. Here, we propose and demonstrate a novel, to our knowledge, multiplicative-noise-multiplexing holography by utilizing the orthogonality between multiplicative noises as a multiplexing dimension. The results prove that this holography can provide a new multiplexing dimension, significantly enhancing information capacity and effectively lowering cross talk. This promising scheme for ultrahigh-capacity holography has the potential to address the limitations of traditional holographic multiplexing technologies.

Liquid lens based holographic camera for real 3D scene hologram acquisition using end-to-end physical model-driven network

With the development of artificial intelligence, neural network provides unique opportunities for holography, such as high fidelity and dynamic calculation. How to obtain real 3D scene and generate high fidelity hologram in real time is an urgent problem. Here, we propose a liquid lens based holographic camera for real 3D scene hologram acquisition using an end-to-end physical model-driven network (EEPMD-Net). As the core component of the liquid camera, the first 10 mm large aperture electrowetting-based liquid lens is proposed by using specially fabricated solution. The design of the liquid camera ensures that the multi-layers of the real 3D scene can be obtained quickly and with great imaging performance. The EEPMD-Net takes the information of real 3D scene as the input, and uses two new structures of encoder and decoder networks to realize low-noise phase generation. By comparing the intensity information between the reconstructed image after depth fusion and the target scene, the composite loss function is constructed for phase optimization, and the high-fidelity training of hologram with true depth of the 3D scene is realized for the first time. The holographic camera achieves the high-fidelity and fast generation of the hologram of the real 3D scene, and the reconstructed experiment proves that the holographic image has the advantage of low noise. The proposed holographic camera is unique and can be used in 3D display, measurement, encryption and other fields.

Reference-free in situ rapid regional calibration of phase-only spatial light modulators

Spatial light modulators (SLMs) have become an indispensable element in modern optics for their versatile performance in many applications. Among various types of SLMs, such as digital micromirror devices (DMD), liquid crystal-based phase-only spatial light modulators (LC-SLMs), and deformable mirrors (DM), LC-SLMs are often the method of choice due to their high efficiency, precise phase modulation, and abundant number of effective pixels. In general, for research grade applications, an additional SLM calibration step is required due to fabrication imperfection resulting in non-flat liquid crystal panels and varying phase responses over the SLM area. Here, we demonstrate a straightforward approach for reference-free orthogonal calibration of an arbitrary number of SLM subregions which only requires the same measurement time as global calibration. The proposed method requires minimal optical elements and can be applied to any optical setup as is. As a benchmark performance test, we achieved a 2.2-fold enhancement in correction efficiency for wavefront shaping through scattering media utilizing the calibrated 2160 subregions of the SLM, in comparison with a single global look-up table (LUT).

Deep learning denoising diffusion probabilistic model applied to holographic data synthesis

In this Letter, we demonstrate for the first time, to our knowledge, a holographic data synthesis based on a deep learning probabilistic diffusion model (DDPM). Several different datasets of color images corresponding to different types of objects are converted to complex-valued holographic data through backpropagation. Then, we train a DDPM using the resulting holographic datasets. The diffusion model is composed of a noise scheduler, which gradually adds Gaussian noise to each hologram in the dataset, and a U-Net convolutional neural network that is trained to reverse this process. Once the U-Net is trained, any number of holograms with similar features as those of the datasets can be generated just by inputting a Gaussian random noise to the model. We demonstrate the synthesis of holograms containing color images of 2D characters, vehicles, and 3D scenes with different characters at different propagation distances.

PS-NET: an end-to-end phase space depth estimation approach for computer-generated holograms

In the present work, an end-to-end approach is proposed for recovering an RGB-D scene representation directly from a hologram using its phase space representation. The proposed method involves four steps. First, a set of silhouette images is extracted from the hologram phase space representation. Second, a minimal 3D volume that describes these silhouettes is extracted. Third, the extracted 3D volume is decomposed into horizontal slices, and each slice is processed using a neural network to generate a coarse estimation of the scene geometry. Finally, a third neural network is employed to refine the estimation for higher precision applications. Experimental results demonstrate that the proposed approach yields faster and more accurate results compared to numerical reconstruction-based methods. Moreover, the obtained RGB-D representation can be directly utilized for alternative applications such as motion estimation.

High-axial-resolution speckle-free holographic reconstruction via cylindrical quadratic phase method and temporal focusing

Holographic techniques enable precise laser manipulation, but suffer from two considerable limitations: speckle and deterioration of axial distribution. Here, we propose a cylindrical quadratic phase (CQP) method with temporal focusing (TF) to generate speckle-free holographic illumination with high axial resolution. TF-CQP utilizes a superposed cylindrical phase as the initial guess to iteratively optimize phase hologram, realizing speckle-free holographic reconstruction on the target focal plane and eliminating secondary focus on the defocused planes. TF-CQP further disperses defocused beams symmetrically by a blazed grating, placed conjugate to the focal plane, which enhances axial confinement. Simulation and experimental results show that TF-CQP reconstructs speckle-free illumination with arbitrary shapes and <10 µm axial resolution. Compared to TF-GS (Gerchberg-Saxton algorithm), widely used in holographic optogenetics, TF-CQP shows increased uniformity of 200% and improved modulation efficiency of 32.33% for parallel holographic illumination, as well as a 10% increment in axial resolution.

Depth of field expansion method for integral imaging based on diffractive optical element and CNN

In lens-based display systems, lens aberrations and depth of field (DoF) limitation often lead to blurring and distortion of reconstructed images; Meanwhile, expanding the display DoF will face a trade-off between horizontal resolution and axial resolution, restricting the achievement of high-resolution and large DoF three-dimensional (3D) displays. To overcome these constraints and enhance the DoF and resolution of reconstructed scenes, we propose a DoF expansion method based on diffractive optical element (DOE) optimization and image pre-correction through a convolutional neural network (CNN). This method applies DOE instead of the conventional lens and optimizes DOE phase distribution using the Adam algorithm, achieving depth-invariant and concentrated point spread function (PSF) distribution throughout the entire DoF range; Simultaneously, we utilize a CNN to pre-correct the original images and compensate for the image quality reduction introduced by the DOE. The proposed method is applied to a practical integral imaging system, we effectively extend the DoF of the DOE to 400 mm, leading to a high-resolution 3D display in multiple depth planes. To validate the effectiveness and practicality of the proposed method, we conduct numerical simulations and optical experiments.

Three-dimensional computer-generated holography based on the hybrid iterative angular spectrum algorithm

This paper proposes a novel three-dimensional hologram calculation method based on the angular spectrum approach, with the aim of reducing the noise generated during the hologram reconstruction process. The proposed algorithm divides the spatial domain into multiple layers and employs the angular spectrum method to propagate the image between these layers, thus avoiding the paraxial approximation. To enhance the quality of the hologram, an error iteration algorithm is utilized to alleviate the occurrence of aliasing errors when directly superimposing holograms. Moreover, constraint factors are introduced between different layers within the same region to effectively utilize spatial resources for multi-image reconstruction, thereby mitigating the noise caused by inter-layer crosstalk. The feasibility of the proposed method is demonstrated through numerical simulations and optical experiments, highlighting its potential applicability to a wide range of three-dimensional reconstruction algorithms.

Lens array-based holographic 3D display with an expanded field of view and eyebox

Conventional spatial light modulator (SLM)-based holographic 3D display faces limited field of view (FOV) and eyebox, due to its limited pixel number. In this paper, a lens array is used to expand the FOV and eyebox of an SLM-based holographic display. The hologram is calculated to reconstruct a 3D sub-image array, each sub-image corresponding to a specific perspective of the 3D object. Then, the 3D sub-image array is imaged and magnified by the lens array to integrate to the original 3D image. The FOV is expanded due to the large numerical aperture of the lens, and the eyebox is expanded because the lens array generates multiple viewpoints with a large pitch. The optical experiment realizes a 17.6° FOV and 50 mm eyebox, which contains 4 × 4 viewpoints. Apparent motion parallax is observed through the viewpoint array, which is usually hard to observe in a conventional SLM-based holographic display. The proposed method provides a novel, to the best of our knowledge, way to expand the FOV and eyebox of holographic 3D display without increasing the total pixel number of the SLM.

Fast non-iterative algorithm for 3D point-cloud holography

Recently developed iterative and deep learning-based approaches to computer-generated holography (CGH) have been shown to achieve high-quality photorealistic 3D images with spatial light modulators. However, such approaches remain overly cumbersome for patterning sparse collections of target points across a photoresponsive volume in applications including biological microscopy and material processing. Specifically, in addition to requiring heavy computation that cannot accommodate real-time operation in mobile or hardware-light settings, existing sampling-dependent 3D CGH methods preclude the ability to place target points with arbitrary precision, limiting accessible depths to a handful of planes. Accordingly, we present a non-iterative point cloud holography algorithm that employs fast deterministic calculations in order to efficiently allocate patches of SLM pixels to different target points in the 3D volume and spread the patterning of all points across multiple time frames. Compared to a matched-performance implementation of the iterative Gerchberg-Saxton algorithm, our algorithm's relative computation speed advantage was found to increase with SLM pixel count, reaching >100,000x at 512 × 512 array format.

An ultrahigh-fidelity 3D holographic display using scattering to homogenize the angular spectrum

A three-dimensional (3D) holographic display (3DHD) can preserve all the volumetric information about an object. However, the poor fidelity of 3DHD constrains its applications. Here, we present an ultrahigh-fidelity 3D holographic display that uses scattering for homogenization of angular spectrum. A scattering medium randomizes the incident photons and homogenizes the angular spectrum distribution. The redistributed field is recorded by a photopolymer film with numerous modulation modes and a half-wavelength scale pixel size. We have experimentally improved the contrast of a focal spot to 6 × 106 and tightened its spatial resolution to 0.5 micrometers, respectively ~300 and 4.4 times better than digital approaches. By exploiting the spatial multiplexing ability of the photopolymer and the transmission channel selection capability of the scattering medium, we have realized a dynamic holographic display of 3D spirals consisting of 20 foci across 1 millimeter × 1 millimeter × 26 millimeters with uniform intensity.

Flexible dynamic quantitative phase imaging based on division of focal plane polarization imaging technique

This paper proposes a flexible and accurate dynamic quantitative phase imaging (QPI) method using single-shot transport of intensity equation (TIE) phase retrieval achieved by division of focal plane (DoFP) polarization imaging technique. By exploiting the polarization property of the liquid crystal spatial light modulator (LC-SLM), two intensity images of different defocus distances contained in orthogonal polarization directions can be generated simultaneously. Then, with the help of the DoFP polarization imaging, these images can be captured with single exposure, enabling accurate dynamic QPI by solving the TIE. In addition, our approach gains great flexibility in defocus distance adjustment by adjusting the pattern loaded on the LC-SLM. Experiments on microlens array, phase plate, and living human gastric cancer cells demonstrate the accuracy, flexibility, and dynamic measurement performance for various objects. The proposed method provides a simple, flexible, and accurate approach for real-time QPI without sacrificing the field of view.

Crosstalk-free for multi-plane holographic display using double-constraint stochastic gradient descent

Multi-plane crosstalk is a key issue affecting the quality of holographic three-dimensional (3D) displays. The time-multiplexing stochastic gradient descent (TM-SGD) method has been applied to solve the inter-plane crosstalk problem in multi-plane reconstruction. However, the inter-plane crosstalk increases greatly as the inter-plane interval decreases, and the optimization time increases greatly as the number of planes increases. In this paper, we propose a double-constraint stochastic gradient descent method to suppress inter-plane crosstalk in multi-plane reconstruction. In the proposed method, we use the mask to make the optimization process focus more on the signal region and improve the reconstruction quality. Meanwhile, we adopt a constraint strategy of phase regularization to reduce the phase randomness of the signal region and suppress inter-plane crosstalk. Numerical simulation and optical experiment results confirm that our method can effectively suppress the inter-plane crosstalk and improve the quality of the reconstructed planes at a lower inter-plane interval. Moreover, the optimization time of our method is almost 4 times faster than that of TM-SGD. The proposed method can contribute to the realization of tomographic 3D visualization in the biomedical field, which requires the reconstruction of multiple tomographic images without inter-plane crosstalk.

Incoherent diffractive optical elements for extendable field-of-view imaging

We present a diffractive optics design for incoherent imaging with an extendable field-of-view. In our design method, multiple layers of diffractive optical elements (DOEs) are synthesized so that images on the input plane illuminated with spatially incoherent light are reproduced upright on the output plane. In addition, our method removes the need for an approximation of shift invariance, which has been assumed in conventional optical designs for incoherent imaging systems. Once the DOE cascade is calculated, the field-of-view can be extended by using an array of such DOEs without further calculation. We derive the optical condition to calculate the DOEs and numerically demonstrate the proposed method with the condition.

Ensuring privacy and confidentiality of cloud data: A comparative analysis of diverse cryptographic solutions based on run time trend

The cloud is becoming a hub for sensitive data as technology develops, making it increasingly vulnerable, especially as more people get access. Data should be protected and secured since a larger number of individuals utilize the cloud for a variety of purposes. Confidentiality and privacy of data is attained through the use of cryptographic techniques. While each cryptographic method completes the same objective, they all employ different amounts of CPU, memory, throughput, encryption, and decryption times. It is necessary to contrast the various possibilities in order to choose the optimal cryptographic algorithm. An integrated data size of 5n*102 (KB (∈ 1,2,4,10,20,40) is evaluated in this article. Performance metrics including run time, memory use, and throughput time were used in the comparison. To determine the effectiveness of each cryptographic technique, the data sizes were run fifteen (15) times, and the mean simulation results were then reported. In terms of run time trend, NCS is superior to the other algorithms according to Friedman's test and Bonferroni's Post Hoc test.

A Depth-Enhanced Holographic Super Multi-View Display Based on Depth Segmentation

A super multi-view (SMV) near-eye display (NED) effectively provides depth cues for three-dimensional (3D) display by projecting multiple viewpoint or parallax images onto the retina simultaneously. Previous SMV NED have suffered from a limited depth of field (DOF) due to a fixed image plane. In this paper, a holographic SMV Maxwellian display based on depth segmentation is proposed to enhance the DOF. The proposed approach involves capturing a set of parallax images and their corresponding depth maps. According to the depth maps, the parallax images are segmented into N sub-parallax images at different depth ranges. These sub-parallax images are then projected onto N image-recording planes (IRPs) of the corresponding depth for hologram computation. The wavefront at each IRP is calculated by multiplying the sub-parallax images with the corresponding spherical wave phases. Then, they are propagated to the hologram plane and added together to form a DOF-enhanced hologram. The simulation and experimental results are obtained to validate the effectiveness of the proposed method in extending the DOF of the holographic SMV displays, while accurately preserving occlusion.

Fast Hologram Calculation Method Based on Wavefront Precise Diffraction

In this paper, a fast hologram calculation method based on wavefront precise diffraction is proposed. By analyzing the diffraction characteristics of the object point on the 3D object, the effective viewing area of the reproduced image is analyzed. Based on the effective viewing area, the effective hologram size of the object point is obtained, and then the accurate diffraction calculation from the object point to the wavefront recording plane (WRP) is performed. By calculating all the object points on the recorded object, the optimized WRP of the whole 3D object can be obtained. The final hologram is obtained by calculating the diffraction light field from the WRP to the holographic plane. Compared with the traditional method, the proposed method can improve the calculation speed by more than 55%, while the image quality of the holographic 3D display is not affected. The proposed calculation method provides an idea for fast calculation of holograms and is expected to contribute to the development of dynamic holographic displays.

Speckle suppression in arbitrary parallel holographic illumination by the spatial frequency regaining method

This Letter proposes a spatial frequency regaining method for parallel holographic illumination (SFR-PHI) to suppress speckle noise in phase-only computer-generated holography (CGH). Based on the accurate calculation of the beam bandwidth, this method uses the bandwidth-limited quadratic initial phase and weighted constraint iteration to generate the optimized phase hologram, which can provide the accurate spatial frequency of multiple illumination patterns. The results show that SFR-PHI performs superiorly in speckle suppression for generating dozens of illumination patterns in parallel and with arbitrary shapes and numbers. Compared with other speckle-suppression methods, it exhibits significant advantages in terms of accuracy and modulation efficiency.

Efficient rendering by parallelogram-approximation for full analytical polygon-based computer-generated holography using planar texture mapping

We have developed a full analytical method with texture mapping for polygon-based computer-generated holography. A parallel planar projection mapping for holographic rendering along with affine transformation and self-similar segmentation is derived. Based on this method, we further propose a parallelogram-approximation to reduce the number of polygons used in the polygon-based technique. We demonstrate that the overall method can reduce the computational effort by 50% as compared to an existing method without sacrificing the reconstruction quality based on high precision rendering of complex textures. Numerical and optical reconstructions have shown the effectiveness of the overall scheme.

Deep compression network for enhancing numerical reconstruction quality of full-complex holograms

The field of digital holography has been significant developed in recent decades, however, the commercialization of digital holograms is still hindered by the issue of large data sizes. Due to the complex signal characteristics of digital holograms, which are of interferometric nature, traditional codecs are not able to provide satisfactory coding efficiency. Furthermore, in a typical coding scenario, the hologram is encoded and then decoded, leading to a numerical reconstruction via a light wave propagation model. While previous researches have mainly focused on the quality of the decoded hologram, it is the numerical reconstruction that is visible to the viewer, and thus, its quality must also be taken into consideration when designing a coding solution. In this study, the coding performances of existing compression standards, JPEG2000 and HEVC-Intra, are evaluated on a set of digital holograms, then the limitations of these standards are analyzed. Subsequently, we propose a deep learning-based compression network for full-complex holograms that demonstrates superior coding performance when compared to the latest standard codecs such as VVC and JPEG-XL, in addition to JPEG2000 and HEVC. The proposed network incorporates not only the quality of the decoded hologram, but also the quality of the numerical reconstruction as distortion costs for network training. The experimental results validate that the proposed network provides superior objective coding efficiency and better visual quality compared to the existing methods.

Holographic display using layered computer-generated volume hologram

The spatial frequency of the reconstructed image of planar computer-generated hologram(CGH) is limited by the sampling interval and the lack of thickness. To break through this limitation of planar CGH, we propose a new computer-generated volume hologram(CGVH) for full-color dynamic holographic three-dimensional(3D) display, and an iteration-free layered CGVH generation method. The proposed CGVH is equivalent to a volume hologram sampled discretely in three directions. The generation method employs the layered angular spectral diffraction to calculate the light field in the layered CGVH, and then encodes it into a CGVH. Numerical simulation results show that the CGVH can accurately reconstruct full-color 3D objects, where better imaging quality, more concentrated diffraction energy, denser reconstructed spatial frequency information, and larger viewing angle are achieved. The proposed CGVH is expected to be applied to realize dynamic modulation, wavelength multiplexing, and angle multiplexing in various optical fields in the future.

Multi-color complex spatial light modulation with a single digital micromirror device

Spatial light modulators enabling complex light field manipulation has opened up many opportunities in biomedical imaging, holographic display, and adaptive optics. However, traditional spatial light modulators do not allow multi-color operations simultaneously due to their physical constraints, while multi-color modulations are highly desirable in many applications. To overcome this limitation, we demonstrate a multi-color spatial complex light field modulation with a single binary hologram on digital micromirror devices (DMD). This method combines several neighboring micro-mirror pixels into a giant single superpixel, in which the light field's amplitude and phase can be individually determined by internal pixel combinations, and the dynamic range of phase modulation can exceed 2π for the single wavelength. As a result, this extra phase modulation range offers an additional degree of freedom for independent multi-wavelength light modulation. Based on this scheme, multi-color light modulations have been demonstrated in a 2D plane as well as in multiple 3D holographic planes. Moreover, a dual-colored Airy beam has been realized using the same technique. These results bring complex light modulation into a multi-color regime, paving the way for practical applications in information display, imaging, and optical trapping.

Real-time computing for a holographic 3D display based on the sparse distribution of a 3D object and requisite Fourier spectrum

In holographic three-dimensional (3D) displays, the surface structures of 3D objects are reconstructed without their internal parts. In diffraction calculations using 3D fast Fourier transform (FFT), this sparse distribution of 3D objects can reduce the calculation time as the Fourier transform can be analytically solved in the depth direction and the 3D FFT can be resolved into multiple two-dimensional (2D) FFTs. Moreover, the Fourier spectrum required for hologram generation is not the entire 3D spectrum but a partial 2D spectrum located on the hemispherical surface. This sparsity of the required Fourier spectrum also reduces the number of 2D FFTs and improves the acceleration. In this study, a fast calculation algorithm based on two sparsities is derived theoretically and explained in detail. Our proposed algorithm demonstrated a 24-times acceleration improvement compared with a conventional algorithm and realized real-time hologram computing at a rate of 170 Hz.

Color multilayer holographic near-eye augmented reality display

This study demonstrates a full-color near-eye holographic display capable of superimposing color virtual scenes with 2D, 3D, and multiple objects with extended depth upon a real scene, which also has the ability to present different 3D information depending on the focus of the user's eyes using a single computer-generated hologram per color channel. Our setup makes use of a hologram generation method based on two-step propagation and the singular value decomposition of the Fresnel transform impulse response function to efficiently generate the holograms of the target scene. Then, we test our proposal by implementing a holographic display that makes use of a phase-only spatial light modulator and time-division multiplexing for color reproduction. We demonstrate the superior quality and computation speed of this approach compared with other hologram generation techniques with both numerical and experimental results.

High-speed real 3D scene acquisition and 3D holographic reconstruction system based on ultrafast optical axial scanning

The lack of three-dimensional (3D) content is one of the challenges that have been faced by holographic 3D display. Here, we proposed a real 3D scene acquisition and 3D holographic reconstruction system based on ultrafast optical axial scanning. An electrically tunable lens (ETL) was used for high-speed focus shift (up to 2.5 ms). A CCD camera was synchronized with the ETL to acquire multi-focused image sequence of real scene. Then, the focusing area of each multi-focused image was extracted by using Tenengrad operator, and the 3D image were obtained. Finally, 3D holographic reconstruction visible to the naked eye can be achieved by the layer-based diffraction algorithm. The feasibility and effectiveness of the proposed method have been demonstrated by simulation and experiment, and the experimental results agree well with the simulation results. This method will further expand the application of holographic 3D display in the field of education, advertising, entertainment, and other fields.

Diffraction model-driven neural network trained using hybrid domain loss for real-time and high-quality computer-generated holography

Learning-based computer-generated holography (CGH) has demonstrated great potential in enabling real-time, high-quality holographic displays. However, most existing learning-based algorithms still struggle to produce high-quality holograms, due to the difficulty of convolutional neural networks (CNNs) in learning cross-domain tasks. Here, we present a diffraction model-driven neural network (Res-Holo) using hybrid domain loss for phase-only hologram (POH) generation. Res-Holo utilizes the weights of the pretrained ResNet34 as the initialization during the encoder stage of the initial phase prediction network to extract more generic features and also to help prevent overfitting. Also, frequency domain loss is added to further constrain the information that the spatial domain loss is insensitive. The peak signal-to-noise ratio (PSNR) of the reconstructed image is improved by 6.05 dB using hybrid domain loss compared to using spatial domain loss alone. Simulation results show that the proposed Res-Holo can generate high-fidelity 2 K resolution POHs with an average PSNR of 32.88 dB at 0.014 seconds/frame on the DIV2K validation set. Both monochrome and full-color optical experiments show that the proposed method can effectively improve the quality of reproduced images and suppress image artifacts.

Depth-Enhanced Holographic Super Multi-View Maxwellian Display Based on Variable Filter Aperture

The super multi-view (SMV) near-eye display (NED) effectively provides depth cues for three-dimensional (3D) displays by projecting multiple viewpoint images or parallax images onto the retina simultaneously. Previous SMV NED suffers from a limited depth of field (DOF) due to the fixed image plane. Aperture filtering is widely used to enhance the DOF; however, an invariably sized aperture may have opposite effects on objects with different reconstruction depths. In this paper, a holographic SMV display based on the variable filter aperture is proposed to enhance the DOF. In parallax image acquisition, multiple groups of parallax images, each group recording a part of the 3D scene on a fixed depth range, are captured first. In the hologram calculation, each group of wavefronts at the image recording plane (IRP) is calculated by multiplying the parallax images with the corresponding spherical wave phase. Then, they are propagated to the pupil plane and multiplied by the corresponding aperture filter function. The size of the filter aperture is variable which is determined by the depth of the object. Finally, the complex amplitudes at the pupil plane are back-propagated to the holographic plane and added together to form the DOF-enhanced hologram. Simulation and experimental results verify the proposed method could improve the DOF of holographic SMV display, which will contribute to the application of 3D NED.

Fast shadow casting algorithm in analytical polygon-based computer-generated holography

Shadow casting is essential in computer graphics, which can significantly enhance the reality of rendered images. However, shadow casting is rarely studied in polygon-based computer-generated holography (CGH) because state-of-art triangle-based occlusion handling methods are too complicated for shadow casting and unfeasible for complex mutual occlusion handling. We proposed a novel drawing method based on the analytical polygon-based CGH framework and achieved Z-buffer-based occlusion handling instead of the traditional Painter's algorithm. We also achieved shadow casting for parallel and point light sources. Our framework can be generalized to N-edge polygon (N-gon) rendering and accelerated using CUDA hardware, by which the rendering speed can be significantly enhanced.

Double-sided computer-generated holography

We present a method for computer-generated holography (CGH) in which different images are reproduced on both sides of a hologram with a single illumination source. In the proposed method, we use a transmissive spatial light modulator (SLM) and a half mirror (HM) located downstream of the SLM. The light modulated by the SLM is partially reflected by the HM, and the reflected light is modulated again by the SLM for the double-sided image reproduction. We derive an algorithm for double-sided CGH and experimentally demonstrate it.

End-to-end real-time holographic display based on real-time capture of real scenes

Holographic display is considered as a promising three-dimensional (3D) display technology and has been widely studied. However, to date, the real-time holographic display for real scenes is still far from being incorporated in our life. The speed and quality of information extraction and holographic computing need to be further improved. In this paper, we propose an end-to-end real-time holographic display based on real-time capture of real scenes, where the parallax images are collected from the scene and a convolutional neural network (CNN) builds the mapping from the parallax images to the hologram. Parallax images are acquired in real time by a binocular camera, and contain depth information and amplitude information needed for 3D hologram calculation. The CNN, which can transform parallax images into 3D holograms, is trained by datasets consisting of parallax images and high-quality 3D holograms. The static colorful reconstruction and speckle-free real-time holographic display based on real-time capture of real scenes have been verified by the optical experiments. With simple system composition and affordable hardware requirements, the proposed technique will break the dilemma of the existing real-scene holographic display, and open up a new direction for the application of real-scene holographic 3D display such as holographic live video and solving vergence-accommodation conflict (VAC) problems for head-mounted display devices.

Automatic depth map retrieval from digital holograms using a depth-from-focus approach

Recovering the scene depth map from a computer-generated hologram is a problem that remains unsolved, despite the growing interest in the subject. In this paper, we propose to study the application of depth-from-focus (DFF) methods to retrieve the depth information from the hologram. We discuss the different hyperparameters that are required for the application of the method and their impact on the final result. The obtained results show that DFF methods can be used for depth estimation from the hologram if the set of hyperparameters is well chosen.

Reducing crosstalk of a multi-plane holographic display by the time-multiplexing stochastic gradient descent

Multi-plane reconstruction is essential for realizing a holographic three-dimensional (3D) display. One fundamental issue in conventional multi-plane Gerchberg-Saxton (GS) algorithm is the inter-plane crosstalk, mainly caused by the neglect of other planes' interference in the process of amplitude replacement at each object plane. In this paper, we proposed the time-multiplexing stochastic gradient descent (TM-SGD) optimization algorithm to reduce the multi-plane reconstruction crosstalk. First, the global optimization feature of stochastic gradient descent (SGD) was utilized to reduce the inter-plane crosstalk. However, the crosstalk optimization effect would degrade as the number of object planes increases, due to the imbalance between input and output information. Thus, we further introduced the time-multiplexing strategy into both the iteration and reconstruction process of multi-plane SGD to increase input information. In TM-SGD, multiple sub-holograms are obtained through multi-loop iteration and then sequentially refreshed on spatial light modulator (SLM). The optimization condition between the holograms and the object planes converts from one-to-many to many-to-many, improving the optimization of inter-plane crosstalk. During the persistence of vision, multiple sub-hologram jointly reconstruct the crosstalk-free multi-plane images. Through simulation and experiment, we confirmed that TM-SGD could effectively reduce the inter-plane crosstalk and improve image quality.The proposed TM-SGD-based holographic display has wide applications in tomographic 3D visualization for biology, medical science, and engineering design, which need to reconstruct multiple independent tomographic images without inter-plane crosstalk.

Fourier-inspired neural module for real-time and high-fidelity computer-generated holography

Learning-based computer-generated holography (CGH) algorithms appear as novel alternatives to generate phase-only holograms. However, most existing learning-based approaches underperform their iterative peers regarding display quality. Here, we recognize that current convolutional neural networks have difficulty learning cross-domain tasks due to the limited receptive field. In order to overcome this limitation, we propose a Fourier-inspired neural module, which can be easily integrated into various CGH frameworks and significantly enhance the quality of reconstructed images. By explicitly leveraging Fourier transforms within the neural network architecture, the mesoscopic information within the phase-only hologram can be more handily extracted. Both simulation and experiment were performed to showcase its capability. By incorporating it into U-Net and HoloNet, the peak signal-to-noise ratio of reconstructed images is measured at 29.16 dB and 33.50 dB during the simulation, which is 4.97 dB and 1.52 dB higher than those by the baseline U-Net and HoloNet, respectively. Similar trends are observed in the experimental results. We also experimentally demonstrated that U-Net and HoloNet with the proposed module can generate a monochromatic 1080p hologram in 0.015 s and 0.020 s, respectively.

Complex amplitude modulated holographic display system based on polarization grating

We propose a holographic display system for complex amplitude modulation (CAM) using a phase-only spatial light modulator (SLM) and two polarization gratings (PG). The two sub-holograms of the complex-amplitude computed generated hologram (CGH) are loaded in different regions of SLM. Two diffractive components couple in space after longitudinal migration from the double PGs, and finally interfered through the line polarizer. The influence of the system error on the reconstructed image quality is analyzed, which provides a theoretical assessment for adding pre-compensation to CGH to compensate the system error. Moreover, on the base of the proposed system, a large depth of field and enlarged display area display is realized and the real-time display can be achieved because of the analytical complex-amplitude computed generated hologram. The optical experimental results show that the proposed system has high energy efficiency, and can provide high-quality holographic display with a large depth of field and enlarged display area.

Wavefront recording plane-like method for polygon-based holograms

The wavefront recording plane (WRP) method is an algorithm for computer-generated holograms, which has significantly promoted the accelerated computation of point-based holograms. Similarly, in this paper, we propose a WRP-like method for polygon-based holograms. A WRP is placed near the object, and the diffracted fields of all polygons are aggregated in the WRP so that the fields propagating from the polygonal mesh affect only a small region of the plane rather than the full region. Unlike the conventional WRP method used in point-based holograms, the proposed WRP-like method utilizes sparse sampling in the frequency domain to significantly reduce the practical computational kernel size. The proposed WRP-like method and the analytical shading model are used to generate polygon-based holograms of multiple three-dimensional (3D) objects, which are then reproduced to confirm 3D perception. The results indicate that the proposed WRP-like method based on an analytical algorithm is hundreds of times faster than the reference full region sampling case; a hologram with tens of thousands of triangles can be computed in seconds even on a CPU, whereas previous methods required a graphics processing unit to achieve these speeds.

Speckle-free compact holographic near-eye display using camera-in-the-loop optimization with phase constraint

We present a compact holographic near-eye display system with high-quality speckle-free optical reconstructions using camera-in-the-loop (CITL) optimization with phase constraint strategy. The CITL optimization with phase constraint is used to iteratively synthesize the computer-generated holograms (CGHs) based on the in-system optical feedback. The phase constraint embedded into the whole CGH optimization process can effectively smooth the phase profile of reconstructed wave field, which helps to suppress the speckle noise caused by the phase singularities in CGH reconstruction. Numerical and optical experiments have been performed to demonstrate that the proposed method can provide speckle-free optical reconstructions with excellent image fidelity for holographic near-eye display in a compact setup.

Lensless phase-only holographic retinal projection display based on the error diffusion algorithm

Holographic retinal projection display (RPD) can project images directly onto the retina without any lens by encoding a convergent spherical wave phase with the target images. Conventional amplitude-type holographic RPD suffers from strong zero-order light and conjugate. In this paper, a lensless phase-only holographic RPD based on error diffusion algorithm is demonstrated. It is found that direct error diffusion of the complex Fresnel hologram leads to low image quality. Thus, a post-addition phase method is proposed based on angular spectrum diffraction. The spherical wave phase is multiplied after error diffusion process, and acts as an imaging lens. In this way, the error diffusion functions better due to reduced phase difference between adjacent pixels, and a virtual image with improved quality is produced. The viewpoint is easily deflected just by changing the post-added spherical phase. A full-color holographic RPD with adjustable eyebox is demonstrated experimentally with time-multiplexing technique.

Optimized computer-generated hologram for enhancing depth cue based on complex amplitude modulation

In this Letter, we introduce a computer-generated hologram (CGH) optimization method to enhance the depth cue based on complex amplitude modulation (CAM). An iterative algorithm is designed to generate the optimized random phase (ORAP) according to the size of the target image and the bandwidth limitation condition. The ORAP with limited bandwidth is used as the initial phase of the target image and the hologram is encoded based on the analytical formula. Our proposal can maintain the advantages of CAM and achieve holographic three-dimensional (3D) display with an enhanced depth cue. It is expected that the proposed method could be widely used in holographic field in the future.

Optimized phase-only hologram generation for high-quality holographic display

In holographic displays, the Gerchberg-Saxton (GS) method has been widely used to generate the phase-only hologram (POH). However, as the constraint strategy of the GS method cannot optimize the POH well enough, the display quality is degraded. Various methods have been proposed to solve this problem, such as introducing the dummy area, using an initial quadratic phase, and modifying the amplitude constraint strategy. This paper proposes a simple and effective iterative method to optimize POH for the high-quality reconstructed image, which is a combination of the above three methods. In the proposed method, the target image is padded with zeros, and the padded image is divided into the target and non-target region. The initial phase in the target region is reset to the quadratic phase, and the optimized parameters α and β are introduced to modify the amplitude constraint strategy. In the iterative process, this strategy is applied to the target region. Numerical and optical experiments were conducted to verify the effectiveness of the proposed method. The results show that the speckle noise is effectively suppressed, and the quality of the reconstructed image is improved.

Diffraction model-informed neural network for unsupervised layer-based computer-generated holography

Learning-based computer-generated holography (CGH) has shown remarkable promise to enable real-time holographic displays. Supervised CGH requires creating a large-scale dataset with target images and corresponding holograms. We propose a diffraction model-informed neural network framework (self-holo) for 3D phase-only hologram generation. Due to the angular spectrum propagation being incorporated into the neural network, the self-holo can be trained in an unsupervised manner without the need of a labeled dataset. Utilizing the various representations of a 3D object and randomly reconstructing the hologram to one layer of a 3D object keeps the complexity of the self-holo independent of the number of depth layers. The self-holo takes amplitude and depth map images as input and synthesizes a 3D hologram or a 2D hologram. We demonstrate 3D reconstructions with a good 3D effect and the generalizability of self-holo in numerical and optical experiments.

High-performance full-color imaging system based on end-to-end joint optimization of computer-generated holography and metalens

Metasurface has drawn extensive attention due to its capability of modulating light with a high degree of freedom through ultrathin and sub-wavelength optical elements, and metalens, as one of its important applications, promises to replace the bulky refractive optics, facilitating the imaging system light-weight and compact characteristics. Besides, computer-generated holography (CGH) is of substantial interest for three-dimensional (3D) imaging technology by virtue of its ability of restoring the whole optical wave field and re-constructing the true 3D scene. Consequently, the combination of metalens and CGH holds transformative potential in enabling the miniaturization of 3D imaging systems. However, its imaging performance is subject to the aberrations and speckle noises originating from the metalens and CGH. Inspired by recent progress that computational imaging can be applied to close the gap, a novel full-color imaging system, adopting end-to-end joint optimization of metalens and CGH for high imaging quality, is proposed in this paper. The U-net based network as the pre-processing adjusts weights to make the holographic reconstruction offset imaging defects, incorporating the imaging processing into the step of generating hologram. Optimized by deep learning, the proposed imaging system is capable of full-color imaging with high fidelity in a compact form factor, envisioned to take an essential step towards the high-performance miniaturized imaging system.

Color dynamic holographic display based on complex amplitude modulation with bandwidth constraint strategy

In this Letter, we introduce a multiplexing encoding method with a bandwidth constraint strategy to realize a color dynamic holographic display based on complex amplitude modulation (CAM). The method first uses the angular spectrum method (ASM) with a bandwidth constraint strategy to calculate the diffracted wavefronts of red, green, and blue channels. Then the diffracted wavefronts of three channels are synthesized into a color-multiplexed hologram (CMH) based on the double-phase method after they interfere with off-axis reference lights. The color 3D objects can be reconstructed when the combination of red, green, and blue lights is used to illuminate the double-phase CMH, and a 4f system with a slit filter is introduced to extract the desired spectrums. Numerical simulations and optical experiments are performed to verify the effectiveness of the proposed method and the results show that it can achieve a color holographic display with high quality. Our proposal is simple and fast, and the display system is compact. It is expected that our proposed method could in future be widely used in the holographic field.