Progress in virtual reality and augmented reality based on holographic display

Viral Content in Event Management of Hospitality and Socio-Cultural Activities

This chapter examines the determinants of the formation of viral content in the event management of hospitality entities and socio-cultural activities. The methodology covers the assessment of the popularity and potential impact of social networks on marketing and practical opportunities of hospitality entities and socio-cultural activities. The modeling method was applied to create a reference model of the credibility strategy of impression marketing. In the process of forming authenticity and emotional involvement of customers, the technological road map method, which is based on brand marketing approaches, was applied. In addition, the analysis of the involvement of users of social networks was investigated. The selection of key indicators for evaluating the effectiveness of real content and an in-depth review of modern technologies and trends in its creation empirically contributed to the modeling of the strategy of creating real content in the field of event management of hospitality entities and socio-cultural activities.

Non-convex optimization for inverse problem solving in computer-generated holography

Computer-generated holography is a promising technique that modulates user-defined wavefronts with digital holograms. Computing appropriate holograms with faithful reconstructions is not only a problem closely related to the fundamental basis of holography but also a long-standing challenge for researchers in general fields of optics. Finding the exact solution of a desired hologram to reconstruct an accurate target object constitutes an ill-posed inverse problem. The general practice of single-diffraction computation for synthesizing holograms can only provide an approximate answer, which is subject to limitations in numerical implementation. Various non-convex optimization algorithms are thus designed to seek an optimal solution by introducing different constraints, frameworks, and initializations. Herein, we overview the optimization algorithms applied to computer-generated holography, incorporating principles of hologram synthesis based on alternative projections and gradient descent methods. This is aimed to provide an underlying basis for optimized hologram generation, as well as insights into the cutting-edge developments of this rapidly evolving field for potential applications in virtual reality, augmented reality, head-up display, data encryption, laser fabrication, and metasurface design.

Tomographic waveguide-based augmented reality display

A tomographic waveguide-based augmented reality display technique is proposed for near-eye three-dimensional (3D) display with accurate depth reconstructions. A pair of tunable lenses with complementary focuses is utilized to project tomographic virtual 3D images while maintaining the correct perception of the real scene. This approach reconstructs virtual 3D images with physical depth cues, thereby addressing the vergence-accommodation conflict inherent in waveguide augmented reality systems. A prototype has been constructed and optical experiments have been conducted, demonstrating the system's capability in delivering high-quality 3D scenes for waveguide-based augmented reality display.

Computer-generated holography with ordinary display

We propose a method of computer-generated holography (CGH) using incoherent light emitted from a mobile phone screen. In this method, we suppose a cascade of holograms in which the first hologram is a color image displayed on the mobile phone screen. The hologram cascade is synthesized by solving an inverse problem with respect to the propagation of incoherent light. We demonstrate a three-dimensional color image reproduction using a two-layered hologram cascade composed of an iPhone and a spatial light modulator.

Effect of shrinkage in photopolymer film on the information transmitted through the holographic waveguide for near eye displays

Due to shrinkage in photopolymer materials, the angle of the reconstruction beam in holographic optical elements (HOEs) does not match with the Bragg condition, resulting in a decreased amount of light in the desired direction or loss of transmitted information to rematch the Bragg condition. Thus, to ensure final display features it is imperative to precompensate the shrinkage effect. We derived simplified expressions for precompensation in recording geometries of required HOEs in holographic waveguide-based Maxwellian near eye displays. An acceptable range of detuning from the Bragg angle is also analyzed. The experimentally measured 4.95% shrinkage in photopolymer film for 0° and 45° recording angles of beams was precompensated using -0.86∘ and 43.7° recording angles. Theoretical results are validated through simulation and experiments.

Spatial perception in stereoscopic augmented reality based on multifocus sensing

In many areas ranging from medical imaging to visual entertainment, 3D information acquisition and display is a key task. In this regard, in multifocus computational imaging, stacks of images of a certain 3D scene are acquired under different focus configurations and are later combined by means of post-capture algorithms based on image formation model in order to synthesize images with novel viewpoints of the scene. Stereoscopic augmented reality devices, through which is possible to simultaneously visualize the three dimensional real world along with overlaid digital stereoscopic image pair, could benefit from the binocular content allowed by multifocus computational imaging. Spatial perception of the displayed stereo pairs can be controlled by synthesizing the desired point of view of each image of the stereo-pair along with their parallax setting. The proposed method has the potential to alleviate the accommodation-convergence conflict and make augmented reality stereoscopic devices less vulnerable to visual fatigue.

High-brightness hybrid compressive light field display with improved image quality

Previous LCD-based multiplicative compressive light field (CLF) display has the trade-off between the brightness and the depth of field (DOF). In this paper, we propose a hybrid CLF display using a reflective polarizer and RGB mini-LED panel. By the polarization-multiplexing and the reflector dam (RD) designed on the mini-LED panel, the proposed system can preserve high brightness while enhancing the DOF. Then, a decomposition algorithm is proposed to improve the image quality by depth segmentation and limiting the motion parallax. Compared to the conventional hybrid CLF display, the brightness of the proposed system reaches 348 nits and the reconstruction quality achieves structural similarity index measure (SSIM) improvement by 0.12. The experiments also demonstrate that the proposed method could achieve a higher brightness, larger depth of field, and higher image quality.

An ultrahigh-fidelity 3D holographic display using scattering to homogenize the angular spectrum

A three-dimensional (3D) holographic display (3DHD) can preserve all the volumetric information about an object. However, the poor fidelity of 3DHD constrains its applications. Here, we present an ultrahigh-fidelity 3D holographic display that uses scattering for homogenization of angular spectrum. A scattering medium randomizes the incident photons and homogenizes the angular spectrum distribution. The redistributed field is recorded by a photopolymer film with numerous modulation modes and a half-wavelength scale pixel size. We have experimentally improved the contrast of a focal spot to 6 × 106 and tightened its spatial resolution to 0.5 micrometers, respectively ~300 and 4.4 times better than digital approaches. By exploiting the spatial multiplexing ability of the photopolymer and the transmission channel selection capability of the scattering medium, we have realized a dynamic holographic display of 3D spirals consisting of 20 foci across 1 millimeter × 1 millimeter × 26 millimeters with uniform intensity.

Design of freeform phase diffractive optical elements based on the quadratic assignment problem

The design of freeform phase diffractive optical elements is a challenging task, typically necessitating the use of complex differential equations or a large number of iterative calculations. This paper proposes what we believe to be a novel approach to address this problem. In this strategy, we introduce overall comparison optimization (OCO) to ensure the fast convergence of the cost function. The quadratic assignment problem (QAP) is used as the mathematical framework for designing freeform phase diffraction optics. Specifically, the ray mapping calculation problem in geometric optics is simplified as a QAP. To solve this problem, we apply the OCO method, which ensures that the cost function rapidly progresses in the "non-negative" direction, thereby facilitating fast convergence in each optimization iteration. In this manner, the proposed approach alleviates the computational burden associated with repeated evaluations of the cost function and accelerates convergence in the design process. We construct holographic masks using the OCO method and perform simulations to demonstrate the potential of the proposed method in swiftly realizing complex illumination patterns. The results show that the design model has good performance when dealing with complex illumination tasks. The conclusions obtained in this paper can be extended to the realization of phase-only holography and the solution of freeform surfaces illumination design.

Laser nanoprinting of 3D nonlinear holograms beyond 25000 pixels-per-inch for inter-wavelength-band information processing

Nonlinear optics provides a means to bridge between different electromagnetic frequencies, enabling communication between visible, infrared, and terahertz bands through χ(2) and higher-order nonlinear optical processes. However, precisely modulating nonlinear optical waves in 3D space remains a significant challenge, severely limiting the ability to directly manipulate optical information across different wavelength bands. Here, we propose and experimentally demonstrate a three-dimensional (3D) χ(2)-super-pixel hologram with nanometer resolution in lithium niobate crystals, capable of performing advanced processing tasks. In our design, each pixel consists of properly arranged nanodomain structures capable of completely and dynamically manipulating the complex-amplitude of nonlinear waves. Fabricated by femtosecond laser writing, the nonlinear hologram features a pixel diameter of 500 nm and a pixel density of approximately 25000 pixels-per-inch (PPI), reaching far beyond the state of the art. In our experiments, we successfully demonstrate the novel functions of the hologram to process near-infrared (NIR) information at visible wavelengths, including dynamic 3D nonlinear holographic imaging and frequency-up-converted image recognition. Our scheme provides a promising nano-optic platform for high-capacity optical storage and multi-functional information processing across different wavelength ranges.

A Depth-Enhanced Holographic Super Multi-View Display Based on Depth Segmentation

A super multi-view (SMV) near-eye display (NED) effectively provides depth cues for three-dimensional (3D) display by projecting multiple viewpoint or parallax images onto the retina simultaneously. Previous SMV NED have suffered from a limited depth of field (DOF) due to a fixed image plane. In this paper, a holographic SMV Maxwellian display based on depth segmentation is proposed to enhance the DOF. The proposed approach involves capturing a set of parallax images and their corresponding depth maps. According to the depth maps, the parallax images are segmented into N sub-parallax images at different depth ranges. These sub-parallax images are then projected onto N image-recording planes (IRPs) of the corresponding depth for hologram computation. The wavefront at each IRP is calculated by multiplying the sub-parallax images with the corresponding spherical wave phases. Then, they are propagated to the hologram plane and added together to form a DOF-enhanced hologram. The simulation and experimental results are obtained to validate the effectiveness of the proposed method in extending the DOF of the holographic SMV displays, while accurately preserving occlusion.

Real-time computing for a holographic 3D display based on the sparse distribution of a 3D object and requisite Fourier spectrum

In holographic three-dimensional (3D) displays, the surface structures of 3D objects are reconstructed without their internal parts. In diffraction calculations using 3D fast Fourier transform (FFT), this sparse distribution of 3D objects can reduce the calculation time as the Fourier transform can be analytically solved in the depth direction and the 3D FFT can be resolved into multiple two-dimensional (2D) FFTs. Moreover, the Fourier spectrum required for hologram generation is not the entire 3D spectrum but a partial 2D spectrum located on the hemispherical surface. This sparsity of the required Fourier spectrum also reduces the number of 2D FFTs and improves the acceleration. In this study, a fast calculation algorithm based on two sparsities is derived theoretically and explained in detail. Our proposed algorithm demonstrated a 24-times acceleration improvement compared with a conventional algorithm and realized real-time hologram computing at a rate of 170 Hz.

Depth-Enhanced Holographic Super Multi-View Maxwellian Display Based on Variable Filter Aperture

The super multi-view (SMV) near-eye display (NED) effectively provides depth cues for three-dimensional (3D) displays by projecting multiple viewpoint images or parallax images onto the retina simultaneously. Previous SMV NED suffers from a limited depth of field (DOF) due to the fixed image plane. Aperture filtering is widely used to enhance the DOF; however, an invariably sized aperture may have opposite effects on objects with different reconstruction depths. In this paper, a holographic SMV display based on the variable filter aperture is proposed to enhance the DOF. In parallax image acquisition, multiple groups of parallax images, each group recording a part of the 3D scene on a fixed depth range, are captured first. In the hologram calculation, each group of wavefronts at the image recording plane (IRP) is calculated by multiplying the parallax images with the corresponding spherical wave phase. Then, they are propagated to the pupil plane and multiplied by the corresponding aperture filter function. The size of the filter aperture is variable which is determined by the depth of the object. Finally, the complex amplitudes at the pupil plane are back-propagated to the holographic plane and added together to form the DOF-enhanced hologram. Simulation and experimental results verify the proposed method could improve the DOF of holographic SMV display, which will contribute to the application of 3D NED.

Cross talk-free retinal projection display based on a holographic complementary viewpoint array

In near-eye displays (NEDs), retinal projection display (RPD) is one kind of promising technology to alleviate the vergence-accommodation conflict (VAC) issue due to its always-in-focus feature. Viewpoint replication is widely used to enlarge the limited eyebox. However, the mismatch between viewpoint interval and eye pupil diameter will cause the inter-viewpoint cross talk when multiple viewpoints enter the pupil simultaneously. In this Letter, a holographic complementary viewpoint method is proposed to solve this cross talk problem. Instead of avoiding observing multiple viewpoint images simultaneously, it is designed that multiple complementary viewpoints jointly project the complete image on the retina without cross talk. To do this, the target image is segmented into multiple sub-images, each multiplied with a corresponding partial spherical phase to converge to a specific complementary viewpoint. A group of complementary viewpoint enter the eye pupil simultaneously, and each viewpoint project a corresponding sub-image on a specific area of the retina and splice to a complete image. All of the complementary viewpoints are duplicated to an interlaced two-dimensional array to extend the eyebox in both horizontal and vertical directions. Optical experiment verifies that the proposed method could present smooth transition between viewpoints to avoid both inter-viewpoint cross talk and blank image issues.

Bragg degenerate model for fabrication of holographic waveguide-based near-eye displays

The coupling efficiency of light beams is a crucial factor for waveguide displays. Generally, the light beam is not coupled with maximum efficiency in the holographic waveguide without employing a prism in the recording geometry. Use of prisms in recording geometry leads to restricting the propagation angle of the waveguide to a specific value only. The issue of efficient coupling of a light beam without using prisms could be overcome via Bragg degenerate configuration. In this work, the simplified expressions of the Bragg degenerate case are obtained for the realization of normally illuminated waveguide-based displays. Using this model, by tuning the parameters of recording geometry, a range of propagation angles can be produced for a fixed normal incidence of a playback beam. Numerical simulations and experimental investigations of the Bragg degenerate waveguides of different geometries are performed to validate the model. A Bragg degenerate playback beam is successfully coupled in four waveguides recorded with different geometries and yields good diffraction efficiency at normal incidence. The quality of transmitted images is characterized using the structural similarity index measure. The augmentation of a transmitted image in the real world is experimentally demonstrated through the fabricated holographic waveguide for near-eye display applications. Bragg degenerate configuration can provide flexibility in the angle of propagation while maintaining the same coupling efficiency achievable with a prism for holographic waveguide displays.

Equal-intensity beam splitter realization by wire grid polarizers for passive laser speckle reduction

A method to realize an equal-intensity beam splitter (EIBS) using wire grid polarizers (WGPs) is proposed. The EIBS consists of WGPs with predetermined orientations and high-reflectivity mirrors. We demonstrated the generation of three laser sub-beams (LSBs) with equivalent intensities using EIBS. The three LSBs were incoherent by introducing optical path differences larger than the laser coherence length. The LSBs were used to reduce speckle passively, where the objective speckle contrast was reduced from 0.82 to 0.5 when all three LSBs were used. The feasibility of EIBS in speckle reduction was studied using a simplified laser projection system. The structure of the EIBS implemented by WGPs is simpler than EIBSs obtained by other methods.

Compact reconstruction of a Fourier hologram for a 3D object by scaling compensation

The Fourier holographic projection method is compact and computationally fast. However, since the magnification of the displayed image increases with the diffraction distance, this method cannot be used directly to display multi-plane three-dimensional (3D) scenes. We propose a holographic 3D projection method of Fourier holograms by scaling compensation to offset the magnification during optical reconstruction. To achieve a compact system, the proposed method is also used to reconstruct 3D virtual images with Fourier holograms. Different from traditional Fourier holographic displays, images are reconstructed behind a spatial light modulator (SLM) so that the observation position can be placed close to the SLM. The effectiveness of the method and the flexibility of combining it with other methods are confirmed by simulations and experiments. Therefore, our method could have potential applications in the augmented reality (AR) and virtual reality (VR) fields.

Investigating learning-empowered hologram generation for holographic displays with ill-tuned hardware

Existing computational holographic displays often suffer from limited reconstruction image quality mainly due to ill-conditioned optics hardware and hologram generation software. In this Letter, we develop an end-to-end hardware-in-the-loop approach toward high-quality hologram generation for holographic displays. Unlike other hologram generation methods using ideal wave propagation, ours can reduce artifacts introduced by both the light propagation model and the hardware setup, in particular non-uniform illumination. Experimental results reveal that, compared with classical computer-generated hologram algorithm counterparts, better quality of holographic images can be delivered without a strict requirement on both the fine assembly of optical components and the good uniformity of laser sources.

Deep hologram converter from low-precision to middle-precision holograms

We propose a deep hologram converter based on deep learning to convert low-precision holograms into middle-precision holograms. The low-precision holograms were calculated using a shorter bit width. It can increase the amount of data packing for single instruction/multiple data in the software approach and the number of calculation circuits in the hardware approach. One small and one large deep neural network (DNN) are investigated. The large DNN exhibited better image quality, whereas the smaller DNN exhibited a faster inference time. Although the study demonstrated the effectiveness of point-cloud hologram calculations, this scheme could be extended to various other hologram calculation algorithms.

Reducing crosstalk of a multi-plane holographic display by the time-multiplexing stochastic gradient descent

Multi-plane reconstruction is essential for realizing a holographic three-dimensional (3D) display. One fundamental issue in conventional multi-plane Gerchberg-Saxton (GS) algorithm is the inter-plane crosstalk, mainly caused by the neglect of other planes' interference in the process of amplitude replacement at each object plane. In this paper, we proposed the time-multiplexing stochastic gradient descent (TM-SGD) optimization algorithm to reduce the multi-plane reconstruction crosstalk. First, the global optimization feature of stochastic gradient descent (SGD) was utilized to reduce the inter-plane crosstalk. However, the crosstalk optimization effect would degrade as the number of object planes increases, due to the imbalance between input and output information. Thus, we further introduced the time-multiplexing strategy into both the iteration and reconstruction process of multi-plane SGD to increase input information. In TM-SGD, multiple sub-holograms are obtained through multi-loop iteration and then sequentially refreshed on spatial light modulator (SLM). The optimization condition between the holograms and the object planes converts from one-to-many to many-to-many, improving the optimization of inter-plane crosstalk. During the persistence of vision, multiple sub-hologram jointly reconstruct the crosstalk-free multi-plane images. Through simulation and experiment, we confirmed that TM-SGD could effectively reduce the inter-plane crosstalk and improve image quality.The proposed TM-SGD-based holographic display has wide applications in tomographic 3D visualization for biology, medical science, and engineering design, which need to reconstruct multiple independent tomographic images without inter-plane crosstalk.

Low-loss and broadband cascaded SiN RGB coupler with dual-mode interference

We design and demonstrate a cascaded SiN-based RGB coupler with dual-mode interference (DMI) for a micro laser scanning image projector. The DMI configuration in SiN-based waveguides mitigates the adverse effects of self-image point deviation caused by wavelength dispersion, achieving decreased device length and high transmission efficiency. The underlying mechanisms are discussed based on coupled-mode theory and 3D finite difference time domain simulation. A small footprint of 3µm×70µm is achieved for the RGB coupler device without input/output circuits, and the insertion losses are less than 0.56 dB at RGB wavelengths. There are two orders of magnitude reduction in device length as compared with the conventional S i O ₂-based RGB coupler, which greatly promotes the miniaturization of the couplers and displays integration advantages with a laser diode and waveguide photodetector. In addition, the 3 dB bandwidth is over 50 nm for the coupler, and it demonstrates good fabrication tolerance. This design can further be integrated into visible light communication systems and applied to visible light integrated photonics.

Super multi-view near-eye virtual reality with directional backlights from wave-guides

Directional backlights have often been employed for generating multiple view-zones in three-dimensional (3D) display, with each backlight converging into a corresponding view-zone. By designing the view-zone interval for each pupil smaller than the pupil's diameter, super multi-view (SMV) can get implemented for a VAC-free 3D display. However, expanding the backlight from a light-source to cover the corresponding display panel often needs an extra thickness, which results in a thicker structure and is unwanted by a near-eye display. In this paper, two wave-guides are introduced into a near-eye virtual reality (NEVR) system, for sequentially guiding more than one directional backlight to each display panel for SMV display without bringing obvious extra thickness. A prototype SMV NEVR gets demonstrated, with two backlights from each wave-guide converging into two view-zones for a corresponding pupil. Although the additional configured light-sources are positioned far from the corresponding wave-guide in our proof-of-concept prototype, multiple light-sources can be attached to the corresponding wave-guide compactly if necessary. As proof, a 3D scene with defocus-blur effects gets displayed. The design range of the backlights' total reflection angles in the wave-guide is also discussed.

Wavefront recording plane-like method for polygon-based holograms

The wavefront recording plane (WRP) method is an algorithm for computer-generated holograms, which has significantly promoted the accelerated computation of point-based holograms. Similarly, in this paper, we propose a WRP-like method for polygon-based holograms. A WRP is placed near the object, and the diffracted fields of all polygons are aggregated in the WRP so that the fields propagating from the polygonal mesh affect only a small region of the plane rather than the full region. Unlike the conventional WRP method used in point-based holograms, the proposed WRP-like method utilizes sparse sampling in the frequency domain to significantly reduce the practical computational kernel size. The proposed WRP-like method and the analytical shading model are used to generate polygon-based holograms of multiple three-dimensional (3D) objects, which are then reproduced to confirm 3D perception. The results indicate that the proposed WRP-like method based on an analytical algorithm is hundreds of times faster than the reference full region sampling case; a hologram with tens of thousands of triangles can be computed in seconds even on a CPU, whereas previous methods required a graphics processing unit to achieve these speeds.

High Resolution Multiview Holographic Display Based on the Holographic Optical Element

Limited by the low space-bandwidth product of the spatial light modulator (SLM), it is difficult to realize multiview holographic three-dimensional (3D) display. To conquer the problem, a method based on the holographic optical element (HOE), which is regarded as a controlled light element, is proposed in the study. The SLM is employed to upload the synthetic phase-only hologram generated by the angular spectrum diffraction theory. Digital grating is introduced in the generation process of the hologram to achieve the splicing of the reconstructions and adjust the position of the reconstructions. The HOE fabricated by the computer-generated hologram printing can redirect the reconstructed images of multiview into multiple viewing zones. Thus, the modulation function of the HOE should be well-designed to avoid crosstalk between perspectives. The experimental results show that the proposed system can achieve multiview holographic augmented reality (AR) 3D display without crosstalk. The resolution of each perspective is 4K, which is higher than that of the existing multiview 3D display system.

Lensless phase-only holographic retinal projection display based on the error diffusion algorithm

Holographic retinal projection display (RPD) can project images directly onto the retina without any lens by encoding a convergent spherical wave phase with the target images. Conventional amplitude-type holographic RPD suffers from strong zero-order light and conjugate. In this paper, a lensless phase-only holographic RPD based on error diffusion algorithm is demonstrated. It is found that direct error diffusion of the complex Fresnel hologram leads to low image quality. Thus, a post-addition phase method is proposed based on angular spectrum diffraction. The spherical wave phase is multiplied after error diffusion process, and acts as an imaging lens. In this way, the error diffusion functions better due to reduced phase difference between adjacent pixels, and a virtual image with improved quality is produced. The viewpoint is easily deflected just by changing the post-added spherical phase. A full-color holographic RPD with adjustable eyebox is demonstrated experimentally with time-multiplexing technique.

Diffraction model-informed neural network for unsupervised layer-based computer-generated holography

Learning-based computer-generated holography (CGH) has shown remarkable promise to enable real-time holographic displays. Supervised CGH requires creating a large-scale dataset with target images and corresponding holograms. We propose a diffraction model-informed neural network framework (self-holo) for 3D phase-only hologram generation. Due to the angular spectrum propagation being incorporated into the neural network, the self-holo can be trained in an unsupervised manner without the need of a labeled dataset. Utilizing the various representations of a 3D object and randomly reconstructing the hologram to one layer of a 3D object keeps the complexity of the self-holo independent of the number of depth layers. The self-holo takes amplitude and depth map images as input and synthesizes a 3D hologram or a 2D hologram. We demonstrate 3D reconstructions with a good 3D effect and the generalizability of self-holo in numerical and optical experiments.

Binocular holographic display based on the holographic optical element

Due to the limited pixel pitch of the spatial light modulator (SLM), the field of view (FOV) is insufficient to meet binocular observation needs. Here, an optimized controlling light method of a binocular holographic three-dimensional (3D) display system based on the holographic optical element (HOE) is proposed. The synthetic phase-only hologram uploaded onto the SLM is generated with the layer-based angular spectrum diffraction theory, and two different reference waves are introduced to separate the left view and the right view of the 3D scene. The HOE with directional controlling light parameters is employed to guide binocular information into the left-eye and the right-eye viewing zones simultaneously. Optical experiments verify that the proposed system can achieve binocular holographic augmented reality 3D effect successfully with real physical depth, which can eliminate the accommodation-vergence conflict and visual fatigue problem. For each perspective, the FOV is 8.7° when the focal length of the HOE is 10 cm. The width of the viewing zone is 2.3 cm when the viewing distance is 25 cm.

High-performance full-color imaging system based on end-to-end joint optimization of computer-generated holography and metalens

Metasurface has drawn extensive attention due to its capability of modulating light with a high degree of freedom through ultrathin and sub-wavelength optical elements, and metalens, as one of its important applications, promises to replace the bulky refractive optics, facilitating the imaging system light-weight and compact characteristics. Besides, computer-generated holography (CGH) is of substantial interest for three-dimensional (3D) imaging technology by virtue of its ability of restoring the whole optical wave field and re-constructing the true 3D scene. Consequently, the combination of metalens and CGH holds transformative potential in enabling the miniaturization of 3D imaging systems. However, its imaging performance is subject to the aberrations and speckle noises originating from the metalens and CGH. Inspired by recent progress that computational imaging can be applied to close the gap, a novel full-color imaging system, adopting end-to-end joint optimization of metalens and CGH for high imaging quality, is proposed in this paper. The U-net based network as the pre-processing adjusts weights to make the holographic reconstruction offset imaging defects, incorporating the imaging processing into the step of generating hologram. Optimized by deep learning, the proposed imaging system is capable of full-color imaging with high fidelity in a compact form factor, envisioned to take an essential step towards the high-performance miniaturized imaging system.

Physics-enhanced neural network for phase retrieval from two diffraction patterns

In this work, we propose a physics-enhanced two-to-one Y-neural network (two inputs and one output) for phase retrieval of complex wavefronts from two diffraction patterns. The learnable parameters of the Y-net are optimized by minimizing a hybrid loss function, which evaluates the root-mean-square error and normalized Pearson correlated coefficient on the two diffraction planes. An angular spectrum method network is designed for self-supervised training on the Y-net. Amplitudes and phases of wavefronts diffracted by a USAF-1951 resolution target, a phase grating of 200 lp/mm, and a skeletal muscle cell were retrieved using a Y-net with 100 learning iterations. Fast reconstructions could be realized without constraints or a priori knowledge of the samples.

Simultaneous multi-channel near-eye display: a holographic retinal projection display with large information content

Augmented reality (AR) near-eye displays (NEDs) are emerging as the next-generation display platform. The existing AR NED only present one single video channel at a time, same as traditional media such as TVs and smartphones. In this Letter, to the best of our knowledge, we propose for the first time a multi-channel holographic retinal projection display (RPD), which can provide multi-channel image sources simultaneously, thus greatly increasing the information content. Due to the superposition capacity of a hologram, multiple images are projected to different viewpoints simultaneously through multiple spherical wave encoding, so that the viewer can switch among playing channels very fast through eye rotation. A full-color dynamic multi-channel holographic near-eye display is demonstrated in the optical experiment. The proposed method provides a good prospect that the future AR glasses can play dozens of video channels in parallel, and the user can switch among channels freely and efficiently just through a simple eye rotation.

Holographic super multi-view Maxwellian near-eye display with eyebox expansion

A holographic super multi-view (SMV) Maxwellian display based on flexible wavefront modulation is proposed for the first time, to the best of our knowledge. It solves the issue that the previous holographic Maxwellian displays could not provide depth cues for monocular vision. Different from the previous methods, two or more parallax images are multiplied by quadric phase distributions and converged to the viewpoints existing in the pupil to provide 3-D vision. A time division method is proposed to eliminate the cross talk caused by the coherence of different spherical waves. Experiments demonstrate that the proposed method can accurately reconstruct images at different depth without cross talk. The proposed method inherits the previous holographic Maxwellian display's advantages of flexible viewpoint position adjustment and large depth of field (DOF). Superior to geometric optics based SMV displays, the proposed system is compact without lens aberration since only a single spatial light modulator (SLM) is needed without any additional optical elements.

Pixelated volume holographic optical element for augmented reality 3D display

Augmented reality (AR) three-dimensional (3D) display is the hardware entrance of metaverse and attracts great interest. The fusion of physical world with 3D virtual images is non-trivial. In this paper, we proposed an AR 3D display based on a pixelated volume holographic optical element (P-VHOE). The see-through combiner is prepared by spatial multiplexing. A prototype of AR 3D display with high diffraction efficiency (78.59%), high transmission (>80%) and non-repeating views is realized. Virtual 3D objects with high fidelity in depth is reconstructed by P-VHOE, with a complex wavelet structural similarity (CW-SSIM) value of 0.9882. The proposed prototype provides an efficient solution for a compact glasses-free AR 3D display. Potential applications include window display, exhibition, education, teleconference.

Three-dimensional computer holography enabled from a single 2D image

To compute a high-quality computer-generated hologram (CGH) for true 3D real scenes, a huge amount of 3D data must be physically acquired and provided depending on specific devices or 3D rendering techniques. Here, we propose a computational framework for generating a CGH from a single image based on the idea of 2D-to-3D wavefront conversion. We devise a deep view synthesis neural network to synthesize light-field contents from a single image and convert the light-field data to the diffractive wavefront of the hologram using a ray-wave algorithm. The method is able to achieve extremely straightforward 3D CGH generation from hand-accessible 2D image content and outperforms existing real-world-based CGH computation, which inevitably relies on a high-cost depth camera and cumbersome 3D data rendering. We experimentally demonstrate 3D reconstructions of indoor and outdoor scenes from a single image enabled phase-only CGH.

Active optical metasurfaces: comprehensive review on physics, mechanisms, and prospective applications

Optical metasurfaces with subwavelength thickness hold considerable promise for future advances in fundamental optics and novel optical applications due to their unprecedented ability to control the phase, amplitude, and polarization of transmitted, reflected, and diffracted light. Introducing active functionalities to optical metasurfaces is an essential step to the development of next-generation flat optical components and devices. During the last few years, many attempts have been made to develop tunable optical metasurfaces with dynamic control of optical properties (e.g., amplitude, phase, polarization, spatial/spectral/temporal responses) and early-stage device functions (e.g., beam steering, tunable focusing, tunable color filters/absorber, dynamic hologram, etc) based on a variety of novel active materials and tunable mechanisms. These recently-developed active metasurfaces show significant promise for practical applications, but significant challenges still remain. In this review, a comprehensive overview of recently-reported tunable metasurfaces is provided which focuses on the ten major tunable metasurface mechanisms. For each type of mechanism, the performance metrics on the reported tunable metasurface are outlined, and the capabilities/limitations of each mechanism and its potential for various photonic applications are compared and summarized. This review concludes with discussion of several prospective applications, emerging technologies, and research directions based on the use of tunable optical metasurfaces. We anticipate significant new advances when the tunable mechanisms are further developed in the coming years.

Computer-generated holography in the intermediate domain

Iterative Fourier transform algorithms are widely used for hologram generation for phase-modulating spatial light modulators. In this paper, we introduce a new technique called the "intermediate domain," which decomposes the Fourier transforms used into multiple subtransforms, the combination of which can offer major performance benefits over traditional approaches. To demonstrate this, we introduce ID-GS, an implementation of the intermediate domain technique for possibly the best known hologram generation algorithm, Gerchberg-Saxton. We discuss the performance of this across a wide range of configurations with a focus on computational performance.

Polygon-based computer-generated holography: a review of fundamentals and recent progress [Invited]

In this review paper, we first provide comprehensive tutorials on two classical methods of polygon-based computer-generated holography: the traditional method (also called the fast-Fourier-transform-based method) and the analytical method. Indeed, other modern polygon-based methods build on the idea of the two methods. We will then present some selective methods with recent developments and progress and compare their computational reconstructions in terms of calculation speed and image quality, among other things. Finally, we discuss and propose a fast analytical method called the fast 3D affine transformation method, and based on the method, we present a numerical reconstruction of a computer-generated hologram (CGH) of a 3D surface consisting of 49,272 processed polygons of the face of a real person without the use of graphic processing units; to the best of our knowledge, this represents a state-of-the-art numerical result in polygon-based computed-generated holography. Finally, we also show optical reconstructions of such a CGH and another CGH of the Stanford bunny of 59,996 polygons with 31,724 processed polygons after back-face culling. We hope that this paper will bring out some of the essence of polygon-based computer-generated holography and provide some insights for future research.

Accelerated Generation of a Pinhole-Type Holographic Stereogram Based on Human Eye Characteristics in Near-Eye Displays

In near-eye displays (NEDs), issues such as weight, heat, and power consumption mean that the rendering and computing power is usually insufficient. Due to this limitation, algorithms need to be further improved for the rapid generation of holograms. In this paper, we propose two methods based on the characteristics of the human eye in NEDs to accelerate the generation of the pinhole-type holographic stereogram (HS). In the first method, we consider the relatively fixed position of the human eye in NEDs. The number of visible pixels from each elemental image is very small due to the limited pupil size of an observing eye, and the calculated amount can be dramatically reduced. In the second method, the foveated region rendering method is adopted to further enhance the calculation speed. When the two methods are adopted at the same time, the calculation speed can be increased dozens of times. Simulations demonstrate that the proposed method can obviously enhance the generation speed of a pinhole-type HS.

Conjugate wavefront encoding: an efficient eyebox extension approach for holographic Maxwellian near-eye display

Conventional holographic display suffers from the conjugate light issue. In this Letter, we propose to efficiently extend the eyebox of holographic Maxwellian near-eye display by encoding the conjugate wavefront as the multiplication of plane wave phase with the target image. It is interesting that after being focused by the lens, the generated conjugate viewpoints also present erect virtual images with the same image quality as the signal viewpoints. Multiple plane wave encoding is used for eyebox extension, and, because of the utilization of conjugate light, the effect of eyebox extension is doubled. That is, the space bandwidth of the amplitude-type hologram is fully used. A speckless holographic image is produced in mid-air with high quality within a large depth range. The proposed display is compact and promising for the augmented reality near-eye display. Furthermore, it may inspire better solutions for the conjugate light issue of amplitude-type holography.

Hologram computation using the radial point spread function

Holograms are computed by superimposing point spread functions (PSFs), which represent the distribution of light on the hologram plane. The computational cost and the spatial bandwidth product required to generate holograms are significant; therefore, it is challenging to compute high-resolution holograms at the rates required for videos. Among the possible displays, fixed-eye-position holographic displays, such as holographic head-mounted displays, reduce the spatial bandwidth product by fixing eye positions while satisfying almost all human depth cues. In eye-fixed holograms, by calculating a part distribution of the entire PSF, we observe reconstructed images that maintain the image quality and the depth of focus almost as high as those generated by the entire PSF. In this study, we accelerate the calculation of eye-fixed holograms by engineering the PSFs. We propose cross and radial PSFs, and we determine that, out of the two, the radial PSFs have a better image quality. By combining the look-up table method and the wavefront-recording plane method with radial PSFs, we show that the proposed method can rapidly compute holograms.

Lensless full-color holographic Maxwellian near-eye display with a horizontal eyebox expansion

A lensless full-color holographic Maxwellian near-eye display using a single amplitude-type spatial light modulator is proposed in this Letter. The color holographic image is directly projected onto the retina without any eyepiece. The color crosstalk is clearly separated from the signal in the space owing to the encoded spherical wave and carrier wave. An aperture numerical filter and a real polarized filter are used at the pupil plane to accurately stop the crosstalk light. A high-quality dynamic speckless color holographic image was produced in the mid-air within a specific depth range. The horizontal eyebox expansion is achieved simply through multiple spherical wave encoding and verified through an optical experiment. The proposed display is compact and promising as the augmented reality near-eye display.

Metalens Eyepiece for 3D Holographic Near-Eye Display

Near-eye display (NED) systems for virtual reality (VR) and augmented reality (AR) have been rapidly developing; however, the widespread use of VR/AR devices is hindered by the bulky refractive and diffractive elements in the complicated optical system as well as the visual discomfort caused by excessive binocular parallax and accommodation-convergence conflict. To address these problems, an NED system combining a 5 mm diameter metalens eyepiece and a three-dimensional (3D), computer-generated holography (CGH) based on Fresnel diffraction is proposed in this paper. Metalenses have been extensively studied for their extraordinary capabilities at wavefront shaping at a subwavelength scale, their ultrathin compactness, and their significant advantages over conventional lenses. Thus, the introduction of the metalens eyepiece is likely to reduce the issue of bulkiness in NED systems. Furthermore, CGH has typically been regarded as the optimum solution for 3D displays to overcome limitations of binocular systems, since it can restore the whole light field of the target 3D scene. Experiments are carried out for this design, where a 5 mm diameter metalens eyepiece composed of silicon nitride anisotropic nanofins is fabricated with diffraction efficiency and field of view for a 532 nm incidence of 15.7% and 31°, respectively. Furthermore, a novel partitioned Fresnel diffraction and resample method is applied to simulate the wave propagations needed to produce the hologram, with the metalens capable of transforming the reconstructed 3D image into a virtual image for the NED. Our work combining metalens and CGH may pave the way for portable optical display devices in the future.

Extending eyebox with tunable viewpoints for see-through near-eye display

The Maxwellian display presents always-focused images to the viewer, alleviating the vergence-accommodation conflict (VAC) in near-eye displays (NEDs). However, the limited eyebox of the typical Maxwellian display prevents it from wider applications. We propose a Maxwellian see-through NED based on a multiplexed holographic optical element (HOE) and polarization gratings (PGs) to extend the eyebox by viewpoint multiplication. The multiplexed HOE functions as multiple convex lenses to form multiple viewpoints, which are copied to different locations by PGs. To mitigate the imaging problem that multiple viewpoints or no viewpoints enter the eye pupil, the viewpoints can be tuned by mechanically moving a PG. We implement our method in a proof-of-concept system. The optical experiments confirm that the proposed display system provides always in-focus images within a 12 mm eyebox in the horizontal direction with a 32.7° diagonal field of view (FOV) and a 16.5 mm eye relief (ERF), and its viewpoints are tunable to match the actual eye pupil size. Compared with other techniques to extend the eyebox of Maxwellian displays, the proposed method shows competitive performances of a large eyebox, adaptability to the eye pupil size, and focus cues within a large depth range.

Super achromatic wide-angle quarter-wave plates using multi-twist retarders

The achromaticity and wide-angle property of quarter-wave plates (QWPs) are crucial for the color uniformity and image resolution of the future displays such as virtual reality (VR) pancake lens and augmented reality (AR) waveguide/focusing systems. However, most reported achromatic wide-angle QWPs designs composed by stacks of different birefringent plates are too complicated with limited achromaticity and wide-angle performance. The multi-twist retarders (MTR) QWPs presented in previous work already showed its potential to achieve high achromaticity in RGB using one monolithic film in normal incidence, but the incompetent polarization control in blue-violet limits its application in LED-based polarization-sensitive AR/VR headsets. In this work, we theoretically investigate a new type of MTR QWPs achieving super achromaticity from violet to red with average ellipticity 43^° and simultaneously maintaining wide-viewing angle up to ±45^°, which enables a precise polarization control within the field-of-view (FOV) of current AV/VR headset. The new proposed MTR QWP is also reported to obtain average reflection luminance leakage 0.15~% and maximum leakage 0.23~%, making it a promising element to reduce polarization leakage and enhance image resolution in the next-generation displays.

Diffraction Efficiency Characteristics for MEMS-Based Phase-Only Spatial Light Modulator with Nonlinear Phase Distribution

Micro-electro mechanical systems (MEMS)-based phase-only spatial light modulators (PLMs) have the potential to overcome the limited speed of liquid crystal on silicon (LCoS) spatial light modulators (SLMs) and operate at speeds faster than 10 kHz. This expands the practicality of PLMs to several applications, including communications, sensing, and high-speed displays. The complex structure and fabrication requirements for large, 2D MEMS arrays with vertical actuation have kept MEMS-based PLMs out of the market in favor of LCoS SLMs. Recently, Texas Instruments has adapted its existing DMD technology for fabricating MEMS-based PLMs. Here, we characterize the diffraction efficiency for one of these PLMs and examine the effect of a nonlinear distribution of addressable phase states across a range of wavelengths and illumination angles.

Hybrid holographic Maxwellian near-eye display based on spherical wave and plane wave reconstruction for augmented reality display

The holographic Maxwellian display is a promising technique for augmented reality (AR) display because it solves the vergence-accommodation conflict while presenting a high-resolution display. However, conventional holographic Maxwellian display has the inherent trade-off between depth of field (DOF) and image quality. In this paper, two types of holographic Maxwellian displays, the spherical wave type and the plane wave type, are proposed and analyzed. The spherical wavefront and the plane wavefront are produced by a spatial light modulator (SLM) for Maxwellian display. Due to the focusing properties of different wavefronts, the two types of display have complementary DOF ranges. A hybrid approach combining the spherical wavefront and plane wavefront is proposed for a large DOF with high image quality. An optical experiment with AR display is demonstrated to verify the proposed method.

HoloBlade: an open-hardware spatial light modulator driver platform for holographic displays

Spatial light modulators (SLMs) are key research tools in several contemporary applied optics research domains. In this paper, we present the argument that an open platform for interacting with SLMs would dramatically increase their accessibility to researchers. We introduce HoloBlade, an open-hardware implementation of an SLM driver-stack, and provide a detailed exposition of HoloBlade's architecture, key components, and detailed design. An optical verification rig is constructed to demonstrate that HoloBlade can provide Fourier imaging capability in a 4f system. Finally, we discuss HoloBlade's future development roadmap and the opportunities that it presents as a research tool for applied optics.

See-through holographic display with randomly distributed partial computer generated holograms

Holographic displays have the feature to show images out of the plane of the device itself, which is especially favored for augmented reality (AR) applications where the images need to be merged with the real world. In existing cases of AR holographic display, a combiner is used to converge the light path of the display image and surrounding scene toward the viewer's eye. In this paper, the idea of combining the holographic device and the combiner has been proposed, resulting in a see-through holographic display. In order to maintain the see-through quality of the device, the concept of partial hologram has been introduced, which means only a part of the area on the device has the holographic fringe pattern while leaving the rest fully transparent. Experiment and theoretical investigation shows that an evenly yet randomly distributed partial hologram provides the best holographic image quality assuming a fixed percentage of the holographic area on the device. A passive computer generated hologram (CGH) with two phase levels has been designed and fabricated for the verification. With partial hologram sharing 25% of the whole area, the CGH exhibits 90.9% of total transmission and 72.2% of parallel transmission. The demonstration shows a high see-through quality while providing a clear holographic image.

Glued diffraction optical elements with broadband and a large field of view

High diffraction efficiency is an important requirement for hybrid diffractive-refractive optical systems with a wide field of view. The issue is that diffractive optical elements cannot maintain high diffraction efficiency across a designed waveband and range of incident angles simultaneously. Glued diffractive optical elements (GDOEs) consist of two single-layer diffractive elements, and optical adhesives are presented to address the problem. Two diffractive optical elements are glued together to reduce the straylight scattered into unwanted diffraction orders. The parameters of diffractive optical elements are optimized to achieve broadband high diffraction efficiency and modulation transfer function over a wide-incident-angle range. The GDOEs enable the system to realize a diffraction efficiency of over 90% when the incident angle is no more than 58°. Through gluing two single-layer diffractive optical elements together, we can minimize the inner reflection and refraction. Diffraction efficiency losses can be compensated by the optical adhesives layer, and image quality can be improved. Our design method could make possible the use of diffraction elements in different kinds of optical systems.

Enlarging field of view by a two-step method in a near-eye 3D holographic display

The narrow field of view (FOV) has always been one of the most with limitations that drag the development of holographic three-dimensional (3D) near-eye display (NED). The complex amplitude modulation (CAM) technique is one way to realize holographic 3D display in real time with the advantage of high image quality. Previously, we applied the CAM technique on the design and integration of a compact colorful 3D-NED system. In this paper, a viewing angle enlarged CAM based 3D-NED system using a Abbe-Porter scheme and curved reflective structure is proposed. The viewing angle is increased in two steps. An Abbe-Porter filter system, composed of a lens and a grating, is used to enlarge the FOV for the first step and, meanwhile, realize complex amplitude modulation. A curved reflective structure is used to realize the FOV enlargement for the second step. Besides, the system retains the ability of colorful 3D display with high image quality. Optical experiments are performed, and the results show the system could present a 45.2° diagonal viewing angle. The system is able to present dynamic display as well. A compact prototype is fabricated and integrated for wearable and lightweight design.