Roadmap on digital holography [Invited]

Holographic tomography of the diatom Skeletonema pseudocostatum used as a bioindicator of heavy metal-polluted waters

Heavy metal contamination in aquatic environments poses a significant threat to microbial communities, yet the subcellular responses of phytoplankton to metal stress remain poorly understood. In particular, the effects of heavy metal exposure on the structural and physiological properties of diatoms require further investigation. Here, we analyze the impact of cadmium (Cd) and copper (Cu) exposure on the subcellular structures of the diatom Skeletonema pseudocostatum using holographic tomography. This imaging technique enables detailed visualization and quantitative analysis of diatom subcomponents, including frustules, protoplasm, vacuoles, and chloroplasts, under varying metal concentrations. The study aims to understand the changes in the mean refractive index (RI) and concentration (e.g., the ratio among cell dry mass and its biovolume) as indicators of cellular response to metal stress and to infer if such diatom can be used as sentinel species of heavy metal pollution. Findings indicate that diatoms exhibit significant variations in RI and internal cell density when exposed to different metal concentrations. Lower RI values observed at higher metal concentrations, can be considered as a sign of stress due to cytoplasm extrusion and/or vacuolization. The results highlight the potential of using S. pseudocostatum as a bioindicator for monitoring water metal pollution. Moreover, the results show that holographic tomography as useful tool for non-invasive, high-resolution cellular imaging of phytoplankton in environmental studies.

Spatiotemporal heterodyne multiplexing in digital holography for realizing broad recordable bandwidth

To realize high-capacity holographic multiplexing, we propose an off-axis frequency-modulated continuous-wave digital holography (FMCW-DH) that utilizes both temporal and spatial frequencies as the multiplexable bandwidth. The temporal and spatial carrier frequencies for the multiplexed wavefronts are achieved by the frequency-modulated continuous-wave technique of the light source and the off-axis configuration of digital holography, respectively. The effectiveness of the proposed technique is demonstrated by multiplexing the multiple holograms generated by illuminating two objects at two wavelengths. Also, the phase distribution maps reconstructed from the recorded complex wavefronts were used to measure the surface profile of the known step height by using the two-wavelength method.

Diatom Lensless Imaging Using Laser Scattering and Deep Learning

We present a novel approach for imaging diatoms using lensless imaging and deep learning. We used a laser beam to scatter off samples of diatomaceous earth (diatoms) and then recorded and transformed the scattered light into microscopy images of the diatoms. The predicted microscopy images gave an average SSIM of 0.98 and an average RMSE of 3.26 as compared to the experimental data. We also demonstrate the capability of determining the velocity and angle of movement of the diatoms from their scattering patterns as they were translated through the laser beam. This work shows the potential for imaging and identifying the movement of diatoms and other microsized organisms in situ within the marine environment. Implementing such a method for real-time image acquisition and analysis could enhance environmental management, including improving the early detection of harmful algal blooms.

Phase aberration compensation and parasitic fringes elimination in digital holographic microscopy based on polarization

In digital holographic microscopy, parasitic fringes caused by optical components and phase aberration introduced by the optical system are crucial issues that constrain measurement accuracy and reconstructed image quality. This paper presents a straightforward and effective physical approach to simultaneously compensate aberration and eliminate parasitic fringes in reflective holographic microscopy. By modulating the polarization states of both the parasitic beams and the sample beam, combined with a polarized beam splitter element, parasitic fringes can be efficiently eliminated. An improved reflective double exposure optical configuration is integrated into the proposed holographic microscopy. A criterion based on the number of interference fringes is developed to ensure the consistency between the phase aberrations recorded by the flat mirror and that recorded by the sample, which significantly improves the accuracy and robustness of reflective double exposure methods. Experimental results of a terahertz chip and a SoC chip demonstrate that the proposed method can eliminate arbitrary parasitic fringes while preserving image details, which is a challenge with traditional image filtering methods. Moreover, the proposed improved reflective double exposure method can compensate all aberrations regardless of the sample's morphology, without the need for complex numerical computations, prior knowledge of the morphology, or the troublesome and challenging optical alignment process.

Open-source implementation of polarisation-resolved single-shot differential phase contrast microscopy (pDPC) on a modular openFrame-based microscope

We recently demonstrated polarisation differential phase contrast microscopy (pDPC) as a robust, low-cost single-shot implementation of (semi)quantitative phase imaging based on differential phase microscopy. pDPC utilises a polarisation-sensitive camera to simultaneously acquire four obliquely transilluminated images from which phase images mapping spatial variation of optical path difference can be calculated. pDPC microscopy can be implemented on existing or bespoke microscopes and can utilise radiation at a wide range of visible to near infrared wavelengths and so is straightforward to integrate with fluorescence microscopy. Here we present a low-cost open-source pDPC module that is designed for use with the modular open-source microscope stand "openFrame". With improved hardware and software, this new pDPC implementation provides a real-time readout of phase across a field of view that facilitates optimisation of system alignment. We also provide protocols for background subtraction and correction of crosstalk.

Motion Hologram: Jointly optimized hologram generation and motion planning for photorealistic 3D displays via reinforcement learning

Holography is capable of rendering three-dimensional scenes with full-depth control and delivering transformative experiences across numerous domains, including virtual and augmented reality, education, and communication. However, traditional holography presents 3D scenes with unnatural defocus and severe speckles due to the limited space bandwidth product of the spatial light modulator (SLM). Here, we introduce Motion Hologram, a holographic technique that accurately portrays photorealistic and speckle-free 3D scenes. This technique leverages a single hologram and a learnable motion trajectory, which are jointly optimized within a deep reinforcement learning framework. Specifically, we experimentally demonstrated that the proposed technique could achieve a 4- to 5-dB PSNR improvement of focal stacks in comparison with traditional holography and could successfully depict speckle-free, high-fidelity, and full-color 3D displays using only a commercial SLM. We believe that the proposed method promises a prospective form of holographic displays that will offer immersive viewing experiences for audiences.

Direct object recovery from the Fraunhofer diffraction integral

In this Letter, we present a novel, to the best of our knowledge, approach for recovering objects directly from the Fraunhofer diffraction integral, where the diffraction field of an object is approximated by the Fourier transform of this object augmented by an additional phase factor. This phase factor at the observation plane is universal for the diffraction fields generated by objects located at the same plane and illuminated by the same monochromatic plane wave. It can be first extracted from dividing the Fraunhofer diffraction field by the Fourier transform of an object reference. Rapid recovery for unknown objects is then enabled after applying a two-dimensional inverse Fourier transform to the ratio of the Fraunhofer diffraction fields to the phase factor. This approach is verified experimentally by constructing a modified Mach-Zehnder interferometer, with a digital micromirror device (DMD) generating the objects of desired structures. To record the Fraunhofer diffraction and interference patterns on a finite-size CCD camera, a convex lens is introduced with the CCD sensing surface positioned at the focal plane of the lens. The strategy described in [Nat. Commun.7, 10820 (2016)10.1038/ncomms10820] is applied to extract the phase of the diffraction field from the interference pattern. The results demonstrate the efficiency of our approach in swiftly and accurately recovering small objects with elimination of zero-order and conjugate images.

Generalizable deep learning approach for 3D particle imaging using holographic microscopy (HM)

Despite its potential for label-free particle diagnostics, holographic microscopy is limited by specialized processing methods that struggle to generalize across diverse settings. We introduce a deep learning architecture leveraging human perception of longitudinal variation of diffracted patterns of particles, which enables highly generalizable analysis of 3D particle information with orders of magnitude improvement in processing speed. Trained with minimal synthetic and real holograms of simple particles, our method demonstrates exceptional performance across various challenging cases, including high particle concentrations, significant noise, and a wide range of particle sizes, complex shapes, and optical properties, exceeding the diversity of training datasets.

Visualizing the fine structure and dynamics of living cells with temporal polychromatic digital holographic microscopy

Polychromatic digital holographic microscopy (P-DHM) has demonstrated its capacity to generate highly denoised optical path difference images, thereby enabling the label-free visualization of fine cellular structures, such as the dendritic arborization within neuronal cells in culture. So far, however, the sample must remain more or less stationary since P-DHM is performed manually, i.e., all actions are carried out sequentially over several minutes. In this paper, we propose fully automated, robust, and efficient management of the acquisition and reconstruction of the time series of polychromatic hologram sets, transforming P-DHM into temporal P-DHM. Experimental results have demonstrated the ability of the proposed temporal P-DHM implementation to non-invasively and quantitatively reveal the fine structure and dynamics of living cells.

Holography optimization based on combining iterative Green's function algorithm and deep learning method

In this Letter, we present a novel, to the best of our knowledge, approach that combines a new numerical iterative algorithm with a physics-informed neural network (PINN) architecture to solve the Helmholtz equation, thereby achieving highly generalized refractive index modulation holography. Firstly, we design a non-uniform refractive index convolutional neural network (NRI-CNN) to modify the refractive index and extract a feature vector. Then we propose an iterative Green's function algorithm (IGFA) to approximately solve the Helmholtz equation. In order to enhance the generalization ability of the solution, the abstracted vector is utilized as a multiplier term in IGFA, obtaining an approximately spatial distribution of the light field. Ultimately, we design a U-net to handle residuals of the Helmholtz equation and phases of optical fields (ERPU-net). We apply this method for holographic reconstructions on random Gaussian beams, beams with image data, and those altered by simulated turbulent phases.

Phase recovery from Fresnel incoherent correlation holography using differential Zernike fitting

Fresnel incoherent correlation holography (FINCH) was created to improve imaging resolution and 3D imaging capabilities using spatially incoherent illumination. The optical setup of a FINCH-based interferometer is closely related to a radial shearing interferometer, which measures the radial phase difference of an input wavefront. By using phase retrieval methodologies from lateral shearing interferometry, namely, differential Zernike fitting (DZF), we show that FINCH-based and radial shearing interferometry can be used for phase retrieval and adaptive optics (AO). In this paper, we describe the phase retrieval algorithm using least squares-based DZF and demonstrate a simple adaptive optics loop with an aberrated point spread function using wave optics simulation. We find that FINCH-based phase retrieval has the advantages of fast phase retrieval measurements, thanks to well-studied least squares-based phase reconstruction methods, improved resolution compared to the Shack-Hartmann-based wavefront sensing, and the simplified optical setup of radial shearing interferometry.

Roadmap on computational methods in optical imaging and holography [invited]

Computational methods have been established as cornerstones in optical imaging and holography in recent years. Every year, the dependence of optical imaging and holography on computational methods is increasing significantly to the extent that optical methods and components are being completely and efficiently replaced with computational methods at low cost. This roadmap reviews the current scenario in four major areas namely incoherent digital holography, quantitative phase imaging, imaging through scattering layers, and super-resolution imaging. In addition to registering the perspectives of the modern-day architects of the above research areas, the roadmap also reports some of the latest studies on the topic. Computational codes and pseudocodes are presented for computational methods in a plug-and-play fashion for readers to not only read and understand but also practice the latest algorithms with their data. We believe that this roadmap will be a valuable tool for analyzing the current trends in computational methods to predict and prepare the future of computational methods in optical imaging and holography.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00340-024-08280-3.

Single-frame transmission and phase imaging using off-axis holography with undetected photons

Imaging with undetected photons relies upon nonlinear interferometry to extract the spatial image from an infrared probe beam and reveal it in the interference pattern of an easier-to-detect visible beam. Typically, the transmission and phase images are extracted using phase-shifting techniques and combining interferograms from multiple frames. Here we show that off-axis digital holography enables reconstruction of both transmission and phase images at the infrared wavelength from a single interferogram, and hence a single frame, recorded in the visible. This eliminates the need for phase stepping and multiple acquisitions, thereby greatly reducing total measurement time for imaging with long acquisition times at low flux or enabling video-rate imaging at higher flux. With this single-frame acquisition technique, we are able to reconstruct transmission images of an object in the infrared beam with a signal-to-noise ratio of 3.680 ± 0.004 at 10 frames per second, and record a dynamic scene in the infrared beam at 33 frames per second.

Viewing early life without labels: optical approaches for imaging the early embryo†

Embryo quality is an important determinant of successful implantation and a resultant live birth. Current clinical approaches for evaluating embryo quality rely on subjective morphology assessments or an invasive biopsy for genetic testing. However, both approaches can be inherently inaccurate and crucially, fail to improve the live birth rate following the transfer of in vitro produced embryos. Optical imaging offers a potential non-invasive and accurate avenue for assessing embryo viability. Recent advances in various label-free optical imaging approaches have garnered increased interest in the field of reproductive biology due to their ability to rapidly capture images at high resolution, delivering both morphological and molecular information. This burgeoning field holds immense potential for further development, with profound implications for clinical translation. Here, our review aims to: (1) describe the principles of various imaging systems, distinguishing between approaches that capture morphological and molecular information, (2) highlight the recent application of these technologies in the field of reproductive biology, and (3) assess their respective merits and limitations concerning the capacity to evaluate embryo quality. Additionally, the review summarizes challenges in the translation of optical imaging systems into routine clinical practice, providing recommendations for their future development. Finally, we identify suitable imaging approaches for interrogating the mechanisms underpinning successful embryo development.

Holographic phase retrieval via Wirtinger flow: Cartesian form with auxiliary amplitude

We propose a new gradient method for holography, where a phase-only hologram is parameterized by not only the phase but also amplitude. The key idea of our approach is the formulation of a phase-only hologram using an auxiliary amplitude. We optimize the parameters using the so-called Wirtinger flow algorithm in the Cartesian domain, which is a gradient method defined on the basis of the Wirtinger calculus. At the early stage of optimization, each element of the hologram exists inside a complex circle, and it can take a large gradient while diverging from the origin. This characteristic contributes to accelerating the gradient descent. Meanwhile, at the final stage of optimization, each element evolves along a complex circle, similar to previous state-of-the-art gradient methods. The experimental results demonstrate that our method outperforms previous methods, primarily due to the optimization of the amplitude.

Lensless single-shot dual-wavelength digital holography for industrial metrology

We demonstrate lensless single-shot dual-wavelength digital holography for high-speed 3D imaging in industrial inspection. Single-shot measurement is realized by combining off-axis digital holography and spatial frequency multiplexing of the two wavelengths on the detector. The system has 9.1 µm lateral resolution and a 50 µm unambiguous depth range. We determine the theoretical accuracy of off-axis dual-wavelength phase reconstruction for the case of shot-noise-limited detection. Experimental results show good agreement with the proposed model. The system is applied to industrial metrology of calibrated test samples and chip manufacturing.

Autofocusing in digital holography based on an adaptive genetic algorithm

In digital holography (DH), determining the reconstruction distance is critical to the quality of the reconstructed image. However, traditional focal plane detection methods require considerable time investment to reconstruct and evaluate holograms at multiple distances. To address this inefficiency, this paper proposes a fast and accurate autofocusing method based on an adaptive genetic algorithm. This method only needs to find several reconstruction distances in the search area as an initial population, and then adaptively optimize the reconstruction distance through iteration to determine the optimal focal plane in the search area. In addition, an off-axis digital holographic optical system was used to capture the holograms of the USAF resolution test target and the coin. The simulation and experimental results indicated that, compared with the traditional autofocusing, the proposed method can reduce the computation time by about 70% and improve the focal plane accuracy by up to 0.5 mm.

Quantitative phase image stitching guided by reconstructed intensity images in one-shot double field of view multiplexed digital holographic microscopy

In digital holographic microscopy (DHM), achieving large field of view (FOV) imaging while maintaining high resolution is critical for quantitative phase measurements of biological cell tissues and micro-nano structures. We present a quantitative phase image stitching guided by reconstructed intensity images in one-shot double FOV multiplexed DHM. Double FOVs are recorded simultaneously through frequency division multiplexing; intensity feature pairs are accurately extracted by multi-algorithm fusion; aberrations and non-common baselines are effectively corrected by preprocessing. Experimental results show that even if phase images have coherent noise, complex aberrations, low overlap rate and large size, this method can achieve high-quality phase stitching.

Twin-stagnation-free phase retrieval with vortex phase illumination

The recovery of a complex-valued exit wavefront from its Fourier transform magnitude is challenging due to the stagnation problems associated with iterative phase retrieval algorithms. Among the various stagnation artifacts, the twin-image stagnation is the most difficult to address. The upright object and its inverted and complex-conjugated twin correspond to the identical Fourier magnitude data and hence appear simultaneously in the iterative solution. We show that the twin stagnation problem can be eliminated completely if a coherent beam with charge-1 vortex phase is used for illumination. Unlike the usual plane wave illumination case, a charge-1 vortex illumination intentionally introduces an isolated zero near the zero spatial frequency region, where maximal energy in the Fourier space is usually concentrated for most natural objects. The early iterations of iterative phase retrieval algorithms are observed to develop a clockwise or anti-clockwise vortex in the vicinity of this isolated zero. Once the Fourier transform of the solution latches onto a specific vortex profile in the neighborhood of this intentionally introduced intensity zero in early iterations, the solution quickly adjusts to the corresponding twin (upright or inverted) and further iterations are not observed to bring the other twin into the reconstruction. Our simulation studies with the well-known hybrid input-output (HIO) algorithm show that the solution always converges to one of the twins within a few hundred iterations when vortex phase illumination is used. Using a clockwise or anti-clockwise vortex phase as an initial guess is also seen to deterministically lead to a solution consisting of the corresponding twin. The resultant solution still has some faint residual artifacts that can be addressed via the recently introduced complexity guidance methodology. There is an additional vortex phase in the final solution that can simply be subtracted out to obtain the original test object. The near guaranteed convergence to a twin-stagnation-free solution with vortex illumination as described here is potentially valuable for deploying practical imaging systems that work based on the iterative phase retrieval algorithms.

Highly-efficient full-color holographic movie based on silicon nitride metasurface

Metasurface holograms offer various advantages, including wide viewing angle, small volume, and high resolution. However, full-color animation of high-resolution images has been a challenging issue. In this study, a full-color dielectric metasurface holographic movie with a resolution of 2322 × 2322 was achieved by spatiotemporally multiplexing 30 frames with blue, green, and red color channels at the wavelengths of 445 nm, 532 nm, and 633 nm at the maximum reconstruction speed of 55.9 frames per second. The high average transmittance and diffraction efficiency of 92.0 % and 72.7 %, respectively, in the visible range, were achieved by adopting polarization-independent silicon nitride waveguide meta-atoms, resulting in high color reproducibility. The superposition of three wavelengths was achieved by adjusting the resolutions and positions of target images for each wavelength while maintaining the meta-atom pitch constant. The improvement in diffraction efficiency was brought about by the optimization of etching conditions to form high-aspect vertical nanopillar structures.

Hyperspectral digital holography realized by using an electro-optical frequency comb via injection locking

Hyperspectral digital holography (HSDH) is a versatile holographic imaging technique that offers large unambiguous depth range and spectroscopic information. In this Letter, we propose a novel, to the best of our knowledge, HSDH system that is realized by using an electro-optical frequency comb (EOFC) via injection locking. In comparison with conventional dual-comb HSDH, the proposed system only requires one EOFC and few other devices, which not only simplifies the system structure and reduces the cost but also improves the imaging speed. We validated the system using an EOFC with 20 optical frequencies spaced at 18 GHz intervals. In a total measurement time of 0.5 s, we successfully captured images of two targets that were 0.74 mm apart without phase ambiguity and obtained the transmission spectrum of an absorbing gas simultaneously. This work provides valuable insights for HSDH systems relying on an optical frequency comb.

Carrier-frequency estimation for digital holograms of phase objects

Accurate estimation of carrier fringe frequency is essential for the demodulation of off-axis digital holograms. The fringe frequency is often associated with the amplitude peak of the cross-term in the two-dimensional Fourier transform of a digital hologram. We point out that this definition of carrier frequency is not valid in general for holograms associated with phase objects. We examine the carrier-envelope representation for digital holograms from the viewpoint of Mandel's criterion [J. Opt. Soc. Am.57, 613 (1967)10.1364/JOSA.57.000613]. An appropriate definition of carrier frequency is observed to be the centroid of the power spectrum associated with the cross term. This definition is shown to apply uniformly to holograms associated with phase objects, is robust to noise, and leads to the smoothest (or least fluctuating) envelope representation for the demodulated object wave. The proposed definition is illustrated with simulated as well as experimentally recorded off-axis holograms.

Measuring spectral extinction with digital holography

The optical extinction caused by a small particle, such as an aerosol particle, is an important measurable quantity. Understanding the influence of atmospheric aerosols on the climate, assessing visibility in urban environments, and remote sensing applications such as lidar all need accurate measurements of particle extinction. While multiple methods are known to measure extinction, digital in-line holography (DIH) features the unique ability to provide contact-free images of particles simultaneously with estimates for the extinction cross section. This is achieved through an integration of a measured hologram followed by an extrapolation. By means of a supercontinuum laser, we investigate the measurement of the cross section via DIH for stationary particles across a broad spectrum, from 440 nm to 1040 nm. The particles considered include a 50 µm glass microsphere, a volcanic ash particle, and an iron(III) oxide particle. The results show the ability to estimate a particle's cross section to within 10% error across portions of the spectrum and approximately 20% error otherwise. An examination of the accompanying hologram-derived particle images reveals details in the images that evolve with wavelength. The behavior suggests a basic means to resolve whether absorption or scattering dominates a particle's extinction.

General phase-difference imaging of incoherent digital holography

The hologram formed by incoherent holography based on self-interference should preserve the phase difference information of the object, such as the phase difference between the mutually orthogonal polarizations of anisotropic object. How to decode this phase difference from this incoherent hologram, i.e., phase-difference imaging, is of great significance for studying the properties of the measured object. However, there is no general phase-difference imaging theory due to both diverse incoherent holography systems and the complicated reconstruction process from holograms based on the diffraction theory. To realize phase-difference image in incoherent holography, the relationship between the phase difference of the object and the image reconstructed by holograms is derived using a general physical model of incoherent holographic systems, and then the additional phase that will distort this relationship in actual holographic systems is analyzed and eliminated. Finally, the phase-difference imaging that is suitable for the most incoherent holographic systems is realized and the general theory is experimentally verified. This technology can be applied to phase-difference imaging of anisotropic objects, and has potential applications in materials science, biomedicine, polarized optics and other fields.

Fourier Ptychographic Microscopy 10 Years on: A Review

Fourier ptychographic microscopy (FPM) emerged as a prominent imaging technique in 2013, attracting significant interest due to its remarkable features such as precise phase retrieval, expansive field of view (FOV), and superior resolution. Over the past decade, FPM has become an essential tool in microscopy, with applications in metrology, scientific research, biomedicine, and inspection. This achievement arises from its ability to effectively address the persistent challenge of achieving a trade-off between FOV and resolution in imaging systems. It has a wide range of applications, including label-free imaging, drug screening, and digital pathology. In this comprehensive review, we present a concise overview of the fundamental principles of FPM and compare it with similar imaging techniques. In addition, we present a study on achieving colorization of restored photographs and enhancing the speed of FPM. Subsequently, we showcase several FPM applications utilizing the previously described technologies, with a specific focus on digital pathology, drug screening, and three-dimensional imaging. We thoroughly examine the benefits and challenges associated with integrating deep learning and FPM. To summarize, we express our own viewpoints on the technological progress of FPM and explore prospective avenues for its future developments.

Impact of focus cue presentation on perceived realism of 3-D scene structure: Implications for scene perception and for display technology

Stereoscopic imagery often aims to evoke three-dimensional (3-D) percepts that are accurate and realistic-looking. The "gap" between 3-D imagery and real scenes is small, but focus cues typically remain incorrect because images are displayed on a single focal plane. Research has concentrated on the resulting vergence-accommodation conflicts. Yet, incorrect focus cues may also affect the appearance of 3-D imagery. We investigated whether incorrect focus cues reduce perceived realism of 3-D structure ("depth realism"). Experiment 1 used a multiple-focal-planes display to compare depth realism with correct focus cues vs. conventional stereo presentation. The stimuli were random-dot stereograms, which isolated the role of focus cues. Depth realism was consistently lower with incorrect focus cues, providing proof-of-principle evidence that they contribute to perceptual realism. Experiments 2 and 3 examined whether focus cues play a similar role with realistic objects, presented with an almost complete set of visual cues using a high-resolution, high-dynamic-range multiple-focal-planes display. We also examined the efficacy of approximating correct focus cues via gaze-contingent depth-of-field rendering. Improvements in depth realism with correct focus cues were less clear in more realistic scenes, indicating that the role of focus cues in depth realism depends on scene content. Rendering-based approaches, if anything, reduced depth realism, which we attribute to their inability to present higher-order aspects of blur correctly. Our findings suggest future general 3-D display solutions may need to present focus cues correctly to maximise perceptual realism.

Spherical wave illumination scanning digital holographic profilometry

In this work, we proposed what we believe to be a novel scanning solution for the assessment of high-NA samples, referred to as spherical-wave illumination scanning digital holographic profilometry (SWS-DHP). This approach introduces a 2F optimization methodology, based on the measurement of the focal length of the object to determine the spherical component of the scanning. Furthermore, re-optimization of 2F, whether it needs to be operated depends on the measured object's NA to inspect more information. Meanwhile, utilizing phase space analysis shows SWS superiority in information transfer for high-NA samples compared to plane-wave illumination scanning. In addition, this method introduces a shape reconstruction algorithm with volumetric aberration compensation based on the propagation of the aberrated object and illumination waves to obtain high-quality measurements. Finally, the imaging merits of SWS-DHP were proved through simulations and were experimentally verified for the object of NA up to 0.87.

Understanding the nervous system: lessons from Frontiers in Neurophotonics

The Frontiers in Neurophotonics Symposium is a biennial event that brings together neurobiologists and physicists/engineers who share interest in the development of leading-edge photonics-based approaches to understand and manipulate the nervous system, from its individual molecular components to complex networks in the intact brain. In this Community paper, we highlight several topics that have been featured at the symposium that took place in October 2022 in Québec City, Canada.

In-focus quantitative phase imaging from defocused off-axis holograms: synergistic reconstruction framework

Digital holographic microscopy (DHM) enables the three-dimensional (3D) reconstruction of quantitative phase distributions from a defocused hologram. Traditional computational algorithms follow a sequential approach in which one first reconstructs the complex amplitude distribution and later applies focusing algorithms to provide an in-focus phase map. In this work, we have developed a synergistic computational framework to compensate for the linear tilt introduced in off-axis DHM systems and autofocus the defocused holograms by minimizing a cost function, providing in-focus reconstructed phase images without phase distortions. The proposed computational tool has been validated in defocused holograms of human red blood cells and three-dimensional images of dynamic sperm cells.

Simple high-resolution 3D microscopy by a dielectric microsphere: a proof of concept

We present a simple high-resolution approach for 3D and quantitative phase imaging (QPI). Our method makes the most of a glass microsphere (MS) for microscopy and a glass plate for lateral shearing self-referencing interferometry. The single MS serves all the functions of a microscope objective (MO) in digital holographic microscopy (DHM) while offering the advantages of compactness, lightness, and affordability. A proof-of-concept experiment is performed on a standard diffraction grating, and various effective parameters on the imaging performance are investigated. The results are validated by atomic force microscopy and Mirau-DHM, and 3D morphometric information of the sample under inspection is obtained. The technique is then applied for 3D quantitative measurement and visualization of a human red blood cell, proving the principle of our easy-to-implement and vibration-immune arrangement for high-contrast label-free QPI of biological samples, and its utility in cell morphology, identification, and classification.

A Novel Image Processing Method for Obtaining an Accurate Three-Dimensional Profile of Red Blood Cells in Digital Holographic Microscopy

Recently, research on disease diagnosis using red blood cells (RBCs) has been active due to the advantage that it is possible to diagnose many diseases with a drop of blood in a short time. Representatively, there are disease diagnosis technologies that utilize deep learning techniques and digital holographic microscope (DHM) techniques. However, three-dimensional (3D) profile obtained by DHM has a problem of random noise caused by the overlapping DC spectrum and sideband in the Fourier domain, which has the probability of misjudging diseases in deep learning technology. To reduce random noise and obtain a more accurate 3D profile, in this paper, we propose a novel image processing method which randomly selects the center of the high-frequency sideband (RaCoHS) in the Fourier domain. This proposed algorithm has the advantage of filtering while using only recorded hologram information to maintain high-frequency information. We compared and analyzed the conventional filtering method and the general image processing method to verify the effectiveness of the proposed method. In addition, the proposed image processing algorithm can be applied to all digital holography technologies including DHM, and in particular, it is expected to have a great effect on the accuracy of disease diagnosis technologies using DHM.

Digital holographic microscopy of spiropyran-based dynamic materials

Spiropyran (SP)-based dynamic materials undergo structural changes in response to external stimuli. In this paper, we show that digital holographic microscopy (DHM) is an effective candidate for characterisation of SPs (embedded in polymer matrices) and for monitoring of their dynamical changes. The polymer matrices are polylactic acid (PLA) and poly(methyl methacrylate) (PMMA) films, which are decorated with SPs and immobilised on graphene quantum dots (GQDs). GQDs are modified by benzylamines prior to the loading of SP species because of the enhancement of hydrophobic characteristics. UV irradiation is used as the external stimulus and the dynamical changes of the samples before and after UV irradiation are measured. DHM is arranged on a novel self-referencing setup, which substantially reduces the sensitivity of DHM to environmental vibrations. Morphometric information for characterisation of the samples is obtained by analysis of the recorded digital holograms. The experimental results demonstrate the potential of the presented technique to serve as an alternative technique for surface measurement methodologies such as atomic force microscope and stylus profiler for surface characterisation of similar materials.

Lensless microscopy by multiplane recordings: sub-micrometer, diffraction-limited, wide field-of-view imaging

Lensless microscopy is attractive because lenses are often large, heavy and expensive. We report diffraction-limited, sub-micrometer resolution in a lensless imaging system that does not need a reference wave and imposes few restrictions on the density of the sample. We use measurements of the intensity of light scattered by the sample at multiple heights above the sample and a modified Gerchberg-Saxton algorithm to reconstruct the phase of the optical field. We introduce a pixel-splitting algorithm that increases resolution beyond the size of the sensor pixels, and implement high-dynamic-range measurements. The resolution depends on the numerical aperture of the first measurement height only, while the field of view is limited by the last measurement height only. As a result, resolution and field of view can be controlled independently. The pixel-splitting algorithm also allows imaging with light of low spatial coherence, and we show that such low coherence is beneficial for a larger field of view. Using illumination from three LEDs, we produce full-color images of biological samples. Finally, we provide a detailed analysis of the limiting factors of this lensless microscopy system. The good performance demonstrated here can allow lensless systems to replace conventional microscope objectives in some situations.

Photolithographic patterning on multi-wavelength quantum dot film of the improved conversion efficiency for digital holography

A triple-wavelength patterned quantum dot film was fabricated for the light source of digital holography to improve both the axial measurement range and noise reduction. The patterned quantum dot film was fabricated after optimizing the photolithography process condition based on the UV-curable quantum dot solution, which was capable of multiple patterning processes. In addition, an optimized pattern structure was developed by adding TiO2 nanoparticles to both the quantum dot and bank layers to increase the scattering effect for the improved photoluminescence intensity. Finally, the newly developed light source with the balanced spectral distribution was applied to the digital holography, rendering it applicable as an improved light source.

Investigating the Joint Amplitude and Phase Imaging of Stained Samples in Automatic Diagnosis

The diagnosis of many diseases relies, at least on first intention, on an analysis of blood smears acquired with a microscope. However, image quality is often insufficient for the automation of such processing. A promising improvement concerns the acquisition of enriched information on samples. In particular, Quantitative Phase Imaging (QPI) techniques, which allow the digitization of the phase in complement to the intensity, are attracting growing interest. Such imaging allows the exploration of transparent objects not visible in the intensity image using the phase image only. Another direction proposes using stained images to reveal some characteristics of the cells in the intensity image; in this case, the phase information is not exploited. In this paper, we question the interest of using the bi-modal information brought by intensity and phase in a QPI acquisition when the samples are stained. We consider the problem of detecting parasitized red blood cells for diagnosing malaria from stained blood smears using a Deep Neural Network (DNN). Fourier Ptychographic Microscopy (FPM) is used as the computational microscopy framework to produce QPI images. We show that the bi-modal information enhances the detection performance by 4% compared to the intensity image only when the convolution in the DNN is implemented through a complex-based formalism. This proves that the DNN can benefit from the bi-modal enhanced information. We conjecture that these results should extend to other applications processed through QPI acquisition.

On-chip label-free cell classification based directly on off-axis holograms and spatial-frequency-invariant deep learning

We present a rapid label-free imaging flow cytometry and cell classification approach based directly on raw digital holograms. Off-axis holography enables real-time acquisition of cells during rapid flow. However, classification of the cells typically requires reconstruction of their quantitative phase profiles, which is time-consuming. Here, we present a new approach for label-free classification of individual cells based directly on the raw off-axis holographic images, each of which contains the complete complex wavefront (amplitude and quantitative phase profiles) of the cell. To obtain this, we built a convolutional neural network, which is invariant to the spatial frequencies and directions of the interference fringes of the off-axis holograms. We demonstrate the effectiveness of this approach using four types of cancer cells. This approach has the potential to significantly improve both speed and robustness of imaging flow cytometry, enabling real-time label-free classification of individual cells.

Randomness assisted in-line holography with deep learning

We propose and demonstrate a holographic imaging scheme exploiting random illuminations for recording hologram and then applying numerical reconstruction and twin image removal. We use an in-line holographic geometry to record the hologram in terms of the second-order correlation and apply the numerical approach to reconstruct the recorded hologram. This strategy helps to reconstruct high-quality quantitative images in comparison to the conventional holography where the hologram is recorded in the intensity rather than the second-order intensity correlation. The twin image issue of the in-line holographic scheme is resolved by an unsupervised deep learning based method using an auto-encoder scheme. Proposed learning technique leverages the main characteristic of autoencoders to perform blind single-shot hologram reconstruction, and this does not require a dataset of samples with available ground truth for training and can reconstruct the hologram solely from the captured sample. Experimental results are presented for two objects, and a comparison of the reconstruction quality is given between the conventional inline holography and the one obtained with the proposed technique.

Investigation of refractive index dynamics during in vitro embryo development using off-axis digital holographic microscopy

Embryo quality is a crucial factor affecting live birth outcomes. However, an accurate diagnostic for embryo quality remains elusive in the in vitro fertilization clinic. Determining physical parameters of the embryo may offer key information for this purpose. Here, we demonstrate that digital holographic microscopy (DHM) can rapidly and non-invasively assess the refractive index of mouse embryos. Murine embryos were cultured in either low- or high-lipid containing media and digital holograms recorded at various stages of development. The phase of the recorded hologram was numerically retrieved, from which the refractive index of the embryo was calculated. We showed that DHM can detect spatio-temporal changes in refractive index during embryo development that are reflective of its lipid content. As accumulation of intracellular lipid is known to compromise embryo health, DHM may prove beneficial in developing an accurate, non-invasive, multimodal diagnostic.

Structured illumination phase and fluorescence microscopy for bioimaging

This study presents a dual-modality microscopic imaging approach that combines quantitative phase microscopy and fluorescence microscopy based on structured illumination (SI) to provide structural and functional information for the same sample. As the first imaging modality, structured illumination digital holographic microscopy (SI-DHM) is implemented along the transmission beam path. SI-DHM acts as a label-free, noninvasive approach and provides high-contrast and quantitative phase images utilizing the refractive index contrast of the inner structures of samples against the background. As the second imaging modality, structured illumination (fluorescence) microscopy (SIM) is constructed along the reflection beam path. SIM utilizes fluorescent labeling and provides super-resolution images for specific functional structures of samples. We first experimentally demonstrated phase imaging of SI-DHM on rice leaves and fluorescence (SIM) imaging on mouse kidney sections. Then, we demonstrated dual-modality imaging of biological samples, using DHM to acquire the overall cell morphology and SIM to obtain specific functional structures. These results prove that the proposed technique is of great importance in biomedical studies, such as providing insight into cell physiology by visualizing and quantifying subcellular structures.

Deep learning-based incoherent holographic camera enabling acquisition of real-world holograms for holographic streaming system

While recent research has shown that holographic displays can represent photorealistic 3D holograms in real time, the difficulty in acquiring high-quality real-world holograms has limited the realization of holographic streaming systems. Incoherent holographic cameras, which record holograms under daylight conditions, are suitable candidates for real-world acquisition, as they prevent the safety issues associated with the use of lasers; however, these cameras are hindered by severe noise due to the optical imperfections of such systems. In this work, we develop a deep learning-based incoherent holographic camera system that can deliver visually enhanced holograms in real time. A neural network filters the noise in the captured holograms, maintaining a complex-valued hologram format throughout the whole process. Enabled by the computational efficiency of the proposed filtering strategy, we demonstrate a holographic streaming system integrating a holographic camera and holographic display, with the aim of developing the ultimate holographic ecosystem of the future.

Holographic Fabrication of 3D Moiré Photonic Crystals Using Circularly Polarized Laser Beams and a Spatial Light Modulator

A moiré photonic crystal is an optical analog of twisted graphene. A 3D moiré photonic crystal is a new nano-/microstructure that is distinguished from bilayer twisted photonic crystals. Holographic fabrication of a 3D moiré photonic crystal is very difficult due to the coexistence of the bright and dark regions, where the exposure threshold is suitable for one region but not for the other. In this paper, we study the holographic fabrication of 3D moiré photonic crystals using an integrated system of a single reflective optical element (ROE) and a spatial light modulator (SLM) where nine beams (four inner beams + four outer beams + central beam) are overlapped. By modifying the phase and amplitude of the interfering beams, the interference patterns of 3D moiré photonic crystals are systemically simulated and compared with the holographic structures to gain a comprehensive understanding of SLM-based holographic fabrication. We report the holographic fabrication of phase and beam intensity ratio-dependent 3D moiré photonic crystals and their structural characterization. Superlattices modulated in the z-direction of 3D moiré photonic crystals have been discovered. This comprehensive study provides guidance for future pixel-by-pixel phase engineering in SLM for complex holographic structures.

Planar Fourier optics for slab waveguides, surface plasmon polaritons, and 2D materials

Recent experimental work has demonstrated the potential of combining the merits of diffractive and on-chip photonic information processing devices in a single chip by making use of planar (or slab) waveguides. Here, arguments are developed to show that diffraction formulas familiar from 3D Fourier optics can be adapted to 2D under certain mild conditions on the operating speeds of the devices in question. In addition to serving those working in on-chip photonics, this Letter provides analytical tools for the study of surface plasmon polaritons, surface waves, and the optical, acoustic, and crystallographic properties of 2D materials.

Compressive holographic sensing simplifies quantitative phase imaging

Quantitative phase imaging (QPI) has emerged as method for investigating biological specimen and technical objects. However, conventional methods often suffer from shortcomings in image quality, such as the twin image artifact. A novel computational framework for QPI is presented with high quality inline holographic imaging from a single intensity image. This paradigm shift is promising for advanced QPI of cells and tissues.

Off-Axis Layered Displays: Hybrid Direct-View/Near-Eye Mixed Reality with Focus Cues

This work introduces off-axis layered displays, the first approach to stereoscopic direct-view displays with support for focus cues. Off-axis layered displays combine a head-mounted display with a traditional direct-view display for encoding a focal stack and thus, for providing focus cues. To explore the novel display architecture, we present a complete processing pipeline for the real-time computation and post-render warping of off-axis display patterns. In addition, we build two prototypes using a head-mounted display in combination with a stereoscopic direct-view display, and a more widely available monoscopic direct-view display. In addition we show how extending off-axis layered displays with an attenuation layer and with eye-tracking can improve image quality. We thoroughly analyze each component in a technical evaluation and present examples captured through our prototypes.

Single-shot refractive index slice imaging using spectrally multiplexed optical transfer function reshaping

The refractive index (RI) of cells and tissues is crucial in pathophysiology as a noninvasive and quantitative imaging contrast. Although its measurements have been demonstrated using three-dimensional quantitative phase imaging methods, these methods often require bulky interferometric setups or multiple measurements, which limits the measurement sensitivity and speed. Here, we present a single-shot RI imaging method that visualizes the RI of the in-focus region of a sample. By exploiting spectral multiplexing and optical transfer function engineering, three color-coded intensity images of a sample with three optimized illuminations were simultaneously obtained in a single-shot measurement. The measured intensity images were then deconvoluted to obtain the RI image of the in-focus slice of the sample. As a proof of concept, a setup was built using Fresnel lenses and a liquid-crystal display. For validation purposes, we measured microspheres of known RI and cross-validated the results with simulated results. Various static and highly dynamic biological cells were imaged to demonstrate that the proposed method can conduct single-shot RI slice imaging of biological samples with subcellular resolution.

3D reconstruction of unstained weakly scattering cells from a single defocused hologram

We investigate the problem of 3D complex field reconstruction corresponding to unstained red blood cells (RBCs) with a single defocused off-axis digital hologram. The main challenge in this problem is the localization of cells to the correct axial range. While investigating the volume recovery problem for a continuous phase object like the RBC, we observe an interesting feature of the backpropagated field that it does not show a clear focusing effect. Therefore, sparsity enforcement within the iterative optimization framework using a single hologram data frame cannot effectively restrict the reconstruction to the true object volume. For phase objects, it is known that the amplitude contrast of the backpropagated object field at the focus plane is minimum. We use this information available in the recovered object field in the hologram plane to device depth-dependent weights that are proportional to the inverse of amplitude contrast. This weight function is employed in the iterative steps of the optimization algorithm to assist the object volume localization. The overall reconstruction process is performed using the mean gradient descent (MGD) framework. Experimental illustrations of 3D volume reconstruction of the healthy as well as malaria-infected RBCs are presented. A test sample of polystyrene microsphere bead is also used to validate the axial localization capability of the proposed iterative technique. The proposed methodology is simple to implement experimentally and provides an approximate tomographic solution, which is axially restricted and consistent with the object field data.

Suppression of alias and replica noises in phase holograms using fractal topologies

Two-dimensional fractal topologies featuring (scaling) self-similarity, dense set of Bragg (diffraction) peaks, and inherent rotation symmetry, which are not achievable with regular grid-matrix geometries, exhibit optical robustness against structural damage and noise immunity of optical transmission paths. In this work, we numerically and experimentally demonstrate phase holograms using fractal plane-divisions. By taking advantage of the symmetries of the fractal topology, we propose numerical algorithms to design the fractal holograms. This algorithm solves the inapplicability of the conventional iterative Fourier transform algorithm (IFTA) method and enables efficient optimizations of millions of adjustable parameters in the optical element. Experimental samples show that the alias and replica noises in the image plane of fractal holograms are clearly suppressed, facilitating applications for high-accuracy and compact requirements.

Hough transform-based multi-object autofocusing compressive holography

Reconstruction of multiple objects from one hologram can be affected by the focus metric judgment of autofocusing. Various segmentation algorithms are applied to obtain a single object in the hologram. Each object is unambiguously reconstructed to acquire its focal position, which produces complicated calculations. Herein, Hough transform (HT)-based multi-object autofocusing compressive holography is presented. The sharpness of each reconstructed image is computed by using a focus metric such as entropy or variance. According to the characteristics of the object, the standard HT is further used for calibration to remove redundant extreme points. The compressive holographic imaging framework with a filter layer can eliminate the inherent noise in in-line reconstruction including cross talk noise of different depth layers, two-order noise, and twin image noise. The proposed method can effectively obtain 3D information on multiple objects and achieve noise elimination by only reconstructing from one hologram.

sPhaseStation: a whole slide quantitative phase imaging system based on dual-view transport of intensity phase microscopy

Whole slide imaging scans a microscope slide into a high-resolution digital image, and it paves the way from pathology to digital diagnostics. However, most of them rely on bright-field and fluorescence imaging with sample labels. In this work, we designed sPhaseStation, which is a dual-view transport of intensity phase microscopy-based whole slide quantitative phase imaging system for label-free samples. sPhaseStation relies on a compact microscopic system with two imaging recorders that can capture both under and over-focus images. Combined with the field of view (FoV) scan, a series of these defocus images in different FoVs can be captured and stitched into two FoV-extended under and over-focus ones, which are used for phase retrieval via solving the transport of intensity equation. Using a 10× micro-objective, sPhaseStation reaches the spatial resolution of 2.19 µm and obtains the phase with high accuracy. Additionally, it acquires a whole slide image of a 3m m×3m m region in 2 min. The reported sPhaseStation could be a prototype of the whole slide quantitative phase imaging device, which may provide a new perspective for digital pathology.

3D single shot lensless incoherent optical imaging using coded phase aperture system with point response of scattered airy beams

Interferenceless coded aperture correlation holography (I-COACH) techniques have revolutionized the field of incoherent imaging, offering multidimensional imaging capabilities with a high temporal resolution in a simple optical configuration and at a low cost. The I-COACH method uses phase modulators (PMs) between the object and the image sensor, which encode the 3D location information of a point into a unique spatial intensity distribution. The system usually requires a one-time calibration procedure in which the point spread functions (PSFs) at different depths and/or wavelengths are recorded. When an object is recorded under identical conditions as the PSF, the multidimensional image of the object is reconstructed by processing the object intensity with the PSFs. In the previous versions of I-COACH, the PM mapped every object point to a scattered intensity distribution or random dot array pattern. The scattered intensity distribution results in a low SNR compared to a direct imaging system due to optical power dilution. Due to the limited focal depth, the dot pattern reduces the imaging resolution beyond the depth of focus if further multiplexing of phase masks is not performed. In this study, I-COACH has been realized using a PM that maps every object point into a sparse random array of Airy beams. Airy beams during propagation exhibit a relatively high focal depth with sharp intensity maxima that shift laterally following a curved path in 3D space. Therefore, sparse, randomly distributed diverse Airy beams exhibit random shifts with respect to one another during propagation, generating unique intensity distributions at different distances while retaining optical power concentrations in small areas on the detector. The phase-only mask displayed on the modulator was designed by random phase multiplexing of Airy beam generators. The simulation and experimental results obtained for the proposed method are significantly better in SNR than in the previous versions of I-COACH.