Y4-Net: a deep learning solution to one-shot dual-wavelength digital holographic reconstruction

Unsupervised cross talk suppression for self-interference digital holography

Self-interference digital holography extends the application of digital holography to non-coherent imaging fields such as fluorescence and scattered light, providing a new solution, to the best of our knowledge, for wide field 3D imaging of low coherence or partially coherent signals. However, cross talk information has always been an important factor limiting the resolution of this imaging method. The suppression of cross talk information is a complex nonlinear problem, and deep learning can easily obtain its corresponding nonlinear model through data-driven methods. However, in real experiments, it is difficult to obtain such paired datasets to complete training. Here, we propose an unsupervised cross talk suppression method based on a cycle-consistent generative adversarial network (CycleGAN) for self-interference digital holography. Through the introduction of a saliency constraint, the unsupervised model, named crosstalk suppressing with unsupervised neural network (CS-UNN), can learn the mapping between two image domains without requiring paired training data while avoiding distortions of the image content. Experimental analysis has shown that this method can suppress cross talk information in reconstructed images without the need for training strategies on a large number of paired datasets, providing an effective solution for the application of the self-interference digital holography technology.

Single-pixel Fresnel incoherent correlation holography compressed imaging using a Trumpet network

Fresnel incoherent correlation holography (FINCH) can achieve high-precision and non-scanning 3D imaging. However, as a holographic imaging technology, the huge bandwidth requirements and the amount of holographic data transmitted have always been one of the important factors limiting its application. In addition, the hardware cost of pixel array-based CCD or CMOS imaging is very high under high resolution or specific wavelength conditions. Accordingly, a single-pixel Fresnel incoherent correlation holography (SP-FINCH) compressed imaging method is proposed, which replaces pixel array detector with single-pixel detector and designs a Trumpet network to achieve low-cost and high-resolution imaging. Firstly, a modified FINCH imaging system is constructed and data acquisition is carried out using a single-pixel detector. Secondly, a Trumpet network is constructed to directly map the relationship between one-dimensional sampled data and two-dimensional image in an end-to-end manner. Moreover, by comparing the reconstructed images using neural network with that using commonly used single-pixel reconstruction methods, the results indicate that the proposed SP-FINCH compressed imaging method can significantly improve the quality of image reconstruction at lower sampling rate and achieve imaging without phase-shifting operation. The proposed method has been shown to be feasible and advantageous through numerical simulations and optical experiment results.

Towards ultrafast quantitative phase imaging via differentiable microscopy [Invited]

With applications ranging from metabolomics to histopathology, quantitative phase microscopy (QPM) is a powerful label-free imaging modality. Despite significant advances in fast multiplexed imaging sensors and deep-learning-based inverse solvers, the throughput of QPM is currently limited by the pixel-rate of the image sensors. Complementarily, to improve throughput further, here we propose to acquire images in a compressed form so that more information can be transferred beyond the existing hardware bottleneck of the image sensor. To this end, we present a numerical simulation of a learnable optical compression-decompression framework that learns content-specific features. The proposed differentiable quantitative phase microscopy (∂-QPM) first uses learnable optical processors as image compressors. The intensity representations produced by these optical processors are then captured by the imaging sensor. Finally, a reconstruction network running on a computer decompresses the QPM images post aquisition. In numerical experiments, the proposed system achieves compression of × 64 while maintaining the SSIM of ∼0.90 and PSNR of ∼30 dB on cells. The results demonstrated by our experiments open up a new pathway to QPM systems that may provide unprecedented throughput improvements.

Phase retrieval based on deep learning with bandpass filtering in holographic data storage

A phase retrieval method based on deep learning with bandpass filtering in holographic data storage is proposed. The relationship between the known encoded data pages and their near-field diffraction intensity patterns is established by an end-to-end convolutional neural network, which is used to predict the unknown phase data page. We found the training efficiency of phase retrieval by deep learning is mainly determined by the edge details of the adjacent phase codes, which are the high-frequency components of the phase code. Therefore, we can attenuate the low-frequency components to reduce material consumption. Besides, we also filter out the high-order frequency over twice Nyquist size, which is redundant information with poor anti-noise performance. Compared with full-frequency recording, the consumption of storage media is reduced by 2.94 times, thus improving the storage density.

On the use of deep learning for phase recovery

Phase recovery (PR) refers to calculating the phase of the light field from its intensity measurements. As exemplified from quantitative phase imaging and coherent diffraction imaging to adaptive optics, PR is essential for reconstructing the refractive index distribution or topography of an object and correcting the aberration of an imaging system. In recent years, deep learning (DL), often implemented through deep neural networks, has provided unprecedented support for computational imaging, leading to more efficient solutions for various PR problems. In this review, we first briefly introduce conventional methods for PR. Then, we review how DL provides support for PR from the following three stages, namely, pre-processing, in-processing, and post-processing. We also review how DL is used in phase image processing. Finally, we summarize the work in DL for PR and provide an outlook on how to better use DL to improve the reliability and efficiency of PR. Furthermore, we present a live-updating resource ( https://github.com/kqwang/phase-recovery ) for readers to learn more about PR.

Single-shot Fresnel incoherent correlation holography via deep learning based phase-shifting technology

Fresnel incoherent correlation holography (FINCH) realizes non-scanning three-dimension (3D) images using spatial incoherent illumination, but it requires phase-shifting technology to remove the disturbance of the DC term and twin term that appears in the reconstruction field, thus increasing the complexity of the experiment and limits the real-time performance of FINCH. Here, we propose a single-shot Fresnel incoherent correlation holography via deep learning based phase-shifting (FINCH/DLPS) method to realize rapid and high-precision image reconstruction using only a collected interferogram. A phase-shifting network is designed to implement the phase-shifting operation of FINCH. The trained network can conveniently predict two interferograms with the phase shift of 2/3 π and 4/3 π from one input interferogram. Using the conventional three-step phase-shifting algorithm, we can conveniently remove the DC term and twin term of the FINCH reconstruction and obtain high-precision reconstruction through the back propagation algorithm. The Mixed National Institute of Standards and Technology (MNIST) dataset is used to verify the feasibility of the proposed method through experiments. In the test with the MNIST dataset, the reconstruction results demonstrate that in addition to high-precision reconstruction, the proposed FINCH/DLPS method also can effectively retain the 3D information by calibrating the back propagation distance in the case of reducing the complexity of the experiment, further indicating the feasibility and superiority of the proposed FINCH/DLPS method.

Quantitative phase gradient metrology using diffraction phase microscopy and deep learning

In quantitative phase microscopy, measurement of the phase gradient is an important problem for biological cell morphological studies. In this paper, we propose a method based on a deep learning approach that is capable of direct estimation of the phase gradient without the requirement of phase unwrapping and numerical differentiation operations. We show the robustness of the proposed method using numerical simulations under severe noise conditions. Further, we demonstrate the method's utility for imaging different biological cells using diffraction phase microscopy setup.

Coherent modulation imaging using a physics-driven neural network

Coherent modulation imaging (CMI) is a lessness diffraction imaging technique, which uses an iterative algorithm to reconstruct a complex field from a single intensity diffraction pattern. Deep learning as a powerful optimization method can be used to solve highly ill-conditioned problems, including complex field phase retrieval. In this study, a physics-driven neural network for CMI is developed, termed CMINet, to reconstruct the complex-valued object from a single diffraction pattern. The developed approach optimizes the network's weights by a customized physical-model-based loss function, instead of using any ground truth of the reconstructed object for training beforehand. Simulation experiment results show that the developed CMINet has a high reconstruction quality with less noise and robustness to physical parameters. Besides, a trained CMINet can be used to reconstruct a dynamic process with a fast speed instead of iterations frame-by-frame. The biological experiment results show that CMINet can reconstruct high-quality amplitude and phase images with more sharp details, which is practical for biological imaging applications.

Learning-based complex field recovery from digital hologram with various depth objects

In this paper, we investigate a learning-based complex field recovery technique of an object from its digital hologram. Most of the previous learning-based approaches first propagate the captured hologram to the object plane and then suppress the DC and conjugate noise in the reconstruction. To the contrary, the proposed technique utilizes a deep learning network to extract the object complex field in the hologram plane directly, making it robust to the object depth variations and well suited for three-dimensional objects. Unlike the previous approaches which concentrate on transparent biological samples having near-uniform amplitude, the proposed technique is applied to more general objects which have large amplitude variations. The proposed technique is verified by numerical simulations and optical experiments, demonstrating its feasibility.

Deep learning-based method for non-uniform motion-induced error reduction in dynamic microscopic 3D shape measurement

The non-uniform motion-induced error reduction in dynamic fringe projection profilometry is complex and challenging. Recently, deep learning (DL) has been successfully applied to many complex optical problems with strong nonlinearity and exhibits excellent performance. Inspired by this, a deep learning-based method is developed for non-uniform motion-induced error reduction by taking advantage of the powerful ability of nonlinear fitting. First, a specially designed dataset of motion-induced error reduction is generated for network training by incorporating complex nonlinearity. Then, the corresponding DL-based architecture is proposed and it contains two parts: in the first part, a fringe compensation module is developed as network pre-processing to reduce the phase error caused by fringe discontinuity; in the second part, a deep neural network is employed to extract the high-level features of error distribution and establish a pixel-wise hidden nonlinear mapping between the phase with motion-induced error and the ideal one. Both simulations and real experiments demonstrate the feasibility of the proposed method in dynamic macroscopic measurement.

Deep learning in optical metrology: a review

With the advances in scientific foundations and technological implementations, optical metrology has become versatile problem-solving backbones in manufacturing, fundamental research, and engineering applications, such as quality control, nondestructive testing, experimental mechanics, and biomedicine. In recent years, deep learning, a subfield of machine learning, is emerging as a powerful tool to address problems by learning from data, largely driven by the availability of massive datasets, enhanced computational power, fast data storage, and novel training algorithms for the deep neural network. It is currently promoting increased interests and gaining extensive attention for its utilization in the field of optical metrology. Unlike the traditional "physics-based" approach, deep-learning-enabled optical metrology is a kind of "data-driven" approach, which has already provided numerous alternative solutions to many challenging problems in this field with better performances. In this review, we present an overview of the current status and the latest progress of deep-learning technologies in the field of optical metrology. We first briefly introduce both traditional image-processing algorithms in optical metrology and the basic concepts of deep learning, followed by a comprehensive review of its applications in various optical metrology tasks, such as fringe denoising, phase retrieval, phase unwrapping, subset correlation, and error compensation. The open challenges faced by the current deep-learning approach in optical metrology are then discussed. Finally, the directions for future research are outlined.

Dynamic deformation measurement of dual-wavelength arbitrary phase-shifting digital holography with automatic phase-shift detection

Dual-wavelength arbitrary phase-shifting digital holography with automatic phase-shift detection is first proposed in this study. Holograms with two wavelengths and the interference fringes used to detect the phase-shifting amount for each wavelength were simultaneously recorded in one image using the space-division multiplexing technique. Compared with conventional methods, the proposed approach can achieve simultaneous phase shifting of the reference beams of two wavelengths, which substantially reduces recording time and does not require excessive phase-shifting device precision. The proposed and conventional methods were quantitatively evaluated with numerical simulations, and a dynamic deformation measurement was obtained using the system. In the quantitative evaluation of the simulation, the root-mean-square errors of amplitude and phase images reconstructed by the proposed method were reduced by 12% and 19% compared to the conventional method, respectively. Both numerical simulations and experiments verified the effectiveness of the proposed method.

Y-Net: a dual-branch deep learning network for nonlinear absorption tomography with wavelength modulation spectroscopy

This paper demonstrates a new method for solving nonlinear tomographic problems, combining calibration-free wavelength modulation spectroscopy (CF-WMS) with a dual-branch deep learning network (Y-Net). The principle of CF-WMS, as well as the architecture, training and performance of Y-Net have been investigated. 20000 samples are randomly generated, with each temperature or H₂O concentration phantom featuring three randomly positioned Gaussian distributions. Non-uniformity coefficient (NUC) method provides quantitative characterizations of the non-uniformity (i.e., the complexity) of the reconstructed fields. Four projections, each with 24 parallel beams are assumed. The average reconstruction errors of temperature and H₂O concentration for the testing dataset with 2000 samples are 1.55% and 2.47%, with standard deviations of 0.46% and 0.75%, respectively. The reconstruction errors for both temperature and species concentration distributions increase almost linearly with increasing NUC from 0.02 to 0.20. The proposed Y-Net shows great advantages over the state-of-the-art simulated annealing algorithm, such as better noise immunity and higher computational efficiency. This is the first time, to the best of our knowledge, that a dual-branch deep learning network (Y-Net) has been applied to WMS-based nonlinear tomography and it opens up opportunities for real-time, in situ monitoring of practical combustion environments.

Deep learning for digital holography: a review

Recent years have witnessed the unprecedented progress of deep learning applications in digital holography (DH). Nevertheless, there remain huge potentials in how deep learning can further improve performance and enable new functionalities for DH. Here, we survey recent developments in various DH applications powered by deep learning algorithms. This article starts with a brief introduction to digital holographic imaging, then summarizes the most relevant deep learning techniques for DH, with discussions on their benefits and challenges. We then present case studies covering a wide range of problems and applications in order to highlight research achievements to date. We provide an outlook of several promising directions to widen the use of deep learning in various DH applications.

Enlarged range of measurement method with strong noise resistance for dual-wavelength digital holography

A concise and powerful method for dual-wavelength digital holography (DWDH) is proposed. By designing a new algorithm, this proposed method bypasses the phase synthesis process and directly obtains the thickness distribution of the object. This method can enlarge the range of measurement with strong noise resistance. For example, noise analysis results show that the proposed method reduces the reconstruction error from 101 nm to 9 nm when the signal-to-noise ratio is equal to 30. Therefore, this method would prove useful for DWDH, and its effectiveness is verified by both numerical simulations and experimental results.

Lensless phase retrieval based on deep learning used in holographic data storage

This paper proposes a lensless phase retrieval method based on deep learning (DL) used in holographic data storage. By training an end-to-end convolutional neural network between the phase-encoded data pages and the corresponding near-field diffraction intensity images, the new unknown phase data page can be predicted directly from the intensity image by the network model without any iterations. The DL-based phase retrieval method has a higher storage density, lower bit-error-rate (BER), and higher data transfer rate compared to traditional iterative methods. The retrieval optical system is simple, stable, and robust to environment fluctuations which is suitable for holographic data storage. Besides, we studied and demonstrated that the DL method has a good suppression effect on the dynamic noise of the holographic data storage system.

Real-time quantitative phase imaging by single-shot dual-wavelength off-axis digital holographic microscopy

A single-shot dual-wavelength digital holographic microscopy with an adjustable off-axis configuration is presented, which helps realize real-time quantitative phase imaging for living cells. With this configuration, two sets of interference fringes corresponding to their wavelengths can be flexibly recorded onto one hologram in one shot. The universal expression on the dual-wavelength hologram recorded under any wave vector orientation angles of reference beams is given. To avoid as much as possible the effect of zero-order spectrum, we can flexibly select their carry frequencies for the two wavelengths using this adjustable off-axis configuration, according to the distribution feature of object's spatial-frequency spectrum. This merit is verified by a quantitative phase imaging experiment for the microchannel of a microfluidic chip. The reconstructed phase maps of living onion epidermal cells exhibit cellular internal life activities, for the first time to the best of our knowledge, vividly displaying the progress of the nucleus, cell wall, cytoskeleton, and the substance transport in microtubules inside living cells. These imaging results demonstrate the availability and reliability of the presented method for real-time quantitative phase imaging.

Sparse-view imaging of a fiber internal structure in holographic diffraction tomography via a convolutional neural network

Deep learning has recently shown great potential in computational imaging. Here, we propose a deep-learning-based reconstruction method to realize the sparse-view imaging of a fiber internal structure in holographic diffraction tomography. By taking the sparse-view sinogram as the input and the cross-section image obtained by the dense-view sinogram as the ground truth, the neural network can reconstruct the cross-section image from the sparse-view sinogram. It performs better than the corresponding filtered back-projection algorithm with a sparse-view sinogram, both in the case of simulated data and real experimental data.

IPLNet: a neural network for intensity-polarization imaging in low light

Imaging in low light is significant but challenging in many applications. Adding the polarization information into the imaging system compromises the drawbacks of the conventional intensity imaging to some extent. However, generally speaking, the qualities of intensity images and polarization images cannot be compatible due to the characteristic differences in polarimetric operators. In this Letter, we collected, to the best of our knowledge, the first polarimetric imaging dataset in low light and present a specially designed neural network to enhance the image qualities of intensity and polarization simultaneously. Both indoor and outdoor experiments demonstrate the effectiveness and superiority of this neural network-based solution, which may find important applications for object detection and vision in photon-starved environments.

Quantitative phase imaging in dual-wavelength interferometry using a single wavelength illumination and deep learning

In this manuscript, we propose a quantitative phase imaging method based on deep learning, using a single wavelength illumination to realize dual-wavelength phase-shifting phase recovery. By using the conditional generative adversarial network (CGAN), from one interferogram recorded at a single wavelength, we obtain interferograms at other wavelengths, the corresponding wrapped phases and then the phases at synthetic wavelengths. The feasibility of the proposed method is verified by simulation and experiments. The results demonstrate that the measurement range of single-wavelength interferometry (SWI) is improved by keeping a simple setup, avoiding the difficulty caused by using two wavelengths simultaneously. This will provide an effective solution for the problem of phase unwrapping and the measurement range limitation in phase-shifting interferometry.