Sparsity-Based Pixel Super Resolution for Lens-Free Digital In-line Holography

Randomness assisted in-line holography with deep learning

We propose and demonstrate a holographic imaging scheme exploiting random illuminations for recording hologram and then applying numerical reconstruction and twin image removal. We use an in-line holographic geometry to record the hologram in terms of the second-order correlation and apply the numerical approach to reconstruct the recorded hologram. This strategy helps to reconstruct high-quality quantitative images in comparison to the conventional holography where the hologram is recorded in the intensity rather than the second-order intensity correlation. The twin image issue of the in-line holographic scheme is resolved by an unsupervised deep learning based method using an auto-encoder scheme. Proposed learning technique leverages the main characteristic of autoencoders to perform blind single-shot hologram reconstruction, and this does not require a dataset of samples with available ground truth for training and can reconstruct the hologram solely from the captured sample. Experimental results are presented for two objects, and a comparison of the reconstruction quality is given between the conventional inline holography and the one obtained with the proposed technique.

Lensfree on-chip microscopy based on single-plane phase retrieval

We propose a novel single-plane phase retrieval method to realize high-quality sample reconstruction for lensfree on-chip microscopy. In our method, complex wavefield reconstruction is modeled as a quadratic minimization problem, where total variation and joint denoising regularization are designed to keep a balance of artifact removal and resolution enhancement. In experiment, we built a 3D-printed field-portable platform to validate the imaging performance of our method, where resolution chart, dynamic target, transparent cell, polystyrene beads, and stained tissue sections are employed for the imaging test. Compared to state-of-the-art methods, our method eliminates image degradation and obtains a higher imaging resolution. Different from multi-wavelength or multi-height phase retrieval methods, our method only utilizes a single-frame intensity data record to accomplish high-fidelity reconstruction of different samples, which contributes a simple, robust, and data-efficient solution to design a resource-limited lensfree on-chip microscope. We believe that it will become a useful tool for telemedicine and point-of-care application.

Computational Optics for Point-of-Care Breast Cancer Profiling

With the global burden of cancer on the rise, it is critical to developing new modalities that could detect cancer and guide targeted treatments in fast and inexpensive ways. The need for such technologies is vital, especially in underserved regions where severe diagnostic bottlenecks exist. Recently, we developed a low-cost digital diagnostic system for breast cancer using fine-needle aspirates (FNAs). Named, AIDA (artificial intelligence diffraction analysis), the system combines lens-free digital diffraction imaging with deep-learning algorithms to achieve automated, rapid, and high-throughput cellular analyses for breast cancer diagnosis of FNA and subtype classification for better-guided treatments (Min et al. ACS Nano 12:9081-9090, 2018). Although primarily validated for breast cancer and lymphoma (Min et al. ACS Nano 12:9081-9090, 2018; Im et al. Nat Biomed Eng 2:666-674, 2018), the system could be easily adapted to diagnosing other prevalent cancers and thus find widespread use for global health.

Deep learning for digital holography: a review

Recent years have witnessed the unprecedented progress of deep learning applications in digital holography (DH). Nevertheless, there remain huge potentials in how deep learning can further improve performance and enable new functionalities for DH. Here, we survey recent developments in various DH applications powered by deep learning algorithms. This article starts with a brief introduction to digital holographic imaging, then summarizes the most relevant deep learning techniques for DH, with discussions on their benefits and challenges. We then present case studies covering a wide range of problems and applications in order to highlight research achievements to date. We provide an outlook of several promising directions to widen the use of deep learning in various DH applications.

Subsampled digital holographic image reconstruction by a compressive sensing approach

We show an in-line digital holographic image reconstruction from subsampled holograms with resolution improvement and lensless magnification with high noise immunity by a compressive sensing approach. Our method treats the sensed field as subsampled, low-pass filtered and projected on a Fresnel-Bluestein base in an inverse problem approach to image reconstruction with controlled lensless magnification. So, we have demonstrated by simulation and experimental results that the approach can reconstruct images with quality even when used in holograms obtained from unusual subsampling schemes.

Color lens-free imaging using multi-wavelength illumination based phase retrieval

Accurate image reconstruction in color lens-free imaging has proven challenging. The color image reconstruction of a sample is impacted not only by how strongly the illumination intensity is absorbed at a given spectral range, but also by the lack of phase information recorded on the image sensor. We present a compact and cost-effective approach of addressing the need for phase retrieval to enable robust color image reconstruction in lens-free imaging. The amplitude images obtained at transparent wavelength bands are used to estimate the phase in highly absorbed wavelength bands. The accurate phase information, obtained through our iterative algorithm, removes the color artefacts due to twin-image noise in the reconstructed image and improves image reconstruction quality to allow accurate color reconstruction. This could enable the technique to be applied for imaging of stained pathology slides, an important tool in medical diagnostics.

Holography applications toward medical field: An overview

Purpose:

3D Holography is a commercially available, disruptive innovation, which can be customised as per the requirements and is supporting Industry 4.0. The purpose of this paper is to study the potential applications of 3D holography in the medical field. This paper explores the concept of holography and its significant benefits in the medical field.

Methods:

The paper is derived through the study of various research papers on Holography and its applications in the medical field. The study tries to identify the direction of research &development and see how this innovative technology can be used effectively for better treatment of patients.

Results:

Holography uses digital imaging inputs and provides an extensive visualisation of the data for training doctors, surgeons and students. Holography converts information about the body into a digital format and has the potential to inform, promote and entertain the medical students and doctors. However, it needs a large amount of space for data storage and extensive software support for analysis and skills for customising. This technology seems good to solve a variety of medical issues by storing and using patient data in developing 3D holograms, which are useful to assist successful treatment and surgery. It seems useful in providing flexible solutions in the area of medical research. Finally, the paper identifies 13 significant applications of this technology in the medical field and discusses them appropriately.

Conclusion:

The paper explores holographic applications in medical research due to its extensive capability of image processing. Holographic images are non-contact 3D images having a large field of depth. A physician can now zoom the holographic image for a better view of the medical part. This innovative technology can create advancements in the diagnosis and treatment process, which can improve medical practice. It helps in quick detection of problems in various organs like brain, heart, liver, kidney etc. By using this technology, medical practitioners can see colourful organs at multiple angles with better accuracy. It opens up an innovative way of planning, testing of procedures and diagnosis. With technological developments, compact hardware and software are now available to help medical research and related applications.

From Fienup's phase retrieval techniques to regularized inversion for in-line holography: tutorial

This paper includes a tutorial on how to reconstruct in-line holograms using an inverse problems approach, starting with modeling the observations, selecting regularizations and constraints, and ending with the design of a reconstruction algorithm. A special focus is placed on the connections between the numerous alternating projections strategies derived from Fienup's phase retrieval technique and the inverse problems framework. In particular, an interpretation of Fienup's algorithm as iterates of a proximal gradient descent for a particular cost function is given. Reconstructions from simulated and experimental holograms of micrometric beads illustrate the theoretical developments. The results show that the transition from alternating projections techniques to the inverse problems formulation is straightforward and advantageous.

Pixel super-resolution for lens-free holographic microscopy using deep learning neural networks

Lens-free holographic microscopy (LFHM) provides a cost-effective tool for large field-of-view imaging in various biomedical applications. However, due to the unit optical magnification, its spatial resolution is limited by the pixel size of the imager. Pixel super-resolution (PSR) technique tackles this problem by using a series of sub-pixel shifted low-resolution (LR) lens-free holograms to form the high-resolution (HR) hologram. Conventional iterative PSR methods require a large number of measurements and a time-consuming reconstruction process, limiting the throughput of LFHM in practice. Here we report a deep learning-based PSR approach to enhance the resolution of LFHM. Compared with the existing PSR methods, our neural network-based approach outputs the HR hologram in an end-to-end fashion and maintains consistency in resolution improvement with a reduced number of LR holograms. Moreover, by exploiting the resolution degradation model in the imaging process, the network can be trained with a data set synthesized from the LR hologram itself without resorting to the HR ground truth. We validated the effectiveness and the robustness of our method by imaging various types of samples using a single network trained on an entirely different data set. This deep learning-based PSR approach can significantly accelerate both the data acquisition and the HR hologram reconstruction processes, therefore providing a practical solution to fast, lens-free, super-resolution imaging.

Machine Learning with Optical Phase Signatures for Phenotypic Profiling of Cell Lines

Robust and reproducible profiling of cell lines is essential for phenotypic screening assays. The goals of this study were to determine robust and reproducible optical phase signatures of cell lines for classification with machine learning and to correlate optical phase parameters to motile behavior. Digital holographic microscopy (DHM) reconstructed phase maps of cells from two pairs of cancer and non-cancer cell lines. Seventeen image parameters were extracted from each cell's phase map, used for linear support vector machine learning, and correlated to scratch wound closure and Boyden chamber chemotaxis. The classification accuracy was between 90% and 100% for the six pairwise cell line comparisons. Several phase parameters correlated with wound closure rate and chemotaxis across the four cell lines. The level of cell confluence in culture affected phase parameters in all cell lines tested. Results indicate that optical phase features of cell lines are a robust set of quantitative data of potential utility for phenotypic screening and prediction of motile behavior. © 2019 International Society for Advancement of Cytometry.

Sparsity-based continuous wave terahertz lens-free on-chip holography with sub-wavelength resolution

We demonstrate terahertz (THz) lens-free in-line holography on a chip in order to achieve 40 μm spatial resolution corresponding to ~0.7λ with a numerical aperture of ~0.87. We believe that this is the first time that sub-wavelength resolution in THz holography and the 40 μm resolution were both far better than what was already reported. The setup is based on a self-developed high-power continuous wave THz laser at 5.24 THz (λ = 57.25 μm) and a high-resolution microbolometer detector array (640 × 512 pixels) with a pitch of 17 μm. This on-chip in-line holography, however, suffers from the twin-image artifacts which obfuscate the reconstruction. To address this problem, we propose an iterative optimization framework, where the conventional object constraint and the L₁ sparsity constraint can be combined to efficiently reconstruct the complex amplitude distribution of the sample. Note that the proposed framework and the sparsity-based algorithm can be applied to holography in other wavebands without limitation of wavelength. We demonstrate the success of this sparsity-based on-chip holography by imaging biological samples (i.e., a dragonfly wing and a bauhinia leaf).

Optimized sensing of sparse and small targets using lens-free holographic microscopy

Lens-free holographic microscopy offers sub-micron resolution over an ultra-large field-of-view >20 mm², making it suitable for bio-sensing applications that require the detection of small targets at low concentrations. Various pixel super-resolution techniques have been shown to enhance resolution and boost signal-to-noise ratio (SNR) by combining multiple partially-redundant low-resolution frames. However, it has been unclear which technique performs best for small-target sensing. Here, we quantitatively compare SNR and resolution in experiments using no regularization, cardinal-neighbor regularization, and a novel implementation of sparsity-promoting regularization that uses analytically-calculated gradients from Bayer-pattern image sensors. We find that sparsity-promoting regularization enhances the SNR by ~8 dB compared to the other methods when imaging micron-scale beads with surface coverages up to ~4%.

Computational Optics Enables Breast Cancer Profiling in Point-of-Care Settings

The global burden of cancer, severe diagnostic bottlenecks in underserved regions, and underfunded health care systems are fueling the need for inexpensive, rapid, and treatment-informative diagnostics. On the basis of advances in computational optics and deep learning, we have developed a low-cost digital system, termed AIDA (artificial intelligence diffraction analysis), for breast cancer diagnosis of fine needle aspirates. Here, we show high accuracy (>90%) in (i) recognizing cells directly from diffraction patterns and (ii) classifying breast cancer types using deep-learning-based analysis of sample aspirates. The image algorithm is fast, enabling cellular analyses at high throughput (∼3 s per 1000 cells), and the unsupervised processing allows use by lower skill health care workers. AIDA can perform quantitative molecular profiling on individual cells, revealing intratumor molecular heterogeneity, and has the potential to improve cancer diagnosis and treatment. The system could be further developed for other cancers and thus find widespread use in global health.

Regularized reconstruction of absorbing and phase objects from a single in-line hologram, application to fluid mechanics and micro-biology

Reconstruction of phase objects is a central problem in digital holography, whose various applications include microscopy, biomedical imaging, and fluid mechanics. Starting from a single in-line hologram, there is no direct way to recover the phase of the diffracted wave in the hologram plane. The reconstruction of absorbing and phase objects therefore requires the inversion of the non-linear hologram formation model. We propose a regularized reconstruction method that includes several physically-grounded constraints such as bounds on transmittance values, maximum/minimum phase, spatial smoothness or the absence of any object in parts of the field of view. To solve the non-convex and non-smooth optimization problem induced by our modeling, a variable splitting strategy is applied and the closed-form solution of the sub-problem (the so-called proximal operator) is derived. The resulting algorithm is efficient and is shown to lead to quantitative phase estimation on reconstructions of accurate simulations of in-line holograms based on the Mie theory. As our approach is adaptable to several in-line digital holography configurations, we present and discuss the promising results of reconstructions from experimental in-line holograms obtained in two different applications: the tracking of an evaporating droplet (size ∼ 100μm) and the microscopic imaging of bacteria (size ∼ 1μm).

Coded aperture structured illumination digital holographic microscopy for superresolution imaging

This paper proposes a coded aperture structured illumination (CASI) technique in digital holographic microscopy (DHM). A CASI wave is generated using two binary phase codes (0° and 120°) for spatial phase shifting. The generated CASI wave then interferes with a reference wave to form a coded Fresnel hologram at a single exposure with compressive sensing (CS) to avoid the temporal phase-shifting process of the structured illumination (SI). The CS algorithm is applied to retrieve the missing data of decoded phase-shifted SI-modulated waves, which are used to separate overlapped spatial frequencies for obtaining a larger spatial frequency coverage to provide superresolution imaging. Two phase-only spatial light modulators are applied to generate a directional SI pattern for obtaining a coded aperture with a suitable size to perform one-shot acquisition in the DHM system.

Learning-based single-shot superresolution in diffractive imaging

We present a method of retrieving a superresolved object field from a single captured intensity image in diffraction-limited diffractive imaging based on machine learning. In this method, the inverse process of diffractive imaging is regressed by using a number of pairs, each consisting of object and captured images. The proposed method is experimentally demonstrated by using a lensless imaging setup with or without scattering media.

Holographic Writing of Ink-Based Phase Conjugate Nanostructures via Laser Ablation

The optical phase conjugation (OPC) through photonic nanostructures in coherent optics involves the utilization of a nonlinear optical mechanism through real-time processing of electromagnetic fields. Their applications include spectroscopy, optical tomography, wavefront sensing, and imaging. The development of functional and personalized holographic devices in the visible and near-infrared spectrum can be improved by introducing cost-effective, rapid, and high-throughput fabrication techniques and low-cost recording media. Here, we develop flat and thin phase-conjugate nanostructures on low-cost ink coated glass substrates through a facile and flexible single pulsed nanosecond laser based reflection holography and a cornercube retroreflector (CCR). Fabricated one/two-dimensional (1D/2D) nanostructures exhibited far-field phase-conjugated patterns through wavefront reconstruction by means of diffraction. The optical phase conjugation property had correlation with the laser light (energy) and structural parameters (width, height and exposure angle) variation. The phase conjugated diffraction property from the recorded nanostructures was verified through spectral measurements, far-field diffraction experiments, and thermal imaging. Furthermore, a comparison between the conventional and phase-conjugated nanostructures showed two-fold increase in diffracted light intensity under monochromatic light illumination. It is anticipated that low-cost ink based holographic phase-conjugate nanostructures may have applications in flexible and printable displays, polarization-selective flat waveplates, and adaptive diffraction optics.

Studies on the sparsifying operator in compressive digital holography

In compressive digital holography, we reconstruct sparse object wavefields from undersampled holograms by solving an ℓ₁-minimization problem. Applying wavelet transformations to the object wavefields produces the necessary sparse representations, but prior work clings to transformations with too few vanishing moments. We put several wavelet transformations belonging to different wavelet families to the test by evaluating their sparsifying properties, the number of hologram samples that are required to reconstruct the sparse wavefields perfectly, and the robustness of the reconstructions to additive noise and sparsity defects. In particular, we recommend the CDF 9/7 and 17/11 wavelet transformations, as well as their reverse counter-parts, because they yield sufficiently sparse representations for most accustomed wavefields in combination with robust reconstructions. These and other recommendations are procured from simulations and are validated using biased, noisy holograms.

Compressive holographic video

Compressed sensing has been discussed separately in spatial and temporal domains. Compressive holography has been introduced as a method that allows 3D tomographic reconstruction at different depths from a single 2D image. Coded exposure is a temporal compressed sensing method for high speed video acquisition. In this work, we combine compressive holography and coded exposure techniques and extend the discussion to 4D reconstruction in space and time from one coded captured image. In our prototype, digital in-line holography was used for imaging macroscopic, fast moving objects. The pixel-wise temporal modulation was implemented by a digital micromirror device. In this paper we demonstrate 10× temporal super resolution with multiple depths recovery from a single image. Two examples are presented for the purpose of recording subtle vibrations and tracking small particles within 5 ms.