Accurate quantitative phase digital holographic microscopy with single- and multiple-wavelength telecentric and nontelecentric configurations

Visualizing the fine structure and dynamics of living cells with temporal polychromatic digital holographic microscopy

Polychromatic digital holographic microscopy (P-DHM) has demonstrated its capacity to generate highly denoised optical path difference images, thereby enabling the label-free visualization of fine cellular structures, such as the dendritic arborization within neuronal cells in culture. So far, however, the sample must remain more or less stationary since P-DHM is performed manually, i.e., all actions are carried out sequentially over several minutes. In this paper, we propose fully automated, robust, and efficient management of the acquisition and reconstruction of the time series of polychromatic hologram sets, transforming P-DHM into temporal P-DHM. Experimental results have demonstrated the ability of the proposed temporal P-DHM implementation to non-invasively and quantitatively reveal the fine structure and dynamics of living cells.

Digital Holographic Microscopy to Assess Cell Behavior

Digital holographic microscopy is an imaging technique particularly well suited to the study of living cells in culture, as no labeling is required and computed phase maps produce high contrast, quantitative pixel information. A full experiment involves instrument calibration, cell culture quality checks, selection and setup of imaging chambers, a sampling plan, image acquisition, phase and amplitude map reconstruction, and parameter map post-processing to extract information about cell morphology and/or motility. Each step is described below, focusing on results from imaging four human cell lines. Several post-processing approaches are detailed, with an aim of tracking individual cells and dynamics of cell populations.

Deep learning in optical metrology: a review

With the advances in scientific foundations and technological implementations, optical metrology has become versatile problem-solving backbones in manufacturing, fundamental research, and engineering applications, such as quality control, nondestructive testing, experimental mechanics, and biomedicine. In recent years, deep learning, a subfield of machine learning, is emerging as a powerful tool to address problems by learning from data, largely driven by the availability of massive datasets, enhanced computational power, fast data storage, and novel training algorithms for the deep neural network. It is currently promoting increased interests and gaining extensive attention for its utilization in the field of optical metrology. Unlike the traditional "physics-based" approach, deep-learning-enabled optical metrology is a kind of "data-driven" approach, which has already provided numerous alternative solutions to many challenging problems in this field with better performances. In this review, we present an overview of the current status and the latest progress of deep-learning technologies in the field of optical metrology. We first briefly introduce both traditional image-processing algorithms in optical metrology and the basic concepts of deep learning, followed by a comprehensive review of its applications in various optical metrology tasks, such as fringe denoising, phase retrieval, phase unwrapping, subset correlation, and error compensation. The open challenges faced by the current deep-learning approach in optical metrology are then discussed. Finally, the directions for future research are outlined.

Computational multi-wavelength phase synthesis using convolutional neural networks [Invited]

Multi-wavelength digital holographic microscopy (MWDHM) provides indirect measurements of the refractive index for non-dispersive samples. Successive-shot MWDHM is not appropriate for dynamic samples and single-shot MWDHM significantly increases the complexity of the optical setup due to the need for multiple lasers or a wavelength tunable source. Here we consider deep learning convolutional neural networks for computational phase synthesis to obtain high-speed simultaneous phase estimates on different wavelengths and thus single-shot estimates of the integral refractive index without increased experimental complexity. This novel, to the best of our knowledge, computational concept is validated using cell phantoms consisting of internal refractive index variations representing cytoplasm and membrane-bound organelles, respectively, and a simulation of a realistic holographic recording process. Specifically, in this work we employed data-driven computational techniques to perform accurate dual-wavelength hologram synthesis (hologram-to-hologram prediction), dual-wavelength phase synthesis (unwrapped phase-to-phase prediction), direct phase-to-index prediction using a single wavelength, hologram-to-phase prediction, and 2D phase unwrapping with sharp discontinuities (wrapped-to-unwrapped phase prediction).

Dual-modality digital holographic and polarization microscope to quantify phase and birefringence signals in biospecimens with a complex microstructure

Optical phase and birefringence signals occur in cells and thin, semi-transparent biomaterials. A dual-modality quantitative phase and polarization microscope was designed to study the interaction of cells with extracellular matrix networks and to relate optical pathlength and birefringence signals within structurally anisotropic biomaterial constructs. The design was based on an existing, custom-built digital holographic microscope, to which was added a polarization microscope utilizing liquid crystal variable retarders. Phase and birefringence channels were calibrated, and data was acquired sequentially from cell-seeded collagen hydrogels and electrofabricated chitosan membranes. Computed phase height and retardance from standard targets were accurate within 99.7% and 99.8%, respectively. Phase height and retardance channel background standard deviations were 35 nm and 0.6 nm, respectively. Human fibroblasts, visible in the phase channel, aligned with collagen network microstructure, with retardance and azimuth visible in the polarization channel. Electrofabricated chitosan membranes formed in 40 µm tall microfluidic channels possessed optical retardance ranging from 7 to 11 nm, and phase height from 37 to 39 µm. These results demonstrate co-registered dual-channel acquisition of phase and birefringence parameter maps from microstructurally-complex biospecimens using a novel imaging system combining digital holographic microscopy with voltage-controlled polarization microscopy.

Keratin 19 maintains E-cadherin localization at the cell surface and stabilizes cell-cell adhesion of MCF7 cells

A cytoskeletal protein keratin 19 (K19) is highly expressed in breast cancer but its effects on breast cancer cell mechanics are unclear. In MCF7 cells where K19 expression is ablated,we found that K19 is required to maintain rounded epithelial-like shape and tight cell-cell adhesion. A loss of K19 also lowered cell surface E-cadherin levels. Inhibiting internalization restored cell-cell adhesion of KRT19  knockout cells, suggesting that E-cadherin internalization contributed to defective adhesion. Ultimately, while K19 inhibited cell migration and invasion, it was required for cells to form colonies in suspension. Our results suggest that K19 stabilizes E-cadherin complexes at the cell membrane to maintain cell-cell adhesion which inhibits cell invasiveness but provides growth and survival advantages for circulating tumor cells.

Aberration-free digital holographic phase imaging using the derivative-based principal component analysis

SIGNIFICANCE

Digital holographic microscopy is widely used to get the quantitative phase information of transparent cells.

AIM

However, the sample phase is superimposed with aberrations. To quantify the phase information, aberrations need to be fully compensated.

APPROACH

We propose a technique to obtain aberration-free phase imaging, using the derivative-based principal component analysis (dPCA).

RESULTS

With dPCA, almost all aberrations can be extracted and compensated without requirements on background segmentation, making it efficient and convenient.

CONCLUSIONS

It solves the problem that the conventional principal component analysis (PCA) algorithm cannot compensate the common but intricate higher order cross-term aberrations, such as astigmatism and coma. Moreover, the dPCA strategy proposed here is not only suitable for aberration compensation but also applicable for other cases where there exist cross-terms that cannot be analyzed with the PCA algorithm.

Simulation of digital holographic recording and reconstruction using a generalized matrix method

In recent years, research efforts in the field of digital holography have expanded significantly, due to the ability to obtain high-resolution intensity and phase images. The information contained in these images have become of great interest to the machine learning community, with applications spanning a wide portfolio of research areas, including bioengineering. In this work, we seek to demonstrate a high-fidelity simulation of holographic recording. By accurately and numerically simulating the propagation of a coherent light source through a series of optical elements and the object itself, we accurately predict the optical interference of the object and reference wave at the recording plane, including diffraction effects, aberrations, and speckle. We show that the optical transformation that predicts the complex field at the recording plane can be generalized for arbitrary holographic recording configurations using a matrix method. In addition, we provide a detailed description of digital phase reconstruction and aberration compensation for a variety of off-axis holographic configurations. Reconstruction errors are presented for the various holographic recording geometries and complex field objects. While the primary objective of this work is not to evaluate phase reconstruction approaches, the reconstruction of simulated holograms provides validation of the generalized simulation method. The long-term goal of this work is that the generalized holographic simulation motivates the use of phase reconstruction of the simulated holograms to populate databases for training machine-learning algorithms aimed at classifying relevant objects recorded through a variety of holographic setups.

Morphology, Motility, and Cytoskeletal Architecture of Breast Cancer Cells Depend on Keratin 19 and Substrate

Cancer cells gain motility through events that accompany modulation of cell shape and include altered expression of keratins. However, the role of keratins in change of cancer cell architecture is not well understood. Therefore, we ablated the expression of keratin 19 (K19) in breast cancer cells of the MDA-MB-231 cell line and found that cells lacking K19 become more elongated in culture, with morphological reversion toward the parental phenotype upon transduction of KRT19. Also, the number of actin stress fibers and focal adhesions were significantly reduced in KRT19 knockout (KO) cells. The altered morphology of KRT19 KO cells was then characterized quantitatively using digital holographic microscopy (DHM), which not only confirmed the phenotypic change of KRT19 KO cells but also identified that the K19-dependent morphological change is dependent on the substrate type. A new quantitative method of single cell analysis from DHM, via average phase difference maps, facilitated evaluation of K19-substrate interactive effects on cell morphology. When plated on collagen substrate, KRT19 KO cells were less elongated and resembled parental cells. Assessing single cell motility further showed that while KRT19 KO cells moved faster than parental cells on a rigid surface, this increase in motility became abrogated when cells were plated on collagen. Overall, our study suggests that K19 inhibits cell motility by regulating cell shape in a substrate-dependent manner. Thus, this study provides a potential basis for the altered expression of keratins associated with change in cell shape and motility of cancer cells. © 2020 International Society for Advancement of Cytometry.

Quantitative scoring of epithelial and mesenchymal qualities of cancer cells using machine learning and quantitative phase imaging

SIGNIFICANCE

We introduce an application of machine learning trained on optical phase features of epithelial and mesenchymal cells to grade cancer cells' morphologies, relevant to evaluation of cancer phenotype in screening assays and clinical biopsies.

AIM

Our objective was to determine quantitative epithelial and mesenchymal qualities of breast cancer cells through an unbiased, generalizable, and linear score covering the range of observed morphologies.

APPROACH

Digital holographic microscopy was used to generate phase height maps of noncancerous epithelial (Gie-No3B11) and fibroblast (human gingival) cell lines, as well as MDA-MB-231 and MCF-7 breast cancer cell lines. Several machine learning algorithms were evaluated as binary classifiers of the noncancerous cells that graded the cancer cells by transfer learning.

RESULTS

Epithelial and mesenchymal cells were classified with 96% to 100% accuracy. Breast cancer cells had scores in between the noncancer scores, indicating both epithelial and mesenchymal morphological qualities. The MCF-7 cells skewed toward epithelial scores, while MDA-MB-231 cells skewed toward mesenchymal scores. Linear support vector machines (SVMs) produced the most distinct score distributions for each cell line.

CONCLUSIONS

The proposed epithelial-mesenchymal score, derived from linear SVM learning, is a sensitive and quantitative approach for detecting epithelial and mesenchymal characteristics of unknown cells based on well-characterized cell lines. We establish a framework for rapid and accurate morphological evaluation of single cells and subtle phenotypic shifts in imaged cell populations.

Machine Learning with Optical Phase Signatures for Phenotypic Profiling of Cell Lines

Robust and reproducible profiling of cell lines is essential for phenotypic screening assays. The goals of this study were to determine robust and reproducible optical phase signatures of cell lines for classification with machine learning and to correlate optical phase parameters to motile behavior. Digital holographic microscopy (DHM) reconstructed phase maps of cells from two pairs of cancer and non-cancer cell lines. Seventeen image parameters were extracted from each cell's phase map, used for linear support vector machine learning, and correlated to scratch wound closure and Boyden chamber chemotaxis. The classification accuracy was between 90% and 100% for the six pairwise cell line comparisons. Several phase parameters correlated with wound closure rate and chemotaxis across the four cell lines. The level of cell confluence in culture affected phase parameters in all cell lines tested. Results indicate that optical phase features of cell lines are a robust set of quantitative data of potential utility for phenotypic screening and prediction of motile behavior. © 2019 International Society for Advancement of Cytometry.

Fast phase retrieval in off-axis digital holographic microscopy through deep learning

Traditional digital holographic imaging algorithms need multiple iterations to obtain focused reconstructed image, which is time-consuming. In terms of phase retrieval, there is also the problem of phase compensation in addition to focusing task. Here, a new method is proposed for fast digital focus, where we use U-type convolutional neural network (U-net) to recover the original phase of microscopic samples. Generated data sets are used to simulate different degrees of defocused image, and verify that the U-net can restore the original phase to a great extent and realize phase compensation at the same time. We apply this method in the construction of real-time off-axis digital holographic microscope and obtain great breakthroughs in imaging speed.

Design of Metamaterial Absorber using Eight-Resistive-Arm Cell for Simultaneous Broadband and Wide-Incidence-Angle Absorption

In this paper, a broadband metamaterial (MM) absorber is presented for X-band applications. A novel eight-resistive-arm (ERA) cell is proposed as an MM unit cell to achieve both broadband absorption and wide incidence angles. The proposed ERA cell is designed using equivalent circuit model and full-wave analysis in order to achieve an absorption ratio higher than 90% in the range of 8.2–13.4 GHz. The experimental results indicate that the absorptivity was greater than 90% in the range of 8–13 GHz for all polarization angles under normal incidence. Under oblique incidence, the measured absorptivity was greater than 90% in the range of 8.2–12.2 GHz up to 60° and in the range of 9.2–12 GHz up to 65° in the transverse electric (TE) mode. In the transverse magnetic (TM) mode, the measured absorptivity was higher than 90% in the range of 9.5–12.4 GHz when the incidence angle was varied from 0° to 60° and remaining a 90% absorption bandwidth in the range of 10–12 GHz up to 65°. Compared to other broadband MM absorbers, the proposed MM absorber exhibited the widest incidence angles in both TE and TM modes.

Quantitative assessment of cancer cell morphology and motility using telecentric digital holographic microscopy and machine learning

The noninvasive, fast acquisition of quantitative phase maps using digital holographic microscopy (DHM) allows tracking of rapid cellular motility on transparent substrates. On two-dimensional surfaces in vitro, MDA-MB-231 cancer cells assume several morphologies related to the mode of migration and substrate stiffness, relevant to mechanisms of cancer invasiveness in vivo. The quantitative phase information from DHM may accurately classify adhesive cancer cell subpopulations with clinical relevance. To test this, cells from the invasive breast cancer MDA-MB-231 cell line were cultured on glass, tissue-culture treated polystyrene, and collagen hydrogels, and imaged with DHM followed by epifluorescence microscopy after staining F-actin and nuclei. Trends in cell phase parameters were tracked on the different substrates, during cell division, and during matrix adhesion, relating them to F-actin features. Support vector machine learning algorithms were trained and tested using parameters from holographic phase reconstructions and cell geometric features from conventional phase images, and used to distinguish between elongated and rounded cell morphologies. DHM was able to distinguish between elongated and rounded morphologies of MDA-MB-231 cells with 94% accuracy, compared to 83% accuracy using cell geometric features from conventional brightfield microscopy. This finding indicates the potential of DHM to detect and monitor cancer cell morphologies relevant to cell cycle phase status, substrate adhesion, and motility. © 2017 International Society for Advancement of Cytometry.

Bandwidth-enhanced and Wide-angle-of-incidence Metamaterial Absorber using a Hybrid Unit Cell

In this paper, a bandwidth-enhanced and wide-angle-of-incidence metamaterial absorber is proposed using a hybrid unit cell. Owing to symmetric unit cells, high absorptivity is maintained for all polarization angles. A circular-sector unit cell enables high absorptivity under the oblique incidence of both transverse electric (TE) and transverse magnetic (TM) modes. To enhance the bandwidth, we introduced a hybrid unit cell comprising four circular sectors. Two sectors resonate at 10.38 GHz, and two resonate at 10.55 GHz. Since the two absorption frequencies are near each other, the bandwidth increases. The proposed idea is demonstrated with both full-wave simulations and measurements. The simulated absorptivity exceeds 91% around 10.45 GHz at an angle of incidence up to 70° in both TM and TE polarizations. The measured absorptivity at 10.45 GHz is close to 96.5% for all polarization angles under normal incidence. As the angle of incidence changes from 0° to 70°, the measured absorptivity at 10.45 GHz remains above 90% in the TE mode and higher than 94% in the TM mode. Under an oblique incidence, the measured 90% absorption bandwidth is 1.95% from 10.1-10.2 GHz and 10.4-10.5 GHz up to 70° at the TE mode, and 3.39% from 10.15-10.5 GHz up to 70° at the TM mode.

One-step real-virtual combined reflection hologram: a 4f relay approach

The production of conventional optical reflection holograms can be classified into the one-step method or the two-step method. In the one-step method, only the diverging light of an object can be recorded, and the reconstructed scene is a virtual one behind the recording medium. In the two-step method, the diverging light or the converging light can be recorded alternatively. However, the process is complicated considering double exposures. The object is first imaged by a 4f system, and then the interference patterns are recorded by single exposure. The reconstructed image can be either a virtual image behind the recording medium or a real image in front of the recording medium. The ideal imaging property of a 4f system has been demonstrated theoretically, and the proposed method has been verified experimentally.

Three-dimensional imaging of distribution of refractive index by parallel phase-shifting digital holography using Abel inversion

Although digital holography is a powerful technique obtaining a phase image of a transparent object, the image reconstructed by the technique merely expresses phase distribution of the light wave after transmitting through the object. Phase variation of inside of the object is difficult to be obtained. Then, we applied Abel inversion method to the high-speed phase image of a dynamic transparent object assumed axially symmetric. The phase is accurately recorded by phase-shifting method. We experimentally recorded transparent dynamic gas flow, assumed axially symmetric along the direction in which gas flowed, at 3,000 frame/s and reconstructed motion picture of 3D distribution of the refractive index of the gas from the high-speed phase motion picture obtained by parallel phase-shifting digital holography.

Simple and flexible phase compensation for digital holographic microscopy with electrically tunable lens

In a digital holographic microscopy (DHM) system, different microscope objectives (MOs) will introduce different phase distortions and thus lead to measurement errors. To address this problem, we present a simple and flexible method to compensate all phase distortions by introducing an electrically tunable lens (ETL) in the reference arm for a DHM system with multiple MOs. By exactly controlling the external currents of the ETL, we can change the reference wave front to match the wave front introduced by different MOs without complex alignment or additional numerical postprocessing manipulations. This method is suitable for quantitative real-time phase imaging especially when it refers to multiple MOs. To demonstrate the validity and effectiveness of our scheme, we did a series of simulations and carried out some real experiments with two different MOs (4× and 10×).

Automatic phase aberration compensation for digital holographic microscopy based on deep learning background detection

We propose a fully automatic technique to obtain aberration free quantitative phase imaging in digital holographic microscopy (DHM) based on deep learning. The traditional DHM solves the phase aberration compensation problem by manually detecting the background for quantitative measurement. This would be a drawback in real time implementation and for dynamic processes such as cell migration phenomena. A recent automatic aberration compensation approach using principle component analysis (PCA) in DHM avoids human intervention regardless of the cells' motion. However, it corrects spherical/elliptical aberration only and disregards the higher order aberrations. Traditional image segmentation techniques can be employed to spatially detect cell locations. Ideally, automatic image segmentation techniques make real time measurement possible. However, existing automatic unsupervised segmentation techniques have poor performance when applied to DHM phase images because of aberrations and speckle noise. In this paper, we propose a novel method that combines a supervised deep learning technique with convolutional neural network (CNN) and Zernike polynomial fitting (ZPF). The deep learning CNN is implemented to perform automatic background region detection that allows for ZPF to compute the self-conjugated phase to compensate for most aberrations.

Versatile spectral modulation of a broadband source for digital holographic microscopy

We demonstrate the potential of spatial light modulators for the spectral control of a broadband source in digital holographic microscopy. Used in a 'pulse-shaping' geometry, the spatial light modulator provides a versatile control over the bandwidth and wavelength of the light source. The control of these properties enables adaptation to various experimental conditions. As a first application, we show that the source bandwidth can be adapted to the off-axis geometry to provide quantitative phase imaging over the whole field of view. As a second application, we generate sequences of appropriate wavelengths for a hierarchical optical phase unwrapping algorithm, which enables the measurement of the topography of high-aspect ratio structures without phase ambiguity. Examples are given with step heights up to 50 µm.