Optically-sectioned two-shot structured illumination microscopy with Hilbert-Huang processing

Superresolution structured illumination microscopy reconstruction algorithms: a review

Structured illumination microscopy (SIM) has become the standard for next-generation wide-field microscopy, offering ultrahigh imaging speed, superresolution, a large field-of-view, and long-term imaging. Over the past decade, SIM hardware and software have flourished, leading to successful applications in various biological questions. However, unlocking the full potential of SIM system hardware requires the development of advanced reconstruction algorithms. Here, we introduce the basic theory of two SIM algorithms, namely, optical sectioning SIM (OS-SIM) and superresolution SIM (SR-SIM), and summarize their implementation modalities. We then provide a brief overview of existing OS-SIM processing algorithms and review the development of SR-SIM reconstruction algorithms, focusing primarily on 2D-SIM, 3D-SIM, and blind-SIM. To showcase the state-of-the-art development of SIM systems and assist users in selecting a commercial SIM system for a specific application, we compare the features of representative off-the-shelf SIM systems. Finally, we provide perspectives on the potential future developments of SIM.

Fringe analysis: single-shot or two-frames? Quantitative phase imaging answers

Conditions of the digital recording of the fringe pattern determine the phase reconstruction procedure, which in turn directly shapes the final accuracy and throughput of the full-field (non-scanning) optical measurement technique and defines the system capabilities. In this way, the fringe pattern analysis plays a crucial role in the ubiquitous optical measurements and thus is under constant development focused on high temporal/spatial resolution. It is especially valuable in the quantitative phase imaging technology, which emerged in the high-contrast label-free biomedical microscopy. In this paper, I apply recently blossomed two-frame phase-shifting techniques to the QPI and merge them with advanced adaptive interferogram pre-filtering algorithms. Next, I comprehensively test such frameworks against classical and adaptive single-shot methods applied for phase reconstruction in dynamic QPI enabling highest phase time-space-bandwidth product. The presented study systematically tackles important question: what is the gain, if any, in QPI realized by recording two phase-shifted interferograms? Counterintuitively, the results show that single-shot demodulation exhibited higher phase reconstruction accuracy than two-frame phase-shifting methods in low and medium interferogram signal-to-noise ratio regimes. Thus, the single-shot approach is promoted due to not only high temporal resolution but also larger phase-information throughput. Additionally, in the majority of scenarios, the best option is to shift the paradigm and employ two-frame pre-filtering rather than two-frame phase retrieval. Experimental fringe analysis in QPI of LSEC/RWPE cell lines successfully corroborated all novel numerical findings. Hence, the presented numerical-experimental research advances the important field of fringe analysis solutions for optical full-field measurement methods with widespread bio-engineering applications.

High-speed volumetric imaging in vivo based on structured illumination microscopy with interleaved reconstruction

Wide-field fluorescence microscopy (WFFM) is widely adopted in biomedical studies, due to its high imaging speed over large field-of-views. However, WFFM is susceptible to out-of-focus background. To overcome this problem, structured illumination microscopy (SIM) was proposed as a wide-field, optical-sectioning technique, which needs multiple raw images for image reconstruction and thus has a lower imaging speed. Here we propose SIM with interleaved reconstruction, to make SIM of lossless speed. We apply this method in volumetric imaging of neural network dynamics in brains of zebrafish larva in vivo.

Deep learning based one-shot optically-sectioned structured illumination microscopy for surface measurement

Optically-sectioned structured illumination microscopy (OS-SIM) is broadly used for biological imaging and engineering surface measurement owing to its simple, low-cost, scanning-free experimental setup and excellent optical sectioning capability. However, the efficiency of current optically-sectioned methods in OS-SIM is yet limited for surface measurement because a set of wide-field images under uniform or structured illumination are needed to derive an optical section at each scanning height. In this paper, a deep-learning-based one-shot optically-sectioned method, called Deep-OS-SIM, is proposed to improve the efficiency of OS-SIM for surface measurement. Specifically, we develop a convolutional neural network (CNN) to learn the statistical invariance of optical sectioning across structured illumination images. By taking full advantage of the high entropy properties of structured illumination images to train the CNN, fast convergence and low training error are achieved in our method even for low-textured surfaces. The well-trained CNN is then applied to a plane mirror for testing, demonstrating the ability of the method to reconstruct high-quality optical sectioning from only one instead of two or three raw structured illumination frames. Further measurement experiments on a standard step and milled surface show that the proposed method has similar accuracy to OS-SIM techniques but with higher imaging speed.

Automatic fringe pattern enhancement using truly adaptive period-guided bidimensional empirical mode decomposition

Fringe patterns encode the information about the result of a measurement performed via widely used optical full-field testing methods, e.g., interferometry, digital holographic microscopy, moiré techniques, structured illumination etc. Affected by the optical setup, changing environment and the sample itself fringe patterns are often corrupted with substantial noise, strong and uneven background illumination and exhibit low contrast. Fringe pattern enhancement, i.e., noise minimization and background term removal, at the pre-processing stage prior to the phase map calculation (for the measurement result decoding) is therefore essential to minimize the jeopardizing effect the mentioned error sources have on the optical measurement outcome. In this contribution we propose an automatic, robust and highly effective fringe pattern enhancement method based on the novel period-guided bidimensional empirical mode decomposition algorithm (PG-BEMD). The spatial distribution of the fringe period is estimated using the novel windowed approach and then serves as an indicator for the truly adaptive decomposition with the filter size locally adjusted to the fringe pattern density. In this way the fringe term is successfully extracted in a single (first) decomposition component alleviating the cumbersome mode mixing phenomenon and greatly simplifying the automatic signal reconstruction. Hence, the fringe term is dissected without the need for modes selection nor summation. The noise removal robustness is ensured employing the block matching 3D filtering of the fringe pattern prior to its decomposition. Performance validation against previously reported modified empirical mode decomposition techniques is provided using numerical simulations and experimental data verifying the versatility and effectiveness of the proposed approach.

Single-shot two-frame π-shifted spatially multiplexed interference phase microscopy

Single-shot, two-frame, π-shifted spatially multiplexed interference microscopy (π-SMIM) is presented as an improvement to previous SMIM implementations, introducing a versatile, robust, fast, and accurate method for cumbersome, noisy, and low-contrast phase object analysis. The proposed π-SMIM equips a commercially available nonholographic microscope with a high-speed (video frame rate) enhanced quantitative phase imaging (QPI) capability by properly placing a beam-splitter in the microscope embodiment to simultaneously (in a single shot) record two holograms mutually phase shifted by π radians at the expense of reducing the field of view. Upon subsequent subtractive superimposition of holograms, a π-hologram is generated with reduced background and improved modulation of interference fringes. These features determine superior phase retrieval quality, obtained by employing the Hilbert spiral transform on the π-hologram, as compared with a single low-quality (low signal-to-noise ratio) hologram analysis. In addition, π-SMIM enables accurate in-vivo analysis of high dynamic range phase objects, otherwise measurable only in static regime using time-consuming phase-shifting. The technique has been validated utilizing a 20  ×    /  0.46 NA objective in a regular Olympus BX-60 upright microscope for QPI of different lines of prostate cancer cells and flowing microbeads.

A concept for single-shot volumetric fluorescence imaging via orthogonally polarized excitation lattices

The deconvolution of widefield fluorescence images provides only guesses of spatial frequency information along the optical axis due to the so called missing cone in the optical transfer function. Retaining the single-shot imaging speed of deconvolution microscopy while gaining access to missing cone information is thus highly desirable for microscopy of volumetric samples. Here, we present a concept that superimposes two orthogonally polarized excitation lattices with a phase-shift of p between them. In conjunction with a non-iterative image reconstruction algorithm this permits the restoration of missing cone information. We show how fluorescence anisotropy could be used as a method to encode and decode the patterns simultaneously and develop a rigorous theoretical framework for the method. Through in-silico experiments and imaging of fixed biological cells on a structured illumination microscope that emulates the proposed setup we validate the feasibility of the method.

Self-adaptive terahertz spectroscopy from atmospheric vapor based on Hilbert-Huang transform

Absorption lines of atmospheric vapor commonly appear in terahertz (THz) spectra measured in a humid air environment. However, these effects are generally undesirable because they may mask critical spectroscopic information. Here, a self-adaptive method is demonstrated for effectively identifying and eliminating atmospheric vapor noise from THz spectra of an all-fiber THz system with the Hilbert-Huang transform. The THz signal was decomposed into eight components in different time scales called the intrinsic mode functions and the interference of atmospheric vapor was accurately isolated. A series of experiments confirmed the effectiveness and strong self-adaptiveness of the proposed system in vapor noise elimination.

Fast two-snapshot structured illumination for temporal focusing microscopy with enhanced axial resolution

We present a new two-snapshot structured light illumination (SLI) reconstruction algorithm for fast image acquisition. The new algorithm, which only requires two mutually π phase-shifted raw structured images, is implemented on a custom-built temporal focusing fluorescence microscope (TFFM) to enhance its axial resolution via a digital micromirror device (DMD). First, the orientation of the modulated sinusoidal fringe patterns is automatically identified via spatial frequency vector detection. Subsequently, the modulated in-focal-plane images are obtained via rotation and subtraction. Lastly, a parallel amplitude demodulation method, derived based on Hilbert transform, is applied to complete the decoding processes. To demonstrate the new SLI algorithm, a TFFM is custom-constructed, where a DMD replaces the generic blazed grating in the system and simultaneously functions as a diffraction grating and a programmable binary mask, generating arbitrary fringe patterns. The experimental results show promising depth-discrimination capability with an axial resolution enhancement factor of 1.25, which matches well with the theoretical estimation, i.e, 1.27. Imaging experiments on pollen grain and mouse kidney samples have been performed. The results indicate that the two-snapshot algorithm presents comparable contrast reconstruction and optical cross-sectioning capability than those adopting the conventional root-mean-square (RMS) reconstruction method. The two-snapshot method can be readily applied to any sinusoidally modulated illumination systems to realize high-speed 3D imaging as less frames are required for each in-focal-plane image restoration, i.e., the image acquisition speed is improved by 2.5 times for any two-photon systems.

Resolution enhancement of the microscopic imaging by unknown sinusoidal structured illumination with iterative algorithm

Microscopy by sinusoidal structured illumination is a conventional method to improve resolution, which largely depends on accurate knowledge of the illumination pattern. Two steps are included in the reconstruction process of our proposed technique. The first step solves the parameters of the structured illumination in the spatial domain. Besides the phase-shifting amounts, the period, the modulation factor, and the background intensity of the pattern are extracted from three segmented raw images by the iterative algorithm. The second step is retrieval and synthesis of the low- and high-frequency information of the object in the Fourier space with obtained data. Since the unknown object information is not involved in the pattern parameters' solving process, it is possible to figure out the problem with higher precision and less requirements. We test the performance of this method in the experiments. The resolution is improved with the designed carrier frequency of the illumination pattern.

Quantitative phase imaging by single-shot Hilbert-Huang phase microscopy

We propose a novel single-shot Hilbert-Huang transform-based algorithm applied to digital holographic microscopy (DHM) for robust, fast, and accurate single-shot quantitative phase imaging in on-axis and off-axis configurations. Fringe pattern with possible defects and closed fringes are adaptively filtered and accurately phase demodulated using local fringe direction estimation. Experimental validation of the proposed techniques is presented as the DHM study of microbeads and red blood cells phase samples. Obtained results compare very favorably with the Fourier approach (off-axis) and temporal phase shifting (on-axis).

Two-frame phase-shifting interferometry for testing optical surfaces

Standard phase-shifting interferometry (PSI) generally requires collecting at least three phase-shifted interferograms to extract the physical quantity being measured. Here, we propose the application of a simple two-frame PSI for the testing of a range of optical surfaces, including flats, spheres, and aspheres. The two-frame PSI extracts modulated phase from two randomly phase-shifted interferograms using a Gram-Schmidt algorithm, and can work in either null testing or non-null testing modes. Since only two interferograms are used for phase demodulation and the phase shift amount can be random, requirements on environmental conditions and phase shifter calibration are greatly relaxed. Experimental results of three different mirrors suggest that the two-frame PSI can achieve comparable measurement precision with conventional multi-frame PSI, but has faster data acquisition speed and less stringent hardware requirements. The proposed two-frame PSI expands the flexibility of PSI and holds great potential in many applications.

Demodulation of two-shot fringe patterns with random phase shifts by use of orthogonal polynomials and global optimization

We propose a simple and robust phase demodulation algorithm for two-shot fringe patterns with random phase shifts. Based on a smoothness assumption, the phase to be recovered is decomposed into a linear combination of finite terms of orthogonal polynomials, and the expansion coefficients and the phase shift are exhaustively searched through global optimization. The technique is insensitive to noise or defects, and is capable of retrieving phase from low fringe-number (less than one) or low-frequency interferograms. It can also cope with interferograms with very small phase shifts. The retrieved phase is continuous and no further phase unwrapping process is required. The method is expected to be promising to process interferograms with regular fringes, which are common in optical shop testing. Computer simulation and experimental results are presented to demonstrate the performance of the algorithm.

Double-exposure optical sectioning structured illumination microscopy based on Hilbert transform reconstruction

Structured illumination microscopy (SIM) with axially optical sectioning capability has found widespread applications in three-dimensional live cell imaging in recent years, since it combines high sensitivity, short image acquisition time, and high spatial resolution. To obtain one sectioned slice, three raw images with a fixed phase-shift, normally 2π/3, are generally required. In this paper, we report a data processing algorithm based on the one-dimensional Hilbert transform, which needs only two raw images with arbitrary phase-shift for each single slice. The proposed algorithm is different from the previous two-dimensional Hilbert spiral transform algorithm in theory. The presented algorithm has the advantages of simpler data processing procedure, faster computation speed and better reconstructed image quality. The validity of the scheme is verified by imaging biological samples in our developed DMD-based LED-illumination SIM system.

Two-shot fringe pattern phase-amplitude demodulation using Gram-Schmidt orthonormalization with Hilbert-Huang pre-filtering

Gram-Schmidt orthonormalization is a very fast and efficient method for the fringe pattern phase demodulation. It requires only two arbitrarily phase-shifted frames. Images are treated as vectors and upon orthogonal projection of one fringe vector onto another the quadrature fringe pattern pair is obtained. Orthonormalization process is very susceptible, however, to noise, uneven background and amplitude modulation fluctuations. The Hilbert-Huang transform based preprocessing is proposed to enhance fringe pattern phase demodulation by filtering out the spurious noise and background illumination and performing fringe normalization. The Gram-Schmidt orthonormalization process error analysis is provided and its filtering-expanded capabilities are corroborated analyzing DSPI fringes and performing amplitude demodulation of Bessel fringes. Synthetic and experimental fringe pattern analyses presented to validate the proposed technique show that it compares favorably with other pre-filtering schemes, i.e., Gaussian filtering and continuous wavelet transform.