Image filtering in structured illumination microscopy using the Lukosz bound

Electron-beam patterned calibration structures for structured illumination microscopy

Super-resolution fluorescence microscopy can be achieved by image reconstruction after spatially patterned illumination or sequential photo-switching and read-out. Reconstruction algorithms and microscope performance are typically tested using simulated image data, due to a lack of strategies to pattern complex fluorescent patterns with nanoscale dimension control. Here, we report direct electron-beam patterning of fluorescence nanopatterns as calibration standards for super-resolution fluorescence. Patterned regions are identified with both electron microscopy and fluorescence labelling of choice, allowing precise correlation of predefined pattern dimensions, a posteriori obtained electron images, and reconstructed super-resolution images.

Nanometer-Scale Molecular Mapping by Super-resolution Fluorescence Microscopy

The structural organization of macromolecules and their association in assemblies and organelles is key to understand cellular function. Super-resolution fluorescence microscopy has expanded our toolbox for examining such nanometer-scale cellular structures, by enabling positional mapping of proteins in situ. Here, we detail the workflow to build nanometer-scale maps focusing on two complementary super-resolution modalities: structured illumination microscopy (SIM) and stochastic optical reconstruction microscopy (STORM).

Structured illumination microscopy with noise-controlled image reconstructions

Super-resolution structured illumination microscopy (SIM) has become a widely used method for biological imaging. Standard reconstruction algorithms, however, are prone to generate noise-specific artifacts that limit their applicability for lower signal-to-noise data. Here we present a physically realistic noise model that explains the structured noise artifact, which we then use to motivate new complementary reconstruction approaches. True-Wiener-filtered SIM optimizes contrast given the available signal-to-noise ratio, and flat-noise SIM fully overcomes the structured noise artifact while maintaining resolving power. Both methods eliminate ad hoc user-adjustable reconstruction parameters in favor of physical parameters, enhancing objectivity. The new reconstructions point to a trade-off between contrast and a natural noise appearance. This trade-off can be partly overcome by further notch filtering but at the expense of a decrease in signal-to-noise ratio. The benefits of the proposed approaches are demonstrated on focal adhesion and tubulin samples in two and three dimensions, and on nanofabricated fluorescent test patterns.

Impact of partial coherence on the apparent optical transfer function derived from the response to amplitude edges

We present an investigation of the impact of partial coherence on optical imaging systems with the focus on whole slide imaging (WSI) systems for digital pathology. The investigation is based on the analysis of the edge response of the optical system, which gives rise to an apparent optical transfer function (OTF) that can be linked to two elementary complex functions Q and U. The function Q is directly related to the transmission cross coefficient (TCC) and can be identified with the performance function first introduced by Kintner and Sillitto. The function U depends on the TCC in a more involved way. When there are no aberrations the Q-function corresponds to the real part of the apparent OTF and the U function to the imaginary part of the apparent OTF. Close to the incoherent limit the effect of the U function is a mere shift of the edge compared to the fully incoherent case. We propose a new expression for the dependence of the depth of focus (DOF) on spatial frequency and on the partial coherence factor σ, and validate it by simulation. Partial coherence effects are investigated experimentally on a WSI system with a compact LED-based Köhler illumination unit with variable condenser NA. This unit incorporates a top hat diffuser for providing a reasonably uniform illumination field, with variations below 10% across the imaged field of view. The measurements of the apparent through-focus OTF derived from edges on a custom resolution chart for different σ were substantially in agreement with the simulations. Finding an optimal value for σ is not straightforward as lateral resolution and the level of edge ringing improve with increasing σ, whereas edge contrast and DOF improve with decreasing σ. We assess that the trade-off for the particular application of WSI systems for digital pathology is optimized for a σ value in the range of 0.55-0.75.

Deconvolution methods for structured illumination microscopy

We compare two recently developed multiple-frame deconvolution approaches for the reconstruction of structured illumination microscopy (SIM) data: the pattern-illuminated Fourier ptychography algorithm (piFP) and the joint Richardson-Lucy deconvolution (jRL). The quality of the images reconstructed by these methods is compared in terms of the achieved resolution improvement, noise enhancement, and inherent artifacts. Furthermore, we study the issue of object-dependent resolution improvement by considering the modulation transfer functions derived from different types of objects. The performance of the considered methods is tested in experiments and benchmarked with a commercial SIM microscope. We find that the piFP method resolves periodic and isolated structures equally well, whereas the jRL method provides significantly higher resolution for isolated objects compared to periodic ones. Images reconstructed by the piFP and jRL algorithms are comparable to the images reconstructed using the generalized Wiener filter applied in most commercial SIM microscopes. An advantage of the discussed algorithms is that they allow the reconstruction of SIM images acquired under different types of illumination, such as multi-spot or random illumination.

Studying different illumination patterns for resolution improvement in fluorescence microscopy

Various types of non-uniform illumination can be used for resolution improvement in fluorescence microscopy. Here we study the differences between several types of incoherent illumination patterns, such as multi-spot, line and pseudo-random patterns. This requires an imaging setup and an image reconstruction algorithm that are flexible enough to incorporate any type of illumination pattern. We employ fluorescence microscope with structured illumination generated by a Digital Micro-mirror Device (DMD) and the pattern-illuminated Fourier Ptychography reconstruction algorithm (piFP) to this end. The piFP method is modified and improved by identifying the algorithm as steepest descent optimization of a least squares function. We find that illumination patterns with regular structure are superior to those with irregular structure in terms of resolution enhancement and noise level in the reconstructed images.

SIMToolbox: a MATLAB toolbox for structured illumination fluorescence microscopy

SIMToolbox is an open-source, modular set of functions for MATLAB equipped with a user-friendly graphical interface and designed for processing two-dimensional and three-dimensional data acquired by structured illumination microscopy (SIM). Both optical sectioning and super-resolution applications are supported. The software is also capable of maximum a posteriori probability image estimation (MAP-SIM), an alternative method for reconstruction of structured illumination images. MAP-SIM can potentially reduce reconstruction artifacts, which commonly occur due to refractive index mismatch within the sample and to imperfections in the illumination.

AVAILABILITY AND IMPLEMENTATION

SIMToolbox, example data and the online documentation are freely accessible at http://mmtg.fel.cvut.cz/SIMToolbox.

CONTACT

ghagen@uccs.edu

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Super-Resolution Scanning Laser Microscopy Based on Virtually Structured Detection

Light microscopy plays a key role in biological studies and medical diagnosis. The spatial resolution of conventional optical microscopes is limited to approximately half the wavelength of the illumination light as a result of the diffraction limit. Several approaches-including confocal microscopy, stimulated emission depletion microscopy, stochastic optical reconstruction microscopy, photoactivated localization microscopy, and structured illumination microscopy-have been established to achieve super-resolution imaging. However, none of these methods is suitable for the super-resolution ophthalmoscopy of retinal structures because of laser safety issues and inevitable eye movements. We recently experimentally validated virtually structured detection (VSD) as an alternative strategy to extend the diffraction limit. Without the complexity of structured illumination, VSD provides an easy, low-cost, and phase artifact-free strategy to achieve super-resolution in scanning laser microscopy. In this article we summarize the basic principles of the VSD method, review our demonstrated single-point and line-scan super-resolution systems, and discuss both technical challenges and the potential of VSD-based instrumentation for super-resolution ophthalmoscopy of the retina.

Three-dimensional structured illumination microscopy using Lukosz bound apodization reduces pixel negativity at no resolution cost

The quality of the reconstructed image in structured illumination microscopy (SIM) depends on various aspects of the image filtering process. To optimize the trade-off between resolution and ringing artifacts, which lead to negative intensities, we extend Lukosz-bound filtering to 3D SIM and derive the parametrization of the 3D SIM cut-off. We compare the use of the Lukosz-bound as apodization filter to triangular apodization and find a tenfold reduction in the most negative pixel value with a minimal resolution loss. We test this algorithm on experimental SIM images of tubulin filaments and DAPI stained DNA structure in cancer cells and find a substantial reduction in the most negative pixel value and the percentage of pixels with a negative value. This means that there is no longer a need to clip the final image to avoid these negative pixel values.