Interpretation of the optical transfer function: Significance for image scanning microscopy

A statistical resolution measure of fluorescence microscopy with finite photons

First discovered by Ernest Abbe in 1873, the resolution limit of a far-field microscope is considered determined by the numerical aperture and wavelength of light, approximatelyλ 2 N A . With the advent of modern fluorescence microscopy and nanoscopy methods over the last century, this definition is insufficient to fully describe a microscope's resolving power. To determine the practical resolution limit of a fluorescence microscope, photon noise remains one essential factor yet to be incorporated in a statistics-based theoretical framework. We proposed an information density measure quantifying the theoretical resolving power of a fluorescence microscope in the condition of finite photons. The developed approach not only allows us to quantify the practical resolution limit of various fluorescence and super-resolution microscopy modalities but also offers the potential to predict the achievable resolution of a microscopy design under different photon levels.

Signal strength and integrated intensity in confocal and image scanning microscopy

The properties of signal strength and integrated intensity in a scanned imaging system are reviewed. These properties are especially applied to confocal imaging systems, including image scanning microscopy. The integrated intensity, equal to the image of a uniform planar (sheet) object, rather than the peak of the point spread function, is a measure of the flux in an image. Analytic expressions are presented for the intensity in the detector plane for a uniform volume object, and for the resulting background. The variation in the integrated intensity with defocus for an offset point detector is presented. This axial fingerprint is independent of any pixel reassignment. The intensity in the detector plane is shown to contain the defocus information, and simple processing of the recorded data can improve optical sectioning and background rejection.

Answering some questions about structured illumination microscopy

Structured illumination microscopy (SIM) provides images of fluorescent objects at an enhanced resolution greater than that of conventional epifluorescence wide-field microscopy. Initially demonstrated in 1999 to enhance the lateral resolution twofold, it has since been extended to enhance axial resolution twofold (2008), applied to live-cell imaging (2009) and combined with myriad other techniques, including interferometric detection (2008), confocal microscopy (2010) and light sheet illumination (2012). Despite these impressive developments, SIM remains, perhaps, the most poorly understood 'super-resolution' method. In this article, we provide answers to the 13 questions regarding SIM proposed by Prakash et al. along with answers to a further three questions. After providing a general overview of the technique and its developments, we explain why SIM as normally used is still diffraction-limited. We then highlight the necessity for a non-polynomial, and not just nonlinear, response to the illuminating light in order to make SIM a true, diffraction-unlimited, super-resolution technique. In addition, we present a derivation of a real-space SIM reconstruction approach that can be used to process conventional SIM and image scanning microscopy (ISM) data and extended to process data with quasi-arbitrary illumination patterns. Finally, we provide a simple bibliometric analysis of SIM development over the past two decades and provide a short outlook on potential future work. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 2)'.

A hyper-resolving polynomial aperture and its application in microscopy

Background

A hyper-resolving aperture composed of a polynomial distribution is suggested. The point spread function (PSF) is computed and compared with that corresponding to linear, quadratic, and circular apertures. In addition, the influence of the number of zones on the PSF is discussed. An application on confocal scanning laser microscope using Siemen’s star pattern as an object considering the polynomial apertures is given.

Results

We have made polynomial apertures using MATLAB code, and we tested the resolution from the computation of the cut-off spatial frequency obtained from the computation of the point spread function.

Conclusions

We get compromised resolution and contrast for the polynomial apertures as compared with uniform circular apertures.

The Development of Microscopy for Super-Resolution: Confocal Microscopy, and Image Scanning Microscopy

Optical methods of super-resolution microscopy, such as confocal microscopy, structured illumination, nonlinear microscopy, and image scanning microscopy are reviewed. These methods avoid strong invasive interaction with a sample, allowing the observation of delicate biological samples. The meaning of resolution and the basic principles and different approaches to superresolution are discussed.

Pixel reassignment in image scanning microscopy with a doughnut beam: example of maximum likelihood restoration

In image scanning microscopy, the pinhole of a confocal microscope is replaced by a detector array. The point spread function for each detector element can be interpreted as the probability density function of the signal, the peak giving the most likely origin. This thus allows a form of maximum likelihood restoration, and compensation for aberrations, with similarities to adaptive optics. As an example of an aberration, we investigate theoretically and experimentally illumination with a vortex doughnut beam. After reassignment and summation over the detector array, the point spread function is compact, and the resolution and signal level higher than in a conventional microscope.

Structured illumination microscopy and image scanning microscopy: a review and comparison of imaging properties

Structured illumination microscopy and image scanning microscopy are two microscopical tech- niques, rapidly increasing in practical application, that can result in improvement in transverse spatial resolution, and/or improvement in axial imaging performance. The history and principles of these techniques are reviewed, and the imaging properties of the two methods compared. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 1)'.

Image scanning microscopy with multiphoton excitation or Bessel beam illumination

Image scanning microscopy is a technique of confocal microscopy in which the confocal pinhole is replaced by a detector array, and the image is reconstructed most straightforwardly by pixel reassignment. In the fluorescence mode, the detector array collects most of the fluorescent light, so the signal-to-noise ratio is much improved compared with confocal microscopy with a small pinhole, while the resolution is improved compared with conventional fluorescence microscopy. Here we consider two cases in which the illumination and detection point spread functions are dissimilar: illumination with a Bessel beam and multiphoton microscopy. It has been shown previously that for Bessel beam illumination in image scanning microscopy with a large array, the imaging performance is degraded. On the other hand, it is also known that the resolution of confocal microscopy is improved by Bessel beam illumination. Here we analyze image scanning microscopy with Bessel beam illumination together with a small array and show that an improvement in transverse resolution (width of the point spread function) by a factor of 1.78 compared with a conventional fluorescence microscope can be obtained. We also examine the behavior of image scanning microscopy in two- or three-photon fluorescence and for two-photon excitation also with Bessel beam illumination. The combination of the optical sectioning effect of image scanning microscopy with multiphoton microscopy reduces background from the sample surface, which can increase penetration depth. For a detector array size of two Airy units, the resolution of two-photon image scanning microscopy is a factor 1.85 better and the peak of the point spread function 2.84 times higher than in nonconfocal two-photon fluorescence. The resolution of three-photon image scanning microscopy is a factor 2.10 better, and the peak of the point spread function is 3.77 times higher than in nonconfocal three-photon fluorescence. The resolution of two-photon image scanning microscopy with Bessel beam illumination is a factor 2.13 better than in standard two-photon fluorescence. Axial resolution and optical sectioning in two-photon or three-photon fluorescence are also improved by using the image scanning modality.

Pixel reassignment in image scanning microscopy: a re-evaluation

Image scanning microscopy is a technique based on confocal microscopy, in which the confocal pinhole is replaced by a detector array, and the resulting image is reconstructed, usually by the process of pixel reassignment. The detector array collects most of the fluorescent light, so the signal-to-noise ratio is much improved compared with confocal microscopy with a small pinhole, while the resolution is improved compared with conventional (wide-field) microscopy. In previous studies, it has usually been assumed that pixels should be reassigned by a constant factor, to a point midway between the illumination and detection spots. Here it is shown that the peak intensity of the effective point spread function (PSF) can be further increased by 4% by a new choice of the pixel reassignment factor. For an array of two Airy units, the peak of the effective PSF is 1.90 times that of a conventional microscope, and the transverse resolution is 1.53 times better. It is confirmed that image scanning microscopy gives optical sectioning strength identical to that of a confocal microscope with a pinhole equal to the size of the detector array. However, it is shown that image scanning microscopy exhibits axial resolution superior to a confocal microscope with a pinhole the same size as the detector array. For a two-Airy-unit array, the axial resolution is 1.34 times better than in a conventional microscope for the standard reassignment factor, and 1.28 times better for the new reassignment factor. The axial resolution of a confocal microscope with a two-Airy-unit pinhole is only 1.04 times better than conventional microscopy. We also examine the signal-to-noise ratio of a point object in a uniform background (called the detectability), and show that it is 1.6 times higher than in a confocal microscope.

Characteristic and improvement on the reconstructed quality of effective perspective images' segmentation and mosaicking-based holographic stereogram

Based on the effective perspective images' segmentation and mosaicking (EPISM) printing method, the influence of the holographic element (hogel) size on the reconstructed quality is analyzed to improve the reconstructed quality of holographic stereogram. The flipping effect and diffraction effect in the spatial domain are also discussed, and the optical transfer function of the holographic stereogram is used in the spectrum domain to evaluate the reconstructed quality. Theoretical analysis and optical experimental results show that the hogel size plays an important role on the flipping effect as well as the clarity of the reconstrued 3D perspective images of the EPISM-based holographic stereogram, and the flipping effect can be improved significantly with optimized hogel size.

Transition between paraxial and Helmholtz Fourier spaces

Different correction methods for paraxial solutions have been used when such solutions extend out of the paraxial regime. Mapping functions play a fundamental role in them. This paper analyzes how to translate directions of paraxial wavevectors into Helmholtz wavevectors. Two possible mappings that are commonly used are examined, and a new mapping is proposed. The three mappings are classified in the framework of the full wave correction schemes.

Optimal illumination pattern for transport-of-intensity quantitative phase microscopy

The transport-of-intensity equation (TIE) is a well-established non-interferometric phase retrieval approach, which enables quantitative phase imaging (QPI) of transparent sample simply by measuring the intensities at multiple axially displaced planes. Nevertheless, it still suffers from two fundamentally limitations. First, it is quite susceptible to low-frequency errors (such as "cloudy" artifacts), which results from the poor contrast of the phase transfer function (PTF) near the zero frequency. Second, the reconstructed phase tends to blur under spatially low-coherent illumination, especially when the defocus distance is beyond the near Fresnel region. Recent studies have shown that the shape of the illumination aperture has a significant impact on the resolution and phase reconstruction quality, and by simply replacing the conventional circular illumination aperture with an annular one, these two limitations can be addressed, or at least significantly alleviated. However, the annular aperture was previously empirically designed based on intuitive criteria related to the shape of PTF, which does not guarantee optimality. In this work, we optimize the illumination pattern to maximize TIE's performance based on a combined quantitative criterion for evaluating the "goodness" of an aperture. In order to make the size of the solution search space tractable, we restrict our attention to binary-coded axis-symmetric illumination patterns only, which are easier to implement and can generate isotropic TIE PTFs. We test the obtained optimal illumination by imaging both a phase resolution target and HeLa cells based on a small-pitch LED array, suggesting superior performance over other suboptimal patterns in terms of both signal-to-noise ratio (SNR) and spatial resolution.

Wide-field anti-aliased quantitative differential phase contrast microscopy

Differential phase contrast (DPC) microscopy is a popular methodology to recover quantitative phase information of thin transparent samples under multi-axis asymmetric illumination patterns. Based on spatially partially coherent illuminations, DPC provides high-quality, speckle-free 3D reconstructions with lateral resolution up to twice the coherent diffraction limit, under the precondition that the pixel size of the imaging sensor is small enough to prevent spatial aliasing/undersampling. However, microscope cameras are in general designed to have a large pixel size so that the intensity information transmitted by the optical system cannot be adequately sampled or digitized. On the other hand, using an image sensor with a smaller pixel size or adding a magnification camera adapter to the camera can resolve the undersampling at the expense of a reduced field of view (FOV). To solve this tradeoff, we introduce a new variation of quantitative DPC approach, termed anti-aliased DPC (AADPC), which uses several aliased intensity images under asymmetric illuminations to recover wide-field aliasing-free phase images. Besides, phase transfer functions under different illumination patterns in DPC are analyzed to design an illumination scheme with better phase transfer characteristics. AADPC starts from an initial phase estimate obtained by a DPC-like deconvolution based on the system's weak phase transfer function under discrete half-annular illumination. Then the obtained initial phase map is further refined by the iterative de-multiplexing algorithm to overcome pixel-aliasing and improve the imaging resolution. The data redundancy requirement as well as the optimal illumination scheme of AADPC are analyzed and discussed based on several simulations, suggesting that the spatial undersampling can be mitigated through the iterative algorithm that uses only 4 images, yielding a nearly 4-fold increase in the space-bandwidth product (SBP) compared to the conventional DPC approach. We experimentally verify that AADPC can achieve a half-pitch imaging resolution of 345 nm, corresponding to 1.88× of the theoretical Nyquist-Shannon sampling resolution limit imposed by the sensor pixel size. The high-speed, high-throughput quantitative phase imaging capabilities of AADPC are also demonstrated by imaging HeLa cells mitosis in vitro, achieving a full-pitch lateral resolution of 665 nm across a wide FOV of 1.77mm² at 25 fps.

Partially coherent microscope in phase space

Explicit expressions are presented for different phase-space representations (mutual intensity, Wigner distribution function, and ambiguity function) of the partially coherent image wave field in a microscope system. These are separated into system- and object-dependent parts. The partially coherent image in phase space can be described in terms of different 6D system-dependent kernels, all Fourier transforms of the system mutual spectrum, the region of overlap of two displaced objective pupils and the effective source. The image intensity can be expressed in terms of a 4D kernel, the convolution in spatial frequency of the source, and the Wigner distribution function of the objective pupil, given by a marginal of, or a section through, the respective phase-space kernels.

Planar Diffractive Lenses: Fundamentals, Functionalities, and Applications

Traditional objective lenses in modern microscopy, based on the refraction of light, are restricted by the Rayleigh diffraction limit. The existing methods to overcome this limit can be categorized into near-field (e.g., scanning near-field optical microscopy, superlens, microsphere lens) and far-field (e.g., stimulated emission depletion microscopy, photoactivated localization microscopy, stochastic optical reconstruction microscopy) approaches. However, they either operate in the challenging near-field mode or there is the need to label samples in biology. Recently, through manipulation of the diffraction of light with binary masks or gradient metasurfaces, some miniaturized and planar lenses have been reported with intriguing functionalities such as ultrahigh numerical aperture, large depth of focus, and subdiffraction-limit focusing in far-field, which provides a viable solution for the label-free superresolution imaging. Here, the recent advances in planar diffractive lenses (PDLs) are reviewed from a united theoretical account on diffraction-based focusing optics, and the underlying physics of nanofocusing via constructive or destructive interference is revealed. Various approaches of realizing PDLs are introduced in terms of their unique performances and interpreted by using optical aberration theory. Furthermore, a detailed tutorial about applying these planar lenses in nanoimaging is provided, followed by an outlook regarding future development toward practical applications.

Characteristic and optimization of the effective perspective images' segmentation and mosaicking (EPISM) based holographic stereogram: an optical transfer function approach

Based on our proposed method for holographic stereogram printing using effective perspective images’ segmentation and mosaicking (EPISM), we analyze the reconstructed wavefront errors, and establish the exit pupil function model of proposed printing system. To evaluate the imaging quality, the optical transfer function (OTF) of the holographic stereogram is modelled from the aspect of frequency response. The characteristic of the OTF with respect to the exit pupil size and the aberration are investigated in detail. We also consider the flipping effect in spatial domain. The optimization of hogel sizes, i.e., the sampling interval of original perspective images and the printing interval of synthetic effective perspective images, are given for the optimized reconstruction. Numerical simulations and optical experiments are implemented, and the results demonstrate the validity of our analysis, and the optimized parameters of hogel sizes can improve the imaging quality of full parallax holographic stereogram effectively.

Airyscan super-resolution microscopy of mitochondrial morphology and dynamics in living tumor cells

Mitochondrial morphology is regulated by continuous fusion-and-fission events that are essential for maintaining normal function. Despite the prominence of mitochondrial function in energy generation and cell signaling, understanding of processes of fusion and fission dynamics has been hampered by the lack of high-resolution optical systems that accommodate live-cell imaging. We have examined different confocal modalities in terms of resolution and signal-to-noise ratio (SNR) in a point scanning confocal microscope with Airyscan super-resolution (AS-SR). Results indicated that Airyscan (AS) provided speed, super-resolution, and high SNR. This modality was then used for monitoring mitochondrial dynamics in live tumor cells modified to harbor green-fluorescent protein localized to mitochondria. We then compared regular AS and fast-Airyscan modalities in terms of gentleness on the live-cell samples. The fast mode provided unprecedented imaging speed that permits monitoring dynamics both in 2D and also in three-dimensional dataset with time lapses (4D). Alterations to the mitochondrial network in U87 glioblastoma cells occurred within seconds and the cells were not affected by modest inhibition of fission. The super-resolution permitted quantitative measurements of mitochondrial diameter with a precision that enabled detection of significant differences in mitochondrial morphology between cell lines. We have observed swelling of mitochondrial tubules in A549 lung cancer cells after 2 hr treatment with deoxynyboquinone, an ROS-generating pharmacologic drug. We also tested different 3D analytical parameters and how they can affect morphometric quantitation. The AS-SR imaging enabled high-speed imaging of mitochondrial dynamics without the compromise to cell morphology or viability that is common with conventional fluorescence imaging due to photo-oxidation.

Image formation in image scanning microscopy, including the case of two-photon excitation

The effect of combining the image scanning microscopy (ISM) technique with two-photon fluorescence microscopy is analyzed. The effective spatial frequency cutoff can be doubled, as compared with conventional two-photon fluorescence microscopy, and the magnitude of the optical transfer function near the cutoff of conventional two-photon microscopy is increased by orders of magnitude. For the two-photon case, it is found that the optimum pixel reassignment factor in ISM is not equal to one half, as is often assumed in single-photon fluoresence image scanning microscopy, because the excitation and detection point spread functions are different. The optimum reassignment factor depends on the noise level, and in general the useful cutoff spatial frequency is about 1.8 times that for conventional two-photon microscopy. The effect of altering the reassignment factor in single-photon fluorescence ISM with a Stokes shift is also investigated. Illumination using pupil filters, such as by a Bessel beam, is considered. Using a ring detector array is found to result in good imaging behavior, exhibiting a sharpening of the point spread function by a factor of 1.7 compared with conventional fluorescence. Image formation in ISM can be considered in a four-dimensional spatial frequency space, giving new insight into the imaging properties. This approach is related to phase space representations such as the Wigner distribution function and the ambiguity function. A noniterative algorithm for image restoration is proposed.