Pixel reassignment in image scanning microscopy: a re-evaluation

Three-dimensional multifocal scanning microscopy for super-resolution cell and tissue imaging

Recent advancements in image-scanning microscopy have significantly enriched super-resolution biological research, providing deeper insights into cellular structures and processes. However, current image-scanning techniques often require complex instrumentation and alignment, constraining their broader applicability in cell biological discovery and convenient, cost-effective integration into commonly used frameworks like epi-fluorescence microscopes. Here, we introduce three-dimensional multifocal scanning microscopy (3D-MSM) for super-resolution imaging of cells and tissue with substantially reduced instrumental complexity. This method harnesses the inherent 3D movement of specimens to achieve stationary, multi-focal excitation and super-resolution microscopy through a standard epi-fluorescence platform. We validated the system using a range of phantom, single-cell, and tissue specimens. The combined strengths of structured illumination, confocal detection, and epi-fluorescence setup result in two-fold resolution improvement in all three dimensions, effective optical sectioning, scalable volume acquisition, and compatibility with general imaging and sample protocols. We anticipate that 3D-MSM will pave a promising path for future super-resolution investigations in cell and tissue biology.

Image scanning lensless fiber-bundle endomicroscopy

Fiber-based confocal endomicroscopy has shown great promise for minimally-invasive deep-tissue imaging. Despite its advantages, confocal fiber-bundle endoscopy inherently suffers from undersampling due to the spacing between fiber cores, and low collection efficiency when the target is not in proximity to the distal fiber facet. Here, we demonstrate an adaptation of image-scanning microscopy (ISM) to lensless fiber bundle endoscopy, doubling the spatial sampling frequency and significantly improving collection efficiency. Our approach only requires replacing the confocal detector with a camera. It improves the spatial resolution for targets placed at a distance from the fiber tip, and addresses the fundamental challenge of aliasing/pixelization artifacts.

Simultaneous Multicolor Multifocal Scanning Microscopy

Super-resolution fluorescence microscopy has revolutionized cell biology over the past decade, enabling the visualization of subcellular complexity with unparalleled clarity and detail. However, the rapid development of image-scanning-based super-resolution systems still restrains convenient access to commonly used instruments such as epi-fluorescence microscopes. Here, we present multifocal scanning microscopy (MSM) for super-resolution imaging with simultaneous multicolor acquisition and minimal instrumental complexity. MSM implements a stationary, interposed multifocal multicolor excitation by exploiting the motion of the specimens, realizing super-resolution microscopy through a general epi-fluorescence platform without compromising the image-scanning mechanism or inducing complex instrument alignment. The system is demonstrated with various phantom and biological specimens, and the results present effective resolution doubling, optical sectioning, and contrast enhancement. We anticipate MSM, as a highly accessible and compatible super-resolution technique, to offer a promising methodological pathway for broad cell biological discoveries.

Image scanning microscopy with a doughnut beam: signal strength and integrated intensity

We discuss the effects of image scanning microscopy using doughnut beam illumination on the properties of signal strength and integrated intensity. Doughnut beam illumination can give better optical sectioning and background rejection than Airy disk illumination. The outer pixels of a detector array give a signal from defocused regions, so digital processing of these (e.g., by simple subtraction) can further improve optical sectioning and background rejection from a single in-focus scan.

Signal strength and integrated intensity in confocal and image scanning microscopy

The properties of signal strength and integrated intensity in a scanned imaging system are reviewed. These properties are especially applied to confocal imaging systems, including image scanning microscopy. The integrated intensity, equal to the image of a uniform planar (sheet) object, rather than the peak of the point spread function, is a measure of the flux in an image. Analytic expressions are presented for the intensity in the detector plane for a uniform volume object, and for the resulting background. The variation in the integrated intensity with defocus for an offset point detector is presented. This axial fingerprint is independent of any pixel reassignment. The intensity in the detector plane is shown to contain the defocus information, and simple processing of the recorded data can improve optical sectioning and background rejection.

Saturated-excitation image scanning microscopy

Image scanning microscopy (ISM) overcomes the trade-off between spatial resolution and signal volume in confocal microscopy by rearranging the signal distribution on a two-dimensional detector array to achieve a spatial resolution close to the theoretical limit achievable by infinitesimal pinhole detection without sacrificing the detected signal intensity. In this paper, we improved the spatial resolution of ISM in three dimensions by exploiting saturated excitation (SAX) of fluorescence. We theoretically investigated the imaging properties of ISM, when the fluorescence signals are nonlinearly induced by SAX, and show combined SAX-ISM fluorescence imaging to demonstrate the improvement of the spatial resolution in three dimensions. In addition, we confirmed that the SNR of SAX-ISM imaging of fluorescent beads and biological samples, which is one of the challenges in conventional SAX microscopy, was improved.

ISM-assisted tomographic STED microscopy

Stimulated emission depletion (STED) microscopy theoretically provides unlimited resolution. However, in practice the achievable resolution in biological samples is essentially limited by photobleaching. One method which overcomes this problem is tomographic STED (tomoSTED) microscopy. In tomoSTED microscopy, one-dimensional depletion patterns facing in different directions are successively applied in order to acquire a highly-resolved image in two dimensions. In this context, the number of addressed directions depends on the desired angular homogeneity of the point spread function or the optical transfer function and thus on the resolution increase as compared to diffraction-limited imaging. At a reasonable angular homogeneity the light dose and thus bleaching can be reduced, as compared to conventional STED microscopy. Here, we propose and demonstrate for the first time, to our knowledge, that the number of required depletion pattern orientations can be reduced by combining tomoSTED microscopy with the concept of image scanning microscopy (ISM). With our realization of an ISM-tomoSTED microscope, we show that approximately a factor of 2 lower number of orientations are required to achieve the same resolution and image quality as in tomoSTED microscopy.

The Development of Microscopy for Super-Resolution: Confocal Microscopy, and Image Scanning Microscopy

Optical methods of super-resolution microscopy, such as confocal microscopy, structured illumination, nonlinear microscopy, and image scanning microscopy are reviewed. These methods avoid strong invasive interaction with a sample, allowing the observation of delicate biological samples. The meaning of resolution and the basic principles and different approaches to superresolution are discussed.

Pixel reassignment in image scanning microscopy with a doughnut beam: example of maximum likelihood restoration

In image scanning microscopy, the pinhole of a confocal microscope is replaced by a detector array. The point spread function for each detector element can be interpreted as the probability density function of the signal, the peak giving the most likely origin. This thus allows a form of maximum likelihood restoration, and compensation for aberrations, with similarities to adaptive optics. As an example of an aberration, we investigate theoretically and experimentally illumination with a vortex doughnut beam. After reassignment and summation over the detector array, the point spread function is compact, and the resolution and signal level higher than in a conventional microscope.

Structured illumination microscopy and image scanning microscopy: a review and comparison of imaging properties

Structured illumination microscopy and image scanning microscopy are two microscopical tech- niques, rapidly increasing in practical application, that can result in improvement in transverse spatial resolution, and/or improvement in axial imaging performance. The history and principles of these techniques are reviewed, and the imaging properties of the two methods compared. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 1)'.

cSPARCOM: Multi-detector reconstruction by confocal super-resolution correlation microscopy

Image scanning microscopy (ISM), an upgraded successor of the ubiquitous confocal microscope, facilitates up to two-fold improvement in lateral resolution, and has become an indispensable element in the toolbox of the bio-imaging community. Recently, super-resolution optical fluctuation image scanning microscopy (SOFISM) integrated the analysis of intensity-fluctuations information into the basic ISM architecture, to enhance its resolving power. Both of these techniques typically rely on pixel-reassignment as a fundamental processing step, in which the parallax of different detector elements to the sample is compensated by laterally shifting the point spread function (PSF). Here, we propose an alternative analysis approach, based on the recent high-performing sparsity-based super-resolution correlation microscopy (SPARCOM) method. Through measurements of DNA origami nano-rulers and fixed cells labeled with organic dye, we experimentally show that confocal SPARCOM (cSPARCOM), which circumvents pixel-reassignment altogether, provides enhanced resolution compared to pixel-reassigned based analysis. Thus, cSPARCOM further promotes the effectiveness of ISM, and particularly that of correlation based ISM implementations such as SOFISM, where the PSF deviates significantly from spatial invariance.

Three-Dimension Resolution Enhanced Microscopy Based on Parallel Detection

Pixel reassignment image scanning microscopy (PRISM) is a useful tool to improve the resolution of confocal laser scanning microscopy (CLSM) only equipped with a detector array. However, while it can improve the lateral resolution, it has little effect on the axial resolution. Here, new microscopy has been proposed which combines three-dimension fluorescence emission difference microscopy (3D FED) with PRISM to further improve three-dimension resolution. We call this method three-dimension pixel reassignment fluorescence emission difference microscopy (3D-PRFED). Detailed theoretical analysis and simulation are presented in this paper. Additionally, the performance of lateral and axial resolution improvement of this method has been demonstrated by imaging 100 nm fluorescent beads and nuclear pore complexes samples. Experiment results show that this method in our system can improve lateral resolution by a factor of 1.85 and axial resolution by a factor of 1.48 compared with CLSM.

Image scanning microscopy with a long depth of focus generated by an annular radially polarized beam

Image scanning microscopy (ISM) is a promising tool for bioimaging owing to its integration of signal to noise ratio (SNR) and super resolution superior to that obtained in confocal scanning microscopy. In this paper, we introduce the annular radially polarized beam to the ISM, which yields an axially extended excitation focus and enhanced resolution, providing a new possibility to obtain the whole information of thick specimen with a single scan. We present the basic principle and a rigorous theoretical model for ISM with annular radially polarized beam (ISM-aRP). Results show that the resolution of ISM-aRP can be enhanced by 4% compared with that in conventional ISM, and the axial extent of the focus is longer than 6λ. The projected view of the simulated fluorescent beads suspension specimen demonstrates the validity of ISM-aRP to obtain the whole information of volume sample. Moreover, this simple method can be easily integrated into the commercial laser scanning microscopy systems.

Image scanning microscopy with multiphoton excitation or Bessel beam illumination

Image scanning microscopy is a technique of confocal microscopy in which the confocal pinhole is replaced by a detector array, and the image is reconstructed most straightforwardly by pixel reassignment. In the fluorescence mode, the detector array collects most of the fluorescent light, so the signal-to-noise ratio is much improved compared with confocal microscopy with a small pinhole, while the resolution is improved compared with conventional fluorescence microscopy. Here we consider two cases in which the illumination and detection point spread functions are dissimilar: illumination with a Bessel beam and multiphoton microscopy. It has been shown previously that for Bessel beam illumination in image scanning microscopy with a large array, the imaging performance is degraded. On the other hand, it is also known that the resolution of confocal microscopy is improved by Bessel beam illumination. Here we analyze image scanning microscopy with Bessel beam illumination together with a small array and show that an improvement in transverse resolution (width of the point spread function) by a factor of 1.78 compared with a conventional fluorescence microscope can be obtained. We also examine the behavior of image scanning microscopy in two- or three-photon fluorescence and for two-photon excitation also with Bessel beam illumination. The combination of the optical sectioning effect of image scanning microscopy with multiphoton microscopy reduces background from the sample surface, which can increase penetration depth. For a detector array size of two Airy units, the resolution of two-photon image scanning microscopy is a factor 1.85 better and the peak of the point spread function 2.84 times higher than in nonconfocal two-photon fluorescence. The resolution of three-photon image scanning microscopy is a factor 2.10 better, and the peak of the point spread function is 3.77 times higher than in nonconfocal three-photon fluorescence. The resolution of two-photon image scanning microscopy with Bessel beam illumination is a factor 2.13 better than in standard two-photon fluorescence. Axial resolution and optical sectioning in two-photon or three-photon fluorescence are also improved by using the image scanning modality.

Second harmonic generation microscopy using pixel reassignment

Second harmonic generation (SHG) microscopy is expected to be a powerful tool for observing the cellular-level functionality and morphology information of thick tissue owe to its unique imaging properties. However, the maximum attainable resolution obtainable by SHG microscopy is limited by the use of long-wavelength, near-infrared excitation. In this paper, we report the use of pixel reassignment to improve the spatial resolution of SHG microscopy. The SHG signal is imaged onto a position-sensitive camera, instead of a point detector typically used in conventional SHG microscope. The data processing is performed through pixel reassignment and subsequent deblurring operation. We present the basic principle and a rigorous theoretical model for SHG microscopy using pixel reassignment (SHG-PR). And for the first time, the optimal reassignment factor for SHG-PR is derived based on the coherent characteristics and the dependence of wavelength in SHG microscopy. To evaluate the spatial resolution improvement, images of nano-beads separated by different distances and of a microtubule array have been simulated. We gain about a 1.5-fold spatial resolution enhancement compared to conventional SHG microscopy. When a further deblurring operation is implemented, this method allows for a total spatial resolution enhancement of about 1.87. Additionally, we demonstrate the validity of SHG-PR for raw data with noise. LAY DESCRIPTION: Second harmonic generation (SHG) microscopy has emerged as a powerful imaging technique in clinical diagnostics and biological research. SHG microscopy is label-free and provides intrinsic optical sectioning for three-dimensional (3D) imaging. However, a near-infrared excitation wavelength results a restriction in the maximum attainable spatial resolution of SHG microscopy. In this paper, we present a simple resolution-enhanced SHG imaging method, SHG microscopy using pixel reassignment (SHG-PR). We demonstrate a rigorous theoretical model for SHG-PR and derive the optimal reassignment factor. The simulation result shows the clear improvement of the image resolution and contrast in the SHG-PR after deblurring operation. The FWHM value of single microtubule shows that SHG-PR enables a spatial resolution enhancement by a factor of 1.5, compared to conventional SHG microscopy. After a proper deblurring operation, this method allows for a total spatial resolution enhancement of about 1.87. The improvements of spatial resolution and contrast are still valid for raw data with noise. It is expected that this method can contribute towards new insights in unstained tissue morphology, interaction of cells, and diseases diagnosis.

Two-photon image-scanning microscopy with SPAD array and blind image reconstruction

Two-photon excitation (2PE) laser scanning microscopy is the imaging modality of choice when one desires to work with thick biological samples. However, its spatial resolution is poor, below confocal laser scanning microscopy. Here, we propose a straightforward implementation of 2PE image scanning microscopy (2PE-ISM) that, by leveraging our recently introduced single-photon avalanche diode (SPAD) array detector and a novel blind image reconstruction method, is shown to enhance the effective resolution, as well as the overall image quality of 2PE microscopy. With our adaptive pixel reassignment procedure ∼1.6 times resolution increase is maintained deep into thick semi-transparent samples. The integration of Fourier ring correlation based semi-blind deconvolution is shown to further enhance the effective resolution by a factor of ∼2 - and automatic background correction is shown to boost the image quality especially in noisy images. Most importantly, our 2PE-ISM implementation requires no calibration measurements or other input from the user, which is an important aspect in terms of day-to-day usability of the technique.

Two-photon image-scanning microscopy with SPAD array and blind image reconstruction

Two-photon excitation (2PE) laser scanning microscopy is the imaging modality of choice when one desires to work with thick biological samples. However, its spatial resolution is poor, below confocal laser scanning microscopy. Here, we propose a straightforward implementation of 2PE image scanning microscopy (2PE-ISM) that, by leveraging our recently introduced single-photon avalanche diode (SPAD) array detector and a novel blind image reconstruction method, is shown to enhance the effective resolution, as well as the overall image quality of 2PE microscopy. With our adaptive pixel reassignment procedure ∼1.6 times resolution increase is maintained deep into thick semi-transparent samples. The integration of Fourier ring correlation based semi-blind deconvolution is shown to further enhance the effective resolution by a factor of ∼2 - and automatic background correction is shown to boost the image quality especially in noisy images. Most importantly, our 2PE-ISM implementation requires no calibration measurements or other input from the user, which is an important aspect in terms of day-to-day usability of the technique.