Multiplane imaging and three dimensional nanoscale particle tracking in biological microscopy

Enhancing axial localization with wavefront control

Enhancing the ability to resolve axial details is crucial in three-dimensional optical imaging. We provide experimental evidence showcasing the ultimate precision achievable in axial localization using vortex beams. For Laguerre-Gauss (LG) beams, this remarkable limit can be attained with just a single intensity scan. This proof-of-principle demonstrates that microscopy techniques based on LG vortex beams can potentially benefit from the introduced quantum-inspired superresolution protocol.

Design and Study of a Reflector-Separated Light Dispersion-Compensated 3D Microscopy System

The secondary-phase grating-based tomographic microscopy system, which is widely used in the biological and life sciences, can observe all the sample multilayer image information simultaneously because it has multifocal points. However, chromatic aberration exists in the grating diffraction, which seriously affects the observation of the image. To correct the chromatic aberration of the tomographic microscope system, this paper proposes a system that adopts blazed gratings and angle-variable reflectors as chromatic aberration correction devices according to the principle of dispersion compensation and Fourier phase-shift theory. A reflector-separated light dispersion-compensated 3D microscopy system is presented to achieve chromatic aberration correction while solving the problem of multilayer image overlap. The theoretical verification and optical design of the system were completed using ZEMAX software. The results show that the proposed system reduced the chromatic aberration of ordinary tomographic microscopy systems by more than 90%, retaining more wavelengths of light information. In addition, the system had a relatively wide range in the color difference compensation element installation position, reducing the difficulty of dispersion compensation element installation. Overall, the results indicate that the proposed system is effective in reducing chromatic aberration in grating diffraction.

OptoRheo: Simultaneous in situ micro-mechanical sensing and imaging of live 3D biological systems

Biomechanical cues from the extracellular matrix (ECM) are essential for directing many cellular processes, from normal development and repair, to disease progression. To better understand cell-matrix interactions, we have developed a new instrument named 'OptoRheo' that combines light sheet fluorescence microscopy with particle tracking microrheology. OptoRheo lets us image cells in 3D as they proliferate over several days while simultaneously sensing the mechanical properties of the surrounding extracellular and pericellular matrix at a sub-cellular length scale. OptoRheo can be used in two operational modalities (with and without an optical trap) to extend the dynamic range of microrheology measurements. We corroborated this by characterising the ECM surrounding live breast cancer cells in two distinct culture systems, cell clusters in 3D hydrogels and spheroids in suspension culture. This cutting-edge instrument will transform the exploration of drug transport through complex cell culture matrices and optimise the design of the next-generation of disease models.

Three-dimensional single particle tracking using 4π self-interference of temporally phase-shifted fluorescence

Single particle tracking in three dimensions is an indispensable tool for studying dynamic processes in various disciplines, including material sciences, physics, and biology, but often shows anisotropic three-dimensional spatial localization precision, which restricts the tracking precision, and/or a limited number of particles that can be tracked simultaneously over extended volumes. Here we developed an interferometric, three-dimensional fluorescence single particle tracking method based on conventional widefield excitation and temporal phase-shift interference of the emitted, high-aperture-angle, fluorescence wavefronts in a greatly simplified, free-running, triangle interferometer that enables tracking of multiple particles at the same time with <10-nm spatial localization precision in all three dimensions over extended volumes (~35 × 35 × 2 μm³) at video rate (25 Hz). We applied our method to characterize the microenvironment of living cells and up to ~40 μm deep in soft materials.

Expanded field of view frequency-selective incoherent holography by using a triple-beam setup

We propose a new, to the best of our knowledge, method of incoherent optical frequency selection called three-pack frequency-selective incoherent holography. Compressed holography is reconstructed using phase shift intercepts and spatial transfer function convolution in the form of separation without loss of magnification or resolution. The frequency-selective reconstruction process removes the conjugate and DC terms along with the interception of the object wave. This work attempts three-dimensional reconstruction and selected-frequency phase extraction of axial slices in submicron steps, and the experimental results show the potential of the proposed method in areas such as compressed holography, extended field of view, and slice tomography.

Instantaneous 4D micro-particle image velocimetry (µPIV) via multifocal microscopy (MUM)

Multifocal microscopy (MUM), a technique to capture multiple fields of view (FOVs) from distinct axial planes simultaneously and on one camera, was used to perform micro-particle image velocimetry (µPIV) to reconstruct velocity and shear stress fields imposed by a liquid flowing around a cell. A diffraction based multifocal relay was used to capture images from three different planes with 630 nm axial spacing from which the axial positions of the flow-tracing particles were calculated using the image sharpness metric. It was shown that MUM can achieve an accuracy on the calculated velocity of around (0.52 ± 0.19) µm/s. Using fixed cells, MUM imaged the flow perturbations at sub-cellular level, which showed characteristics similar to those observed in the literature. Using live cells as an exemplar, MUM observed the effect of changing cell morphology on the local flow during perfusion. Compared to standard confocal laser scanning microscope, MUM offers a clear advantage in acquisition speed for µPIV (over 300 times faster). This is an important characteristic for rapidly evolving biological systems where there is the necessity to monitor in real time entire volumes to correlate the sample responses to the external forces.

Multicolor multifocal 3D microscopy using in-situ optimization of a spatial light modulator

Multifocal microscopy enables high-speed three-dimensional (3D) volume imaging by using a multifocal grating in the emission path. This grating is typically designed to afford a uniform illumination of multifocal subimages for a single emission wavelength. Using the same grating for multicolor imaging results in non-uniform subimage intensities in emission wavelengths for which the grating is not designed. This has restricted multifocal microscopy applications for samples having multicolored fluorophores. In this paper, we present a multicolor multifocal microscope implementation which uses a Spatial Light Modulator (SLM) as a single multifocal grating to realize near-uniform multifocal subimage intensities across multiple wavelength emission bands. Using real-time control of an in-situ-optimized SLM implemented as a multifocal grating, we demonstrate multicolor multifocal 3D imaging over three emission bands by imaging multicolored particles as well as Escherichia coli (E. coli) interacting with human liver cancer cells, at [Formula: see text] multicolor 3D volumes per second acquisition speed. Our multicolor multifocal method is adaptable across SLM hardware, emission wavelength band locations and number of emission bands, making it particularly suited for researchers investigating fast processes occurring across a volume where multiple species are involved.

Snapshot volumetric imaging with engineered point-spread functions

The biological world involves intracellular and intercellular interactions that occur at high speed, at multiple scales and in three dimensions. Acquiring 3D images, however, typically requires a compromise in either spatial or temporal resolution compared to 2D imaging. Conventional 2D fluorescence imaging provides high spatial resolution but requires plane-by-plane imaging of volumes. Conversely, snapshot methods such as light-field microscopy allow video-rate imaging, but at the cost of spatial resolution. Here we introduce 3D engineered point-spread function microscopy (3D-EPM), enabling snapshot imaging of real-world 3D extended biological structures while retaining the native resolution of the microscope in space and time. Our new computational recovery strategy is the key to volumetrically reconstructing arbitrary 3D structures from the information encapsulated in 2D raw EPM images. We validate our technique on both point-like and extended samples, and demonstrate its power by imaging the intracellular motion of chloroplasts undergoing cyclosis in a sample of Egeria densa. Our technique represents a generalised computational methodology for 3D image recovery which is readily adapted to a diverse range of existing microscopy platforms and engineered point-spread functions. We therefore expect it to find broad applicability in the study of rapid biological dynamics in 3D.

Study of the Off-Axis Fresnel Zone Plate of a Microscopic Tomographic Aberration

A tomographic microscopy system can achieve instantaneous three-dimensional imaging, and this type of microscopy system has been widely used in the study of biological samples; however, existing chromatographic microscopes based on off-axis Fresnel zone plates have degraded image quality due to geometric aberrations such as spherical aberration, coma aberration, and image scattering. This issue hinders the further development of chromatographic microscopy systems. In this paper, we propose a method for the design of an off-axis Fresnel zone plate with the elimination of aberrations based on double exposure point holographic surface interference. The aberration coefficient model of the optical path function was used to solve the optimal recording parameters, and the principle of the aberration elimination tomography microscopic optical path was verified. The simulation and experimental verification were carried out utilizing a Seidel coefficient, average gradient, and signal-to-noise ratio. First, the aberration coefficient model of the optical path function was used to solve the optimal recording parameters. Then, the laminar mi-coroscopy optical system was constructed for the verification of the principle. Finally, the simulation calculation results and the experimental results were verified by comparing the Seidel coefficient, average gradient, and signal-to-noise ratio of the microscopic optical system before and after the aberration elimination. The results show that for the diffractive light at the orders 0 and ±1, the spherical aberration W040 decreases by 62–70%, the coma aberration W131 decreases by 96–98%, the image dispersion W222 decreases by 71–82%, and the field curvature W220 decreases by 96–96%, the average gradient increases by 2.8%, and the signal-to-noise ratio increases by 18%.

In-vivo Single-Molecule Imaging in Yeast: Applications and Challenges

Single-molecule imaging has gained momentum to quantify the dynamics of biomolecules in live cells, as it provides direct real-time measurements of various cellular activities under their physiological environment. Yeast, a simple and widely used eukaryote, serves as a good model system to quantify single-molecule dynamics of various cellular processes because of its low genomic and cellular complexities, as well as its facile ability to be genetically manipulated. In the past decade, significant developments have been made regarding the intracellular labeling of biomolecules (proteins, mRNA, fatty acids), the microscopy setups to visualize single-molecules and capture their fast dynamics, and the data analysis pipelines to interpret such dynamics. In this review, we summarize the current state of knowledge for the single-molecule imaging in live yeast cells to provide a ready reference for beginners. We provide a comprehensive table to demonstrate how various labs tailored the imaging regimes and data analysis pipelines to estimate various biophysical parameters for a variety of biological processes. Lastly, we present current challenges and future directions for developing better tools and resources for single-molecule imaging in live yeast cells.

Multifocal imaging for precise, label-free tracking of fast biological processes in 3D

Many biological processes happen on a nano- to millimeter scale and within milliseconds. Established methods such as confocal microscopy are suitable for precise 3D recordings but lack the temporal or spatial resolution to resolve fast 3D processes and require labeled samples. Multifocal imaging (MFI) allows high-speed 3D imaging but is limited by the compromise between high spatial resolution and large field-of-view (FOV), and the requirement for bright fluorescent labels. Here, we provide an open-source 3D reconstruction algorithm for multi-focal images that allows using MFI for fast, precise, label-free tracking spherical and filamentous structures in a large FOV and across a high depth. We characterize fluid flow and flagellar beating of human and sea urchin sperm with a z-precision of 0.15 µm, in a volume of 240 × 260 × 21 µm, and at high speed (500 Hz). The sampling volume allowed to follow sperm trajectories while simultaneously recording their flagellar beat. Our MFI concept is cost-effective, can be easily implemented, and does not rely on object labeling, which renders it broadly applicable.

Multiplexed PSF Engineering for Three-Dimensional Multicolor Particle Tracking

Three-dimensional spatiotemporal tracking of microscopic particles in multiple colors is a challenging optical imaging task. Existing approaches require a trade-off between photon efficiency, field of view, mechanical complexity, spectral specificity, and speed. Here, we introduce multiplexed point-spread-function engineering that achieves photon-efficient, 3D multicolor particle tracking over a large field of view. This is accomplished by first chromatically splitting the emission path of a microscope to different channels, engineering the point-spread function of each, and then recombining them onto the same region of the camera. We demonstrate our technique for simultaneously tracking five types of emitters in vitro as well as colocalization of DNA loci in live yeast cells.

Optical Tweezers with Integrated Multiplane Microscopy (OpTIMuM): a new tool for 3D microrheology

We introduce a novel 3D microrheology system that combines for the first time Optical Tweezers with Integrated Multiplane Microscopy (OpTIMuM). The system allows the 3D tracking of an optically trapped bead, with ~ 20 nm accuracy along the optical axis. This is achieved without the need for a high precision z-stage, separate calibration sample, nor a priori knowledge of either the bead size or the optical properties of the suspending medium. Instead, we have developed a simple yet effective in situ spatial calibration method using image sharpness and exploiting the fact we image at multiple planes simultaneously. These features make OpTIMuM an ideal system for microrheology measurements, and we corroborate the effectiveness of this novel microrheology tool by measuring the viscosity of water in three dimensions, simultaneously.

Intensity-Based Axial Localization at the Quantum Limit

We derive fundamental precision bounds for single-point axial localization. For Gaussian beams, this ultimate limit can be achieved with a single intensity scan, provided the camera is placed at one of two optimal transverse detection planes. Hence, for axial localization there is no need of more complicated detection schemes. The theory is verified with an experimental demonstration of axial resolution 3 orders of magnitude below the classical depth of focus.

Defocus-based three-dimensional particle location with extended depth of field via color coding

A color-coded lens system is designed and built for three-dimensional (3D) particle location. As it has wavelength-dependent focal length, it can obtain a series of multifocus images in a single snapshot by separating the red, green, and blue channels of the image captured by a color camera. A light source consisting of red, green, and blue LEDs is customized to provide narrowband illumination. In this way, the depth of field can be extended. A three-pinhole aperture is introduced into the system to produce defocus-based triangular patterns that help to encode the particle depth. The accurate 3D particle position can finally be obtained by fusing the information from the three color channels. Experimental results demonstrate that the system is well suited for applications where particles may be irregular or extended depth of field is needed.

Lattice-Resonance Metalenses for Fully Reconfigurable Imaging

This paper describes a reconfigurable metalens system that can image at visible wavelengths based on arrays of coupled plasmonic nanoparticles. These lenses manipulated the wavefront and focused light by exciting surface lattice resonances that were tuned by patterned polymer blocks on single-particle sites. Predictive design of the dielectric nanoblocks was performed using an evolutionary algorithm to create a range of three-dimensional focusing responses. For scalability, we demonstrated a simple technique for erasing and writing the polymer nanostructures on the metal nanoparticle arrays in a single step using solvent-assisted nanoscale embossing. This reconfigurable materials platform enables tunable focusing with diffraction-limited resolution and offers prospects for highly adaptive, compact imaging.

Video-rate large-scale imaging with Multi-Z confocal microscopy

Fast, volumetric imaging over large scales has been a long-standing challenge in biological microscopy. To address this challenge, we report an augmented variant of confocal microscopy that uses a series of reflecting pinholes axially distributed in the detection space, such that each pinhole probes a different depth within the sample. We thus obtain simultaneous multiplane imaging without the need for axial scanning. Our microscope technique is versatile and configured here to provide two-color fluorescence imaging with a field of view larger than a millimeter at video rate. Its general applicability is demonstrated with neuronal imaging of both Caenorhabditis elegans and mouse brains in vivo.

All-passive transformable optical mapping near-eye display

We present an all-passive, transformable optical mapping (ATOM) near-eye display based on the “human-centric design” principle. By employing a diffractive optical element, a distorted grating, the ATOM display can project different portions of a two-dimensional display screen to various depths, rendering a real three-dimensional image with correct focus cues. Thanks to its all-passive optical mapping architecture, the ATOM display features a reduced form factor and low power consumption. Moreover, the system can readily switch between a real-three-dimensional and a high-resolution two-dimensional display mode, providing task-tailored viewing experience for a variety of VR/AR applications.

Revealing Nanoscale Morphology of the Primary Cilium Using Super-Resolution Fluorescence Microscopy

Super-resolution (SR) microscopy has been used to observe structural details beyond the diffraction limit of ∼250 nm in a variety of biological and materials systems. By combining this imaging technique with both computer-vision algorithms and topological methods, we reveal and quantify the nanoscale morphology of the primary cilium, a tiny tubular cellular structure (∼2-6 μm long and 200-300 nm in diameter). The cilium in mammalian cells protrudes out of the plasma membrane and is important in many signaling processes related to cellular differentiation and disease. After tagging individual ciliary transmembrane proteins, specifically Smoothened, with single fluorescent labels in fixed cells, we use three-dimensional (3D) single-molecule SR microscopy to determine their positions with a precision of 10-25 nm. We gain a dense, pointillistic reconstruction of the surfaces of many cilia, revealing large heterogeneity in membrane shape. A Poisson surface reconstruction algorithm generates a fine surface mesh, allowing us to characterize the presence of deformations by quantifying the surface curvature. Upon impairment of intracellular cargo transport machinery by genetic knockout or small-molecule treatment of cells, our quantitative curvature analysis shows significant morphological differences not visible by conventional fluorescence microscopy techniques. Furthermore, using a complementary SR technique, two-color, two-dimensional stimulated emission depletion microscopy, we find that the cytoskeleton in the cilium, the axoneme, also exhibits abnormal morphology in the mutant cells, similar to our 3D results on the Smoothened-measured ciliary surface. Our work combines 3D SR microscopy and computational tools to quantitatively characterize morphological changes of the primary cilium under different treatments and uses stimulated emission depletion to discover correlated changes in the underlying structure. This approach can be useful for studying other biological or nanoscale structures of interest.

Design and simulation of a snapshot multi-focal interferometric microscope

Realizing both high temporal and spatial resolution across a large volume is a key challenge for 3D fluorescent imaging. Towards achieving this objective, we introduce an interferometric multifocus microscopy (iMFM) system, a combination of multifocus microscopy (MFM) with two opposing objective lenses. We show that the proposed iMFM is capable of simultaneously producing multiple focal plane interferometry that provides axial super-resolution and hence isotropic 3D resolution with a single exposure. We design and simulate the iMFM microscope by employing two special diffractive optical elements. The point spread function of this new iMFM microscope is simulated and the image formation model is given. For reconstruction, we use the Richardson-Lucy deconvolution algorithm with total variation regularization for 3D extended object recovery, and a maximum likelihood estimator (MLE) for single molecule tracking. A method for determining an initial axial position of the molecule is also proposed to improve the convergence of the MLE. We demonstrate both theoretically and numerically that isotropic 3D nanoscopic localization accuracy is achievable with an axial imaging range of 2um when tracking a fluorescent molecule in three dimensions and that the diffraction limited axial resolution can be improved by 3-4 times in the single shot wide-field 3D extended object recovery. We believe that iMFM will be a useful tool in 3D dynamic event imaging that requires both high temporal and spatial resolution.

Model-based image analysis of a tethered Brownian fibre for shear stress sensing

The measurement of fluid dynamic shear stress acting on a biologically relevant surface is a challenging problem, particularly in the complex environment of, for example, the vasculature. While an experimental method for the direct detection of wall shear stress via the imaging of a synthetic biology nanorod has recently been developed, the data interpretation so far has been limited to phenomenological random walk modelling, small-angle approximation, and image analysis techniques which do not take into account the production of an image from a three-dimensional subject. In this report, we develop a mathematical and statistical framework to estimate shear stress from rapid imaging sequences based firstly on stochastic modelling of the dynamics of a tethered Brownian fibre in shear flow, and secondly on a novel model-based image analysis, which reconstructs fibre positions by solving the inverse problem of image formation. This framework is tested on experimental data, providing the first mechanistically rational analysis of the novel assay. What follows further develops the established theory for an untethered particle in a semi-dilute suspension, which is of relevance to, for example, the study of Brownian nanowires without flow, and presents new ideas in the field of multi-disciplinary image analysis.

Three-Dimensional Tracking of Quantum Dot-Conjugated Molecules in Living Cells

Here, we describe protocols for three-dimensional tracking of single quantum dot-conjugated molecules with nanometer accuracy in living cells using conventional fluorescence microscopy. The technique exploits out-of-focus images of single emitters combined with an automated pattern-recognition open-source software that fits the images with proper model functions to extract the emitter coordinates. We describe protocols for targeting quantum dots to both membrane components and cytosolic proteins.

Highly efficient multicolor multifocus microscopy by optimal design of diffraction binary gratings

Multifocus microscopy (MFM) allows sensitive and fast three-dimensional imaging. It relies on the efficient design of diffraction phase gratings yielding homogeneous intensities in desired diffraction orders. Such performances are however guaranteed only for a specific wavelength. Here, we discuss a novel approach for designing binary phase gratings with dual color properties and improved diffraction efficiency for MFM. We simulate binary diffraction gratings with tunable phase shifts to explore its best diffraction performances. We report the design and fabrication of a binary array generator of 3 × 3 equal-intensity diffraction orders with 74% efficiency, 95% uniformity and dual color capability. The multicolor properties of this new design are highlighted by two-color MFM imaging. Finally, we discuss the basics of extending this approach to a variety of diffraction pattern designs.

Optical mapping near-eye three-dimensional display with correct focus cues

We present an optical mapping near-eye (OMNI) three-dimensional display method for wearable devices. By dividing a display screen into different subpanels and optically mapping them to various depths, we create a multiplane volumetric image with correct focus cues for depth perception. The resultant system can drive the eye's accommodation to the distance that is consistent with binocular stereopsis, thereby alleviating the vergence-accommodation conflict, the primary cause for eye fatigue and discomfort. Compared with the previous methods, the OMNI display offers prominent advantages in adaptability, image dynamic range, and refresh rate.

Three-Dimensional Localization of Single Molecules for Super-Resolution Imaging and Single-Particle Tracking

Single-molecule super-resolution fluorescence microscopy and single-particle tracking are two imaging modalities that illuminate the properties of cells and materials on spatial scales down to tens of nanometers or with dynamical information about nanoscale particle motion in the millisecond range, respectively. These methods generally use wide-field microscopes and two-dimensional camera detectors to localize molecules to much higher precision than the diffraction limit. Given the limited total photons available from each single-molecule label, both modalities require careful mathematical analysis and image processing. Much more information can be obtained about the system under study by extending to three-dimensional (3D) single-molecule localization: without this capability, visualization of structures or motions extending in the axial direction can easily be missed or confused, compromising scientific understanding. A variety of methods for obtaining both 3D super-resolution images and 3D tracking information have been devised, each with their own strengths and weaknesses. These include imaging of multiple focal planes, point-spread-function engineering, and interferometric detection. These methods may be compared based on their ability to provide accurate and precise position information on single-molecule emitters with limited photons. To successfully apply and further develop these methods, it is essential to consider many practical concerns, including the effects of optical aberrations, field dependence in the imaging system, fluorophore labeling density, and registration between different color channels. Selected examples of 3D super-resolution imaging and tracking are described for illustration from a variety of biological contexts and with a variety of methods, demonstrating the power of 3D localization for understanding complex systems.

Intensity-based axial localization approaches for multifocal plane microscopy

Multifocal plane microscopy (MUM) can be used to visualize biological samples in three dimensions over large axial depths and provides for the high axial localization accuracy that is needed in applications such as the three-dimensional tracking of single particles and super-resolution microscopy. This report analyzes the performance of intensity-based axial localization approaches as applied to MUM data using Fisher information calculations. In addition, a new non-parametric intensity-based axial location estimation method, Multi-Intensity Lookup Algorithm (MILA), is introduced that, unlike typical intensity-based methods that make use of a single intensity value per data image, utilizes multiple intensity values per data image in determining the axial location of a point source. MILA is shown to be robust against potential bias induced by differences in the sub-pixel location of the imaged point source. The method's effectiveness on experimental data is also evaluated.

In vivo volumetric fluorescence sectioning microscopy with mechanical-scan-free hybrid illumination imaging

Optical sectioning microscopy in wide-field fashion has been widely used to obtain three-dimensional images of biological samples; however, it requires scanning in depth and considerable time to acquire multiple depth information of a volumetric sample. In this paper, in vivo optical sectioning microscopy with volumetric hybrid illumination, with no mechanical moving parts, is presented. The proposed system is configured such that the optical sectioning is provided by hybrid illumination using a digital micro-mirror device (DMD) for uniform and non-uniform pattern projection, while the depth of imaging planes is varied by using an electrically tunable-focus lens with invariant magnification and resolution. We present and characterize the design, implementation, and experimentally demonstrate the proposed system's ability through 3D imaging of in vivo Canenorhabditis elegans' growth cones.

High-resolution real-time dual-view imaging with multiple point of view microscopy

Most methods to observe three-dimensional processes in living samples are based on imaging a single plane that is sequentially scanned through the sample. Sequential scanning is inherently slow, which can make it difficult to capture objects moving quickly in three dimensions. Here we present a novel method, multiple point-of-view microscopy (MPoVM), that allows simultaneous capturing of the front and side views of a sample with high resolution. MPoVM can be implemented in most fluorescence microscopes, offering new opportunities in the study of dynamic biological processes in three dimensions.

Ultrafast axial scanning for two-photon microscopy via a digital micromirror device and binary holography

In this Letter, we present an ultrafast nonmechanical axial scanning method for two-photon excitation (TPE) microscopy based on binary holography using a digital micromirror device (DMD), achieving a scanning rate of 4.2 kHz, scanning range of ∼180  μm, and scanning resolution (minimum step size) of ∼270  nm. Axial scanning is achieved by projecting the femtosecond laser to a DMD programmed with binary holograms of spherical wavefronts of increasing/decreasing radii. To guide the scanner design, we have derived the parametric relationships between the DMD parameters (i.e., aperture and pixel size), and the axial scanning characteristics, including (1) maximum optical power, (2) minimum step size, and (3) scan range. To verify the results, the DMD scanner is integrated with a custom-built TPE microscope that operates at 60 frames per second. In the experiment, we scanned a pollen sample via both the DMD scanner and a precision z-stage. The results show the DMD scanner generates images of equal quality throughout the scanning range. The overall efficiency of the TPE system was measured to be ∼3%. With the high scanning rate, the DMD scanner may find important applications in random-access imaging or high-speed volumetric imaging that enables visualization of highly dynamic biological processes in 3D with submillisecond temporal resolution.

Three-dimensional particle tracking via tunable color-encoded multiplexing

We present a novel 3D tracking approach capable of locating single particles with nanometric precision over wide axial ranges. Our method uses a fast acousto-optic liquid lens implemented in a bright field microscope to multiplex light based on color into different and selectable focal planes. By separating the red, green, and blue channels from an image captured with a color camera, information from up to three focal planes can be retrieved. Multiplane information from the particle diffraction rings enables precisely locating and tracking individual objects up to an axial range about 5 times larger than conventional single-plane approaches. We apply our method to the 3D visualization of the well-known coffee-stain phenomenon in evaporating water droplets.

A review of snapshot multidimensional optical imaging: measuring photon tags in parallel

Multidimensional optical imaging has seen remarkable growth in the past decade. Rather than measuring only the two-dimensional spatial distribution of light, as in conventional photography, multidimensional optical imaging captures light in up to nine dimensions, providing unprecedented information about incident photons' spatial coordinates, emittance angles, wavelength, time, and polarization. Multidimensional optical imaging can be accomplished either by scanning or parallel acquisition. Compared with scanning-based imagers, parallel acquisition-also dubbed snapshot imaging-has a prominent advantage in maximizing optical throughput, particularly when measuring a datacube of high dimensions. Here, we first categorize snapshot multidimensional imagers based on their acquisition and image reconstruction strategies, then highlight the snapshot advantage in the context of optical throughput, and finally we discuss their state-of-the-art implementations and applications.

3D tracking of single nanoparticles and quantum dots in living cells by out-of-focus imaging with diffraction pattern recognition

Live cells are three-dimensional environments where biological molecules move to find their targets and accomplish their functions. However, up to now, most single molecule investigations have been limited to bi-dimensional studies owing to the complexity of 3d-tracking techniques. Here, we present a novel method for three-dimensional localization of single nano-emitters based on automatic recognition of out-of-focus diffraction patterns. Our technique can be applied to track the movements of single molecules in living cells using a conventional epifluorescence microscope. We first demonstrate three-dimensional localization of fluorescent nanobeads over 4 microns depth with accuracy below 2 nm in vitro. Remarkably, we also establish three-dimensional tracking of Quantum Dots, overcoming their anisotropic emission, by adopting a ligation strategy that allows rotational freedom of the emitter combined with proper pattern recognition. We localize commercially available Quantum Dots in living cells with accuracy better than 7 nm over 2 microns depth. We validate our technique by tracking the three-dimensional movements of single protein-conjugated Quantum Dots in living cell. Moreover, we find that important localization errors can occur in off-focus imaging when improperly calibrated and we give indications to avoid them. Finally, we share a Matlab script that allows readily application of our technique by other laboratories.

Precise measurement of three-dimensional positions of transparent ellipsoidal particles using digital holographic microscopy

The dynamic motions of various particles suspended in microscale flows are essential phenomena in the scientific and engineering fields. These motions can be precisely measured by using 3D quantitative flow visualization techniques. Moreover, most cells and particles in nature possess a nonspherical shape. Digital holographic microscopy (DHM) is employed to measure the 3D positional information of transparent ellipsoidal particles. Both in-plane and out-of-plane positional information are obtained by analyzing the distinctive light scattering from the microsized ellipsoidal particles. The performance of the 3D position measurement method is experimentally verified for ellipsoidal particles seeded in a planar surface and a microtube. This DHM technique exhibits promising potential in the dynamic analysis of ellipsoidal particles and cells suspended in various microscale fluid flows.

Efficient generation of diffraction-limited multi-sheet pattern for biological imaging

We demonstrate a new technique to generate multiple light-sheets for fluorescence microscopy. This is possible by illuminating the cylindrical lens using multiple copies of Gaussian beams. A diffraction grating placed just before the cylindrical lens splits the incident Gaussian beam into multiple beams traveling at different angles. Subsequently, this gives rise to diffraction-limited light-sheets after the Gaussian beams pass through the combined cylindrical lens-objective sub-system. Direct measurement of field at and around the focus of objective lens shows multi-sheet pattern with an average thickness of 7.5 μm and inter-sheet separation of 380 μm. Employing an independent orthogonal detection sub-system, we successfully imaged fluorescently-coated yeast cells (≈4  μm) encaged in agarose gel-matrix. Such a diffraction-limited sheet-pattern equipped with dedicated detection system may find immediate applications in the field of optical microscopy and fluorescence imaging.

An information-theoretic approach to designing the plane spacing for multifocal plane microscopy

Multifocal plane microscopy (MUM) is a 3D imaging modality which enables the localization and tracking of single molecules at high spatial and temporal resolution by simultaneously imaging distinct focal planes within the sample. MUM overcomes the depth discrimination problem of conventional microscopy and allows high accuracy localization of a single molecule in 3D along the z-axis. An important question in the design of MUM experiments concerns the appropriate number of focal planes and their spacings to achieve the best possible 3D localization accuracy along the z-axis. Ideally, it is desired to obtain a 3D localization accuracy that is uniform over a large depth and has small numerical values, which guarantee that the single molecule is continuously detectable. Here, we address this concern by developing a plane spacing design strategy based on the Fisher information. In particular, we analyze the Fisher information matrix for the 3D localization problem along the z-axis and propose spacing scenarios termed the strong coupling and the weak coupling spacings, which provide appropriate 3D localization accuracies. Using these spacing scenarios, we investigate the detectability of the single molecule along the z-axis and study the effect of changing the number of focal planes on the 3D localization accuracy. We further review a software module we recently introduced, the MUMDesignTool, that helps to design the plane spacings for a MUM setup.

Three dimensional multi-molecule tracking in thick samples with extended depth-of-field

We present a non-z-scanning multi-molecule tracking system with nano-resolution in all three dimensions and extended depth of field (DOF), which based on distorted grating (DG) and double-helix point spread function (DH-PSF) combination microscopy (DDCM). The critical component in DDCM is a custom designed composite phase mask (PM) combining the functions of DG and DH-PSF. The localization precision and the effective DOF of the home-built DDCM system based on the designed PM were tested. Our experimental results show that the three-dimensional (3D) localization precision for the three diffraction orders of the grating are σ(-1st)(x, y, z) = (6.5 nm, 9.2nm, 23.4 nm), σ(0th)(x, y, z) = (3.7 nm, 2.8nm, 10.3 nm), and σ(+1s)(x, y, z) = (5.8 nm, 6.9 nm, 18.4 nm), respectively. Furthermore, the total effective DOF of the DDCM system is extended to 14 μm. Tracking experiment demonstrated that beads separated over 12 μm along the axial direction at some instants can be localized and tracked successfully.

High resolution depth-resolved imaging from multi-focal images for medical ultrasound

An ultrasound imaging technique providing sub-diffraction limit axial resolution for point sources is proposed. It is based on simultaneously acquired multi-focal images of the same object, and on the image metric of sharpness. The sharpness is extracted by image data and presents higher values for in-focus images. The technique is derived from biological microscopy and is validated here with simulated ultrasound data. A linear array probe is used to scan a point scatterer phantom that moves in depth with a controlled step. From the beamformed responses of each scatterer position the image sharpness is assessed. Values from all positions plotted together form a curve that peaks at the receive focus, which is set during the beamforming. Selection of three different receive foci for each acquired dataset will result in the generation of three overlapping sharpness curves. A set of three calibration curves combined with the use of a maximum-likelihood algorithm is then able to estimate, with high precision, the depth location of any emitter fron each single image. Estimated values are compared with the ground truth demonstrating that an accuracy of 28.6 μm (0.13λ) is achieved for a 4 mm depth range.

Rapid determination of cell mass and density using digitally controlled electric field in a microfluidic chip

The density of a single cell is a fundamental property of cells. Cells in the same cycle phase have similar volume, but the differences in their mass and density could elucidate each cell's physiological state. Here we report a novel technique to rapidly measure the density and mass of a single cell using an optically induced electrokinetics (OEK) microfluidic platform. Presently, single cellular mass and density measurement devices require a complicated fabrication process and their output is not scalable, i.e., it is extremely difficult to measure the mass and density of a large quantity of cells rapidly. The technique reported here operates on a principle combining sedimentation theory, computer vision, and microparticle manipulation techniques in an OEK microfluidic platform. We will show in this paper that this technique enables the measurement of single-cell volume, density, and mass rapidly and accurately in a repeatable manner. The technique is also scalable - it allows simultaneous measurement of volume, density, and mass of multiple cells. Essentially, a simple time-controlled projected light pattern is used to illuminate the selected area on the OEK microfluidic chip that contains cells to lift the cells to a particular height above the chip's surface. Then, the cells are allowed to "free fall" to the chip's surface, with competing buoyancy, gravitational, and fluidic drag forces acting on the cells. By using a computer vision algorithm to accurately track the motion of the cells and then relate the cells' motion trajectory to sedimentation theory, the volume, mass, and density of each cell can be rapidly determined. A theoretical model of micro-sized spheres settling towards an infinite plane in a microfluidic environment is first derived and validated experimentally using standard micropolystyrene beads to demonstrate the viability and accuracy of this new technique. Next, we show that the yeast cell volume, mass, and density could be rapidly determined using this technology, with results comparable to those using the existing method suspended microchannel resonator.

Interferometric three-dimensional single molecule localization microscopy using a single high-numerical-aperture objective

Interferometric detection of the fluorescence emission from a single molecule [interferometric photoactivated localization microscopy (iPALM)] enables a localization accuracy of nanometers in axial localization for 3D superresolution imaging. However, iPALM uses two high-numerical-aperture (NA) objectives in juxtaposition for fluorescence collection (a 4Pi microscope geometry), increasing expense and limiting samples that can be studied. Here, we propose an interferometric single molecule localization microscopy method using a single high-NA objective. The axial position of single molecules can be unambiguously determined from the phase-shifted interference signals with nanometer precision and over a range of 2λ. The use of only one objective simplifies the system configuration and sample mounting. In addition, due to the use of wavefront-splitting interference in our approach, the two parts of the wavefront that eventually merge and interfere with each other travel along nearly equivalent optical paths, which should minimize the effect of drift for long-term 3D superresolution imaging.

Temporal resolution in fluorescence imaging

Temporal resolution is a key factor for imaging rapidly occurring events in biology. In this feature article, I investigate an approximate estimate for determining the temporal resolution limit. The condition that led to this limit is, the time taken by the ensemble (99.9%) of excited molecules to relax to ground state, assuming all the emitted photons are detected. In a simplistic three-level system, the temporal resolution is, ≈3τ p , where τ p = (log e 10)/(kf + knr ) and, kf and knr are respectively the radiative and non-radiative emission rates. This further assumes the ideal condition that, the quantum efficiency of the detector is unity and there are no other loses. We discuss few state-of-art microscopy techniques that are capable of high temporal resolution. This includes techniques such as multifocal multiphoton microscopy (MMM), multifocal plane microscopy, multiple excitation spot optical microscopy (MESO), multiplane microscopy and multiple light-sheet microscopy (MLSM).

Nano-scale measurement of biomolecules by optical microscopy and semiconductor nanoparticles

Over the past decade, great developments in optical microscopy have made this technology increasingly compatible with biological studies. Fluorescence microscopy has especially contributed to investigating the dynamic behaviors of live specimens and can now resolve objects with nanometer precision and resolution due to super-resolution imaging. Additionally, single particle tracking provides information on the dynamics of individual proteins at the nanometer scale both in vitro and in cells. Complementing advances in microscopy technologies has been the development of fluorescent probes. The quantum dot, a semi-conductor fluorescent nanoparticle, is particularly suitable for single particle tracking and super-resolution imaging. This article overviews the principles of single particle tracking and super resolution along with describing their application to the nanometer measurement/observation of biological systems when combined with quantum dot technologies.

Designing the focal plane spacing for multifocal plane microscopy

Multifocal plane microscopy (MUM) has made it possible to study subcellular dynamics in 3D at high temporal and spatial resolution by simultaneously imaging distinct planes within the specimen. MUM allows high accuracy localization of a point source along the z-axis since it overcomes the depth discrimination problem of conventional single plane microscopy. An important question in MUM experiments is how the number of focal planes and their spacings should be chosen to achieve the best possible localization accuracy along the z-axis. Here, we propose approaches based on the Fisher information matrix and report spacing scenarios called strong coupling and weak coupling which yield an appropriate 3D localization accuracy. We examine the effect of numerical aperture, magnification, photon count, emission wavelength and extraneous noise on the spacing scenarios. In addition, we investigate the effect of changing the number of focal planes on the 3D localization accuracy. We also introduce a new software package that provides a user-friendly framework to find appropriate plane spacings for a MUM setup. These developments should assist in optimizing MUM experiments.

Three dimensional fluorescence imaging using multiple light-sheet microscopy

We developed a multiple light-sheet microscopy (MLSM) system capable of 3D fluorescence imaging. Employing spatial filter in the excitation arm of a SPIM system, we successfully generated multiple light-sheets. This improves upon the existing SPIM system and is capable of 3D volume imaging by simultaneously illuminating multiple planes in the sample. Theta detection geometry is employed for data acquisition from multiple specimen layers. This detection scheme inherits many advantages including, background reduction, cross-talk free fluorescence detection and high-resolution at long working distance. Using this technique, we generated equi-intense light-sheets of thickness approximately with an inter-sheet separation of . Moreover, the light-sheets generated by MLSM is found to be 2 times thinner than the state-of-art SPIM system. Imaging of fluorescently coated yeast cells of size (encaged in Agarose gel-matrix) is achieved. Proposed imaging technique may accelerate the field of fluorescence microscopy, cell biology and biophotonics.

Approach to multiparticle parallel tracking in thick samples with three-dimensional nanoresolution

This Letter proposes a method referred to as distorted grating (DG) and double-helix point spread function (DH-PSF) combination microscopy (DDCM), which is capable of multiparticle parallel localization and tracking in a transparent sample thicker than 10 μm, the thickness of cells. A special phase mask, combining the field depth extension capabilities of DG with the three-dimensional (3D) nanolocalization capabilities of the DH-PSF, is designed for multiparticle parallel localization. Time-lapse tracking of one particle moving along the z axis and parallel tracking of two particles are simulated. Results demonstrate that, with only a single snapshot, particles can be localized, tracking with 3D nanoresolution wherever they are. The theoretical localization precisions of DDCM, DH-PSF, and multifocus microscopy are compared. DDCM results in almost constant localization precisions in all three dimensions for a depth of field larger than 10 μm. DDCM is expected to become a tool in investigations of important dynamic events in living cells.

Rapid 3D light-sheet microscopy with a tunable lens

The in-vivo investigation of highly dynamic biological samples, for example the beating zebrafish heart, requires high-speed volume imaging techniques. Light-sheet microscopy is ideal for such samples as it records high-contrast images of entire planes within large samples at once. However, in order to obtain images of different planes, it has been necessary to move the sample relative to the fixed focal plane of the detection objective lens. This mechanical movement limits speed, precision and may be harmful to the sample. We have built a light-sheet microscope that uses remote focusing with an electrically tunable lens (ETL). Without moving specimen or objective we have thereby achieved flexible volume imaging at much higher speeds than previously reported. Our high-speed microscope delivers 3D snapshots of sensitive biological samples. As an example, we imaged 17 planes within a beating zebrafish heart at 510 frames per second, equivalent to 30 volume scans per second. Movements, shape changes and signals across the entire volume can be followed which has been impossible with existing reconstruction techniques.

Four-dimensional spatial nanometry of single particles in living cells using polarized quantum rods

Single particle tracking is widely used to study protein movement with high spatiotemporal resolution both in vitro and in cells. Quantum dots, which are semiconductor nanoparticles, have recently been employed in single particle tracking because of their intense and stable fluorescence. Although single particles inside cells have been tracked in three spatial dimensions (X, Y, Z), measurement of the angular orientation of a molecule being tracked would significantly enhance our understanding of the molecule's function. In this study, we synthesized highly polarized, rod-shaped quantum dots (Qrods) and developed a coating method that optimizes the Qrods for biological imaging. We describe a Qrod-based single particle tracking technique that blends optical nanometry with nanomaterial science to simultaneously measure the three-dimensional and angular movements of molecules. Using Qrods, we spatially tracked a membrane receptor in living cells in four dimensions with precision close to the single-digit range in nanometers and degrees.

Distorted Dammann grating

We introduce the Dammann phase-encoding method into original distorted gratings and propose a modified distorted grating, called a distorted Dammann grating (DDG), to realize multiplane imaging of several tens of layers within the object field onto a single image plane. This property implies that the DDG makes it possible to achieve simultaneously high axial resolving power and large axial imaging range without scanning. This DDG should be of high interest for its potential applications in real-time three-dimensional optical imaging and tracking. Multiplane imaging of 7×7 object layers onto a single camera plane is experimentally demonstrated using a 7×7 DDG for an objective of NA=0.127.

Smaller, faster, brighter: advances in optical imaging of living plant cells

The advent of fluorescent proteins and access to modern imaging technologies have dramatically accelerated the pace of discovery in plant cell biology. Remarkable new insights into such diverse areas as plant pathogenesis, cytoskeletal dynamics, sugar transport, cell wall synthesis, secretory control, and hormone signaling have come from careful examination of living cells using advanced optical probes. New technologies, both commercially available and on the horizon, promise a continued march toward more quantitative methods for imaging and for extending the optical exploration of biological structure and activity to molecular scales. In this review, we lay out fundamental issues in imaging plant specimens and look ahead to several technological innovations in molecular tools, instrumentation, imaging methods, and specimen handling that show promise for shaping the coming era of plant cell biology.