Quantitative study of single molecule location estimation techniques

Localization Study of Photostable Alexa 488 at Single Molecule Level

Understanding the relationships between molecular organization and dynamics of a complex system is very important to understand the photophysical properties of such system. This paper focuses on a novel strategy based on single molecule spectroscopy and single molecule localization microscopy to elucidate the photostability and localization of a fluorophore molecule on a 2D biomembrane. Improvement of in-plane resolution of a signal in a nano-dimension within the diffraction limit has been discussed in a new way. And, how this better in-plane resolution information can be used for precise localization of a single molecule on a 2D system has also been discussed.

BNP-Track: A framework for multi-particle superresolved tracking

When tracking fluorescently labeled molecules (termed "emitters") under widefield microscopes, point spread function overlap of neighboring molecules is inevitable in both dilute and especially crowded environments. In such cases, superresolution methods leveraging rare photophysical events to distinguish static targets nearby in space introduce temporal delays that compromise tracking. As we have shown in a companion manuscript, for dynamic targets, information on neighboring fluorescent molecules is encoded as spatial intensity correlations across pixels and temporal correlations in intensity patterns across time frames. We then demonstrated how we used all spatiotemporal correlations encoded in the data to achieve superresolved tracking. That is, we showed the results of full posterior inference over both the number of emitters and their associated tracks simultaneously and self-consistently through Bayesian nonparametrics. In this companion manuscript we focus on testing the robustness of our tracking tool, BNP-Track, across sets of parameter regimes and compare BNP-Track to competing tracking methods in the spirit of a prior Nature Methods tracking competition. We explore additional features of BNP-Track including how a stochastic treatment of background yields greater accuracy in emitter number determination and how BNP-Track corrects for point spread function blur (or "aliasing") introduced by intraframe motion in addition to propagating error originating from myriad sources (such as criss-crossing tracks, out-of-focus particles, pixelation, shot and camera artefact, stochastic background) in posterior inference over emitter numbers and their associated tracks. While head-to-head comparison with other tracking methods is not possible (as competitors cannot simultaneously learn molecule numbers and associated tracks), we can give competing methods some advantages in order to perform approximate head-to-head comparison. We show that even under such optimistic scenarios, BNP-Track is capable of tracking multiple diffraction-limited point emitters conventional tracking methods cannot resolve thereby extending the superresolution paradigm to dynamical targets.

Partition of estimated locations: an approach to accurate quality metrics for stochastic optical localization nanoscopy

Performance evaluation of localization algorithms in stochastic optical localization nanoscopy is necessary and important to applications. By simulation, a localization algorithm estimates a set of emitter locations from a simulated data movie, whose error in comparison with the set of true locations indicates the performance of the algorithm. Since the partition of estimated locations is unknown, the sample root mean square error (RMSE) cannot be computed, and the universal root mean square minimum distance (RMSMD) eventually becomes saturated as localization errors become large. In this paper, we propose a partition algorithm to estimate the partition of estimated locations. It makes use of three facts: (i) the true locations are known; (ii) the number of activations for each emitter is known; (iii) an estimated location is more likely to be associated with the nearest available emitter and vice versa. The estimated partition enables computation of the sample RMSE (RMSE-P) and improvement of the RMSMD with modification (RMSMD-P). Two simulations are carried out to demonstrate the efficacy of the partition algorithm and the metrics of RMSE-P and RMSMD-P. One investigates the effect of a large range of localization biases, and the other examines performance of the unbiased Gaussian information-achieving (UGIA) estimator. As shown by the results of both simulations, the proposed partition algorithm accurately estimates the partition in terms of the F1 score; with the partition estimated by the partition algorithm, the RMSE-P and RMSMD-P are approximately equal to the RMSE with the true partition in a large range of localization biases and errors. This demonstrates their broad applicability in performance evaluation of localization algorithms under the benchmark of the UGIA estimator.

Deep learning in single-molecule imaging and analysis: recent advances and prospects

Single-molecule microscopy is advantageous in characterizing heterogeneous dynamics at the molecular level. However, there are several challenges that currently hinder the wide application of single molecule imaging in bio-chemical studies, including how to perform single-molecule measurements efficiently with minimal run-to-run variations, how to analyze weak single-molecule signals efficiently and accurately without the influence of human bias, and how to extract complete information about dynamics of interest from single-molecule data. As a new class of computer algorithms that simulate the human brain to extract data features, deep learning networks excel in task parallelism and model generalization, and are well-suited for handling nonlinear functions and extracting weak features, which provide a promising approach for single-molecule experiment automation and data processing. In this perspective, we will highlight recent advances in the application of deep learning to single-molecule studies, discuss how deep learning has been used to address the challenges in the field as well as the pitfalls of existing applications, and outline the directions for future development.

A common framework for single-molecule localization using sequential structured illumination

Localization of single fluorescent molecules is key for physicochemical and biophysical measurements, such as single-molecule tracking and super-resolution imaging by single-molecule localization microscopy. Over the last two decades, several methods have been developed in which the position of a single emitter is interrogated with a sequence of spatially modulated patterns of light. Among them, the recent MINFLUX technique outstands for achieving a ∼10-fold improvement compared with wide-field camera-based single-molecule localization, reaching ∼1-2 nm localization precision at moderate photon counts. Here, we present a common framework for this type of measurement. Using the Cramér-Rao bound as a limit for the achievable localization precision, we benchmark reported methods, including recent developments, such as MINFLUX and MINSTED, and long-established methods, such as orbital tracking. In addition, we characterize two new proposed schemes, orbital tracking and raster scanning, with a minimum of intensity. Overall, we found that approaches using an intensity minimum have a similar performance in the central region of the excitation pattern, independent of the geometry of the excitation pattern, and that they outperform methods featuring an intensity maximum.

Combining deep learning with SUPPOSe and compressed sensing for SNR-enhanced localization of overlapping emitters

We present gSUPPOSe, a novel, to the best of our knowledge, gradient-based implementation of the SUPPOSe algorithm that we have developed for the localization of single emitters. We study the performance of gSUPPOSe and compressed sensing STORM (CS-STORM) on simulations of single-molecule localization microscopy (SMLM) images at different fluorophore densities and in a wide range of signal-to-noise ratio conditions. We also study the combination of these methods with prior image denoising by means of a deep convolutional network. Our results show that gSUPPOSe can address the localization of multiple overlapping emitters even at a low number of acquired photons, outperforming CS-STORM in our quantitative analysis and having better computational times. We also demonstrate that image denoising greatly improves CS-STORM, showing the potential of deep learning enhanced localization on existing SMLM algorithms. The software developed in this work is available as open source Python libraries.

Simultaneous visualization of DNA loci in single cells by combinatorial multi-color iFISH

Single-molecule DNA fluorescence in situ hybridization (FISH) techniques enable studying the three-dimensional (3D) organization of the genome at the single cell level. However, there is a major unmet need for open access, high quality, curated and reproducible DNA FISH datasets. Here, we describe a dataset obtained by applying our recently developed iFISH method to simultaneously visualize 16 small (size range: 62–73 kilobases, kb) DNA loci evenly spaced on chromosome 2 in human cells, in a single round of hybridization. We show how combinatorial color coding can be used to precisely localize multiple loci in 3D within single cells, and how inter-locus distances scale inversely with chromosome contact frequencies determined by high-throughput chromosome conformation capture (Hi-C). We provide raw images and 3D coordinates for nearly 10,000 FISH dots. Our dataset provides a free resource that can facilitate studies of 3D genome organization in single cells and can be used to develop automatic FISH analysis algorithms.

Measurement(s)	DNA loci 3D coordinates
Technology Type(s)	Fluorescence In Situ Hybridization
Factor Type(s)	DNA FISH probe target (locus)
Sample Characteristic - Organism	Homo sapiens

Machine-accessible metadata file describing the reported data: 10.6084/m9.figshare.17281358

On the validity domain of maximum likelihood estimators for depth-of-field extension in single-molecule localization microscopy

Localization microscopy approaches with enhanced depth-of-field (EDoF) are commonly optimized using the Cramér-Rao bound (CRB) as a criterion. It is widely believed that the CRB can be attained in practice by using the maximum-likelihood estimator (MLE). This is, however, an approximation, of which we define in this paper the precise domain of validity. Exploring a wide range of settings and noise levels, we show that the MLE is efficient when the signal-to-noise ratio (SNR) is such that the localization standard deviation of a single molecule is less than 20 nm. Thus, our results provide an explicit and quantitative validity boundary for the use of the MLE in EDoF localization microscopy setups optimized with the CRB.

Modification to axial tracking for mobile magnetic microspheres

Three-dimensional particle tracking is a routine experimental procedure for various biophysical applications including magnetic tweezers. A common method for tracking the axial position of particles involves the analysis of diffraction rings whose pattern depends sensitively on the axial position of the bead relative to the focal plane. To infer the axial position, the observed rings are compared with reference images of a bead at known axial positions. Often the precision or accuracy of these algorithms is measured on immobilized beads over a limited axial range, while many experiments are performed using freely mobile beads. This inconsistency raises the possibility of incorrect estimates of experimental uncertainty. By manipulating magnetic beads in a bidirectional magnetic tweezer setup, we evaluated the error associated with tracking mobile magnetic beads and found that the error of tracking a moving magnetic bead increases by almost an order of magnitude compared to the error of tracking a stationary bead. We found that this additional error can be ameliorated by excluding the center-most region of the diffraction ring pattern from tracking analysis. Evaluation of the limitations of a tracking algorithm is essential for understanding the error associated with a measurement. These findings promise to bring increased resolution to three-dimensional bead tracking of magnetic microspheres.

Light Sheet Illumination for 3D Single-Molecule Super-Resolution Imaging of Neuronal Synapses

The function of the neuronal synapse depends on the dynamics and interactions of individual molecules at the nanoscale. With the development of single-molecule super-resolution microscopy over the last decades, researchers now have a powerful and versatile imaging tool for mapping the molecular mechanisms behind the biological function. However, imaging of thicker samples, such as mammalian cells and tissue, in all three dimensions is still challenging due to increased fluorescence background and imaging volumes. The combination of single-molecule imaging with light sheet illumination is an emerging approach that allows for imaging of biological samples with reduced fluorescence background, photobleaching, and photodamage. In this review, we first present a brief overview of light sheet illumination and previous super-resolution techniques used for imaging of neurons and synapses. We then provide an in-depth technical review of the fundamental concepts and the current state of the art in the fields of three-dimensional single-molecule tracking and super-resolution imaging with light sheet illumination. We review how light sheet illumination can improve single-molecule tracking and super-resolution imaging in individual neurons and synapses, and we discuss emerging perspectives and new innovations that have the potential to enable and improve single-molecule imaging in brain tissue.

dNEMO: a tool for quantification of mRNA and punctate structures in time-lapse images of single cells

MOTIVATION

Many biological processes are regulated by single molecules and molecular assemblies within cells that are visible by microscopy as punctate features, often diffraction limited. Here, we present detecting-NEMO (dNEMO), a computational tool optimized for accurate and rapid measurement of fluorescent puncta in fixed-cell and time-lapse images.

RESULTS

The spot detection algorithm uses the à trous wavelet transform, a computationally inexpensive method that is robust to imaging noise. By combining automated with manual spot curation in the user interface, fluorescent puncta can be carefully selected and measured against their local background to extract high-quality single-cell data. Integrated into the workflow are segmentation and spot-inspection tools that enable almost real-time interaction with images without time consuming pre-processing steps. Although the software is agnostic to the type of puncta imaged, we demonstrate dNEMO using smFISH to measure transcript numbers in single cells in addition to the transient formation of IKK/NEMO puncta from time-lapse images of cells exposed to inflammatory stimuli. We conclude that dNEMO is an ideal user interface for rapid and accurate measurement of fluorescent molecular assemblies in biological imaging data.

AVAILABILITY AND IMPLEMENTATION

The data and software are freely available online at https://github.com/recleelab.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

A survey on applications of deep learning in microscopy image analysis

Advanced microscopy enables us to acquire quantities of time-lapse images to visualize the dynamic characteristics of tissues, cells or molecules. Microscopy images typically vary in signal-to-noise ratios and include a wealth of information which require multiple parameters and time-consuming iterative algorithms for processing. Precise analysis and statistical quantification are often needed for the understanding of the biological mechanisms underlying these dynamic image sequences, which has become a big challenge in the field. As deep learning technologies develop quickly, they have been applied in bioimage processing more and more frequently. Novel deep learning models based on convolution neural networks have been developed and illustrated to achieve inspiring outcomes. This review article introduces the applications of deep learning algorithms in microscopy image analysis, which include image classification, region segmentation, object tracking and super-resolution reconstruction. We also discuss the drawbacks of existing deep learning-based methods, especially on the challenges of training datasets acquisition and evaluation, and propose the potential solutions. Furthermore, the latest development of augmented intelligent microscopy that based on deep learning technology may lead to revolution in biomedical research.

Microscope calibration protocol for single-molecule microscopy

Single-molecule microscopy allows for the investigation of the dynamics of individual molecules and the visualization of subcellular structures at high spatial resolution. For single-molecule imaging experiments, and particularly those that entail the acquisition of multicolor data, calibration of the microscope and its optical components therefore needs to be carried out at a high level of accuracy. We propose here a method for calibrating a microscope at the nanometer scale, in the sense of determining optical aberrations as revealed by point source localization errors on the order of nanometers. The method is based on the imaging of a standard sample to detect and evaluate the amount of geometric aberration introduced in the optical light path. To provide support for multicolor imaging, it also includes procedures for evaluating the geometric aberration caused by a dichroic filter and the axial chromatic aberration introduced by an objective lens.

Practical Aspects of Super-Resolution Imaging and Segmentation of Macromolecular Complexes by dSTORM

Super-resolution fluorescence microscopy allows imaging macromolecular complexes down to the nanoscopic scale and thus is a great tool to combine and integrate cellular imaging in the native cellular environment with structural analysis by X-ray crystallography or high-resolution cryo electron microscopy or tomography. Here we describe practical aspects of SMLM imaging by dSTORM, from the initial sample preparation using mounting media, antibodies and fluorescent markers, the experimental setup for data acquisition including multi-color colocalization and 3D data acquisition, and finally tips and clues on advanced data processing that includes image reconstruction and data segmentation using 2D or 3D clustering methods. This approach opens the path toward multi-resolution integration in cellular structural biology.

Effect of Pixelation on the Parameter Estimation of Single Molecule Trajectories

The advent of single molecule microscopy has revolutionized biological investigations by providing a powerful tool for the study of intercellular and intracellular trafficking processes of protein molecules which was not available before through conventional microscopy. In practice, pixelated detectors are used to acquire the images of fluorescently labeled objects moving in cellular environments. Then, the acquired fluorescence microscopy images contain the numbers of the photons detected in each pixel, during an exposure time interval. Moreover, instead of having the exact locations of detection of the photons, we only know the pixel areas in which the photons impact the detector. These challenges make the analysis of single molecule trajectories, from pixelated images, a complex problem. Here, we investigate the effect of pixelation on the parameter estimation of single molecule trajectories. In particular, we develop a stochastic framework to calculate the maximum likelihood estimates of the parameters of a stochastic differential equation that describes the motion of the molecule in living cells. We also calculate the Fisher information matrix for this parameter estimation problem. The analytical results are complicated through the fact that the observation process in a microscope prohibits the use of standard Kalman filter type approaches. The analytical framework presented here is illustrated with examples of low photon count scenarios for which we rely on Monte Carlo methods to compute the associated probability distributions.

Three-Dimensional Single Particle Tracking and Its Applications in Confined Environments

Single particle tracking (SPT) has proven to be a powerful technique in studying molecular dynamics in complicated systems. We review its recent development, including three-dimensional (3D) SPT and its applications in probing nanostructures and molecule-surface interactions that are important to analytical chemical processes. Several frequently used 3D SPT techniques are introduced. Especially of interest are those based on point spread function engineering, which are simple in instrumentation and can be easily adapted and used in analytical labs. Corresponding data analysis methods are briefly discussed. We present several important case studies, with a focus on probing mass transport and molecule-surface interactions in confined environments. The presented studies demonstrate the great potential of 3D SPT for understanding fundamental phenomena in confined space, which will enable us to predict basic principles involved in chemical recognition, separation, and analysis, and to optimize mass transport and responses by structural design and optimization.

Precise 3D particle localization over large axial ranges using secondary astigmatism

We propose an analytical pupil phase function employing cropped secondary astigmatism for extended-depth nanoscale 3D-localization microscopy. The function provides high localization precision in all three dimensions, which can be maintained over extended axial ranges, customizable up to two orders of magnitude relative to the conventional, diffraction-limited imaging. This enables, for example, capturing nanoscale dynamics within a whole cell. The flexibility and simplicity in the implementation of the proposed phase function make its adoption in localization-based microscopy attractive. We demonstrate and validate its application to real-time imaging of 3D fluid flow over a depth of 40 µm with a numerical aperture of 0.8.

EV-Ident: Identifying Tumor-Specific Extracellular Vesicles by Size Fractionation and Single-Vesicle Analysis

Tumor-derived extracellular vesicles (EVs) have emerged as a promising source of circulating biomarkers for liquid biopsies. However, understanding the heterogeneous physical and biochemical properties of EVs originating from multiple complex biogenesis pathways remains a major challenge. Here, we introduce EV-Ident for preparation of subpopulations of EVs in three different size fractions: large EVs (EV_{200 nm}; 200-1 000 nm), medium EVs (EV_{100 nm}; 100-200 nm), and small EVs (EV_{20 nm}; 20-100 nm). Furthermore, this technology enables the in situ labeling of fluorescence markers for the protein profiling of individual EVs. As a proof-of-concept, we analyzed the presence of human epidermal growth factor receptor 2 (HER2) and prostate-specific membrane antigen (PSMA) in breast cancer and prostate cancer cell-derived EVs, respectively, using three different size fractions at the single-EV level. By reducing the complexity of EV heterogeneity in each size fraction, we found that HER2-positive breast cancer cells showed the greatest expression of HER2 in EV_{20 nm}, whereas PSMA expression was the highest in EV_{200 nm} derived from PSMA-expressing prostate cancer cells. This increase in HER2 expression in EV_{20 nm} and PSMA expression in EV_{200 nm} was further confirmed in plasma-derived nanoparticles (PNPs) obtained from breast and prostate cancer patients, respectively. Our study demonstrates that single-EV analysis using EV-Ident provides a practical way to understand EV heterogeneity and to successfully identify potent subpopulation of EVs for breast and prostate cancer, which has promising translational implications for cancer theranostics. Furthermore, these findings have the potential to address fundamental questions surrounding the biology and clinical applications of EVs.

Deep Learning-Based Single-Cell Optical Image Studies

Optical imaging technology that has the advantages of high sensitivity and cost-effectiveness greatly promotes the progress of nondestructive single-cell studies. Complex cellular image analysis tasks such as three-dimensional reconstruction call for machine-learning technology in cell optical image research. With the rapid developments of high-throughput imaging flow cytometry, big data cell optical images are always obtained that may require machine learning for data analysis. In recent years, deep learning has been prevalent in the field of machine learning for large-scale image processing and analysis, which brings a new dawn for single-cell optical image studies with an explosive growth of data availability. Popular deep learning techniques offer new ideas for multimodal and multitask single-cell optical image research. This article provides an overview of the basic knowledge of deep learning and its applications in single-cell optical image studies. We explore the feasibility of applying deep learning techniques to single-cell optical image analysis, where popular techniques such as transfer learning, multimodal learning, multitask learning, and end-to-end learning have been reviewed. Image preprocessing and deep learning model training methods are then summarized. Applications based on deep learning techniques in the field of single-cell optical image studies are reviewed, which include image segmentation, super-resolution image reconstruction, cell tracking, cell counting, cross-modal image reconstruction, and design and control of cell imaging systems. In addition, deep learning in popular single-cell optical imaging techniques such as label-free cell optical imaging, high-content screening, and high-throughput optical imaging cytometry are also mentioned. Finally, the perspectives of deep learning technology for single-cell optical image analysis are discussed. © 2020 International Society for Advancement of Cytometry.

Information theoretic analysis of hyperspectral imaging systems with applications to fluorescence microscopy

We present a general stochastic model for hyperspectral imaging data and derive analytical expressions for the Fisher information matrix for the underlying spectral unmixing problem. We investigate the linear mixing model as a special case and define a linear unmixing performance bound by using the Cramer-Rao inequality. As an application, we consider fluorescence imaging and show how the performance bound provides a spectral resolution limit that predicts how accurately a pair of spectrally similar fluorescent labels can be spectrally unmixed. We also report a novel result that shows how the spectral resolution limit can be overcome by exploiting the phenomenon of anti-Stokes shift fluorescence. In addition, we investigate how photon statistics, channel addition and channel splitting affect the performance bound. Finally by using the performance bound as a benchmark, we compare the performance of the least squares and the maximum likelihood estimators for spectral unmixing. For the imaging conditions tested here, our analysis shows that both estimators are unbiased and that the standard deviation of the maximum likelihood estimator is consistently closer to the performance bound than that of the least squares estimator. The results presented here are based on broad assumptions regarding the underlying data model and are applicable to hyperspectral data acquired with point detectors, sCMOS, CCD and EMCCD imaging detectors.

iFISH is a publically available resource enabling versatile DNA FISH to study genome architecture

DNA fluorescence in situ hybridization (DNA FISH) is a powerful method to study chromosomal organization in single cells. At present, there is a lack of free resources of DNA FISH probes and probe design tools which can be readily applied. Here, we describe iFISH, an open-source repository currently comprising 380 DNA FISH probes targeting multiple loci on the human autosomes and chromosome X, as well as a genome-wide database of optimally designed oligonucleotides and a freely accessible web interface ( http://ifish4u.org ) that can be used to design DNA FISH probes. We individually validate 153 probes and take advantage of our probe repository to quantify the extent of intermingling between multiple heterologous chromosome pairs, showing a much higher extent of intermingling in human embryonic stem cells compared to fibroblasts. In conclusion, iFISH is a versatile and expandable resource, which can greatly facilitate the use of DNA FISH in research and diagnostics.

Recent advances in optical microscopic methods for single-particle tracking in biological samples

With the rapid development of optical microscopic techniques, explorations on the chemical and biological properties of target objects in biological samples at single-molecule/particle level have received great attention recently. In the past decades, various powerful techniques have been developed for single-particle tracking (SPT) in biological samples. In this review, we summarize the commonly used optical microscopic methods for SPT, such as total internal reflection fluorescence microscopy (TIRFM), super-resolution fluorescence microscopy (SRM), dark-field optical microscopy (DFM), total internal reflection scattering microscopy (TIRSM), and differential interference contrast microscopy (DICM). We then discuss the image processing and data analysis methods, including particle localization, trajectory reconstruction, and diffusion behavior analysis. The application of SPT on the cell membrane, within the cell, and the cellular invading process of viruses are introduced. Finally, the challenges and prospects of optical microscopic technologies for SPT are delineated.

Deep learning enables cross-modality super-resolution in fluorescence microscopy

We present deep-learning-enabled super-resolution across different fluorescence microscopy modalities. This data-driven approach does not require numerical modeling of the imaging process or the estimation of a point-spread-function, and is based on training a generative adversarial network (GAN) to transform diffraction-limited input images into super-resolved ones. Using this framework, we improve the resolution of wide-field images acquired with low-numerical-aperture objectives, matching the resolution that is acquired using high-numerical-aperture objectives. We also demonstrate cross-modality super-resolution, transforming confocal microscopy images to match the resolution acquired with a stimulated emission depletion (STED) microscope. We further demonstrate that total internal reflection fluorescence (TIRF) microscopy images of subcellular structures within cells and tissues can be transformed to match the results obtained with a TIRF-based structured illumination microscope. The deep network rapidly outputs these super-resolved images, without any iterations or parameter search, and could serve to democratize super-resolution imaging.

Minimizing Structural Bias in Single-Molecule Super-Resolution Microscopy

Single-molecule localization microscopy (SMLM) depends on sequential detection and localization of individual molecular blinking events. Due to the stochasticity of single-molecule blinking and the desire to improve SMLM’s temporal resolution, algorithms capable of analyzing frames with a high density (HD) of active molecules, or molecules whose images overlap, are a prerequisite for accurate location measurements. Thus far, HD algorithms are evaluated using scalar metrics, such as root-mean-square error, that fail to quantify the structure of errors caused by the structure of the sample. Here, we show that the spatial distribution of localization errors within super-resolved images of biological structures are vectorial in nature, leading to systematic structural biases that severely degrade image resolution. We further demonstrate that the shape of the microscope’s point-spread function (PSF) fundamentally affects the characteristics of imaging artifacts. We built a Robust Statistical Estimation algorithm (RoSE) to minimize these biases for arbitrary structures and PSFs. RoSE accomplishes this minimization by estimating the likelihood of blinking events to localize molecules more accurately and eliminate false localizations. Using RoSE, we measure the distance between crossing microtubules, quantify the morphology of and separation between vesicles, and obtain robust recovery using diverse 3D PSFs with unmatched accuracy compared to state-of-the-art algorithms.

Subnanometer localization accuracy in widefield optical microscopy

The common assumption that precision is the limit of accuracy in localization microscopy and the typical absence of comprehensive calibration of optical microscopes lead to a widespread issue-overconfidence in measurement results with nanoscale statistical uncertainties that can be invalid due to microscale systematic errors. In this article, we report a comprehensive solution to this underappreciated problem. We develop arrays of subresolution apertures into the first reference materials that enable localization errors approaching the atomic scale across a submillimeter field. We present novel methods for calibrating our microscope system using aperture arrays and develop aberration corrections that reach the precision limit of our reference materials. We correct and register localization data from multiple colors and test different sources of light emission with equal accuracy, indicating the general applicability of our reference materials and calibration methods. In a first application of our new measurement capability, we introduce the concept of critical-dimension localization microscopy, facilitating tests of nanofabrication processes and quality control of aperture arrays. In a second application, we apply these stable reference materials to answer open questions about the apparent instability of fluorescent nanoparticles that commonly serve as fiducial markers. Our study establishes a foundation for subnanometer localization accuracy in widefield optical microscopy.

A Theoretical High-Density Nanoscopy Study Leads to the Design of UNLOC, a Parameter-free Algorithm

Single-molecule localization microscopy (SMLM) enables the production of high-resolution images by imaging spatially isolated fluorescent particles. Although challenging, the result of SMLM analysis lists the position of individual molecules, leading to a valuable quantification of the stoichiometry and spatial organization of molecular actors. Both the signal/noise ratio and the density (D_frame), i.e., the number of fluorescent particles per μm² per frame, have previously been identified as determining factors for reaching a given SMLM precision. Establishing a comprehensive theoretical study relying on these two parameters is therefore of central interest to delineate the achievable limits for accurate SMLM observations. Our study reports that in absence of prior knowledge of the signal intensity α, the density effect on particle localization is more prominent than that anticipated from theoretical studies performed at known α. A first limit appears when, under a low-density hypothesis (i.e., one-Gaussian fitting hypothesis), any fluorescent particle distant by less than ∼600 nm from the particle of interest biases its localization. In fact, all particles should be accounted for, even those dimly fluorescent, to ascertain unbiased localization of any surrounding particles. Moreover, even under a high-density hypothesis (i.e., multi-Gaussian fitting hypothesis), a second limit arises because of the impossible distinction of particles located too closely. An increase in D_frame is thus likely to deteriorate the localization precision, the image reconstruction, and more generally the quantification accuracy. Our study firstly provides a density-signal/noise ratio space diagram for use as a guide in data recording toward reaching an achievable SMLM resolution. Additionally, it leads to the identification of the essential requirements for implementing UNLOC, a parameter-free and fast computing algorithm approaching the Cramér-Rao bound for particles at high-density per frame and without any prior knowledge of their intensity. UNLOC is available as an ImageJ plugin.

Light sheet approaches for improved precision in 3D localization-based super-resolution imaging in mammalian cells [Invited]

The development of imaging techniques beyond the diffraction limit has paved the way for detailed studies of nanostructures and molecular mechanisms in biological systems. Imaging thicker samples, such as mammalian cells and tissue, in all three dimensions, is challenging due to increased background and volumes to image. Light sheet illumination is a method that allows for selective irradiation of the image plane, and its inherent optical sectioning capability allows for imaging of biological samples with reduced background, photobleaching, and photodamage. In this review, we discuss the advantage of combining single-molecule imaging with light sheet illumination. We begin by describing the principles of single-molecule localization microscopy and of light sheet illumination. Finally, we present examples of designs that successfully have married single-molecule super-resolution imaging with light sheet illumination for improved precision in mammalian cells.

Rapid, topology-based particle tracking for high-resolution measurements of large complex 3D motion fields

Spatiotemporal tracking of tracer particles or objects of interest can reveal localized behaviors in biological and physical systems. However, existing tracking algorithms are most effective for relatively low numbers of particles that undergo displacements smaller than their typical interparticle separation distance. Here, we demonstrate a single particle tracking algorithm to reconstruct large complex motion fields with large particle numbers, orders of magnitude larger than previously tractably resolvable, thus opening the door for attaining very high Nyquist spatial frequency motion recovery in the images. Our key innovations are feature vectors that encode nearest neighbor positions, a rigorous outlier removal scheme, and an iterative deformation warping scheme. We test this technique for its accuracy and computational efficacy using synthetically and experimentally generated 3D particle images, including non-affine deformation fields in soft materials, complex fluid flows, and cell-generated deformations. We augment this algorithm with additional particle information (e.g., color, size, or shape) to further enhance tracking accuracy for high gradient and large displacement fields. These applications demonstrate that this versatile technique can rapidly track unprecedented numbers of particles to resolve large and complex motion fields in 2D and 3D images, particularly when spatial correlations exist.

Enhancing optical microscopy illumination to enable quantitative imaging

There has been an increasing push to derive quantitative measurements using optical microscopes. While several aspects of microscopy have been identified to enhance quantitative imaging, non-uniform angular illumination asymmetry (ANILAS) across the field-of-view is an important factor that has been largely overlooked. Non-uniform ANILAS results in loss of imaging precision and can lead to, for example, less reliability in medical diagnoses. We use ANILAS maps to demonstrate that objective lens design, illumination wavelength and location of the aperture diaphragm are significant factors that contribute to illumination aberrations. To extract the best performance from an optical microscope, the combination of all these factors must be optimized for each objective lens. This requires the capability to optimally align the aperture diaphragm in the axial direction. Such optimization enhances the quantitative imaging accuracy of optical microscopes and can benefit applications in important areas such as biotechnology, optical metrology, and nanotechnology.

TIRF-Based Single-Molecule Detection of the RecA Presynaptic Filament Dynamics

RecA is a key protein in homologous DNA repair process. On a single-stranded (ss) DNA, which appears as an intermediate structure at a double-strand break site, RecA forms a kilobase-long presynaptic filament that mediates homology search and strand exchange reaction. RecA requires adenosine triphosphate as a cofactor that confers dynamic features to the filament such as nucleation, end-dependent growth and disassembly, scaffold shift along the ssDNA, and conformational change. Due to the complexity of the dynamics, detailed molecular mechanisms of functioning presynaptic filament have been characterized only recently after the advent of single-molecule techniques that allowed real-time observation of each kinetic process. In this chapter, single-molecule fluorescence resonance energy transfer assays, which revealed detailed molecular pictures of the presynaptic filament dynamics, will be discussed.

An Application-Directed, Versatile DNA FISH Platform for Research and Diagnostics

DNA fluorescence in situ hybridization (DNA FISH) has emerged as a powerful microscopy technique that allows a unique view into the composition and arrangement of the genetic material in its natural context-be it the cell nucleus in interphase, or chromosomes in metaphase spreads. The core principle of DNA FISH is the ability of fluorescently labeled DNA probes (either double- or single-stranded DNA fragments) to bind to their complementary sequences in situ in cells or tissues, revealing the location of their target as fluorescence signals detectable with a fluorescence microscope. Numerous variants and improvements of the original DNA FISH method as well as a vast repertoire of applications have been described since its inception more than 4 decades ago. In recent years, the development of many new fluorescent dyes together with drastic advancements in methods for probe generation (Boyle et al., Chromosome Res 19:901-909, 2011; Beliveau et al., Proc Natl Acad Sci U S A 109:21301-21306, 2012; Bienko et al., Nat Methods 10:122-124, 2012), as well as improvements in the resolution of microscopy technologies, have boosted the number of DNA FISH applications, particularly in the field of genome architecture (Markaki et al., Bioessays 34:412-426, 2012; Beliveau et al., Nat Commun 6:7147, 2015). However, despite these remarkable steps forward, choosing which type of DNA FISH sample preparation protocol, probe design, hybridization procedure, and detection method is best suited for a given application remains still challenging for many research labs, preventing a more widespread use of this powerful technology. Here, we present a comprehensive platform to help researchers choose which DNA FISH protocol is most suitable for their particular application. In addition, we describe computational pipelines that can be implemented for efficient DNA FISH probe design and for signal quantification. Our goal is to make DNA FISH a versatile and streamlined technique that can be easily implemented by both research and diagnostic labs.

Observation of live chromatin dynamics in cells via 3D localization microscopy using Tetrapod point spread functions

We report the observation of chromatin dynamics in living budding yeast (Saccharomyces cerevisiae) cells, in three-dimensions (3D). Using dual color localization microscopy and employing a Tetrapod point spread function, we analyze the spatio-temporal dynamics of two fluorescently labeled DNA loci surrounding the GAL locus. From the measured trajectories, we obtain different dynamical characteristics in terms of inter-loci distance and temporal variance; when the GAL locus is activated, the 3D inter-loci distance and temporal variance increase compared to the inactive state. These changes are visible in spite of the large thermally- and biologically-driven heterogeneity in the relative motion of the two loci. Our observations are consistent with current euchromatin vs. heterochromatin models.

Quantitative performance evaluation of a back-illuminated sCMOS camera with 95% QE for super-resolution localization microscopy

Scientific Complementary Metal Oxide Semiconductor (sCMOS) cameras were introduced into the market in 2009 and are now becoming a major type of commercial cameras for low-light imaging. sCMOS cameras provide simultaneously low read noise, high readout speed, and large pixel array; however, the relatively low quantum efficiency (QE) of sCMOS cameras has been a major limitation for its application in single molecule imaging, especially super-resolution localization microscopy which requires high detection sensitivity. Here we report the imaging performance of a newly released back-illuminated sCMOS camera (called Dhyana 95 from Tucsen) which is claimed to be the world's first 95% QE sCMOS camera. The imaging performance evaluation is based on a new methodology which is designed to provide paired images from two tested cameras under almost identical experimental conditions. We verified that this new 95% QE sCMOS camera is able to provide superior imaging performance over a representative front-illuminated sCMOS camera (Hamamatsu Flash 4.0 V2) and a popular back-illuminated EMCCD camera (Andor iXon 897 Ultra) in a wide signal range. We hope this study will inspire more studies on using sCMOS cameras in super-resolution localization microscopy, or even single molecule imaging. © 2017 International Society for Advancement of Cytometry.

Gpufit: An open-source toolkit for GPU-accelerated curve fitting

We present a general purpose, open-source software library for estimation of non-linear parameters by the Levenberg-Marquardt algorithm. The software, Gpufit, runs on a Graphics Processing Unit (GPU) and executes computations in parallel, resulting in a significant gain in performance. We measured a speed increase of up to 42 times when comparing Gpufit with an identical CPU-based algorithm, with no loss of precision or accuracy. Gpufit is designed such that it is easily incorporated into existing applications or adapted for new ones. Multiple software interfaces, including to C, Python, and Matlab, ensure that Gpufit is accessible from most programming environments. The full source code is published as an open source software repository, making its function transparent to the user and facilitating future improvements and extensions. As a demonstration, we used Gpufit to accelerate an existing scientific image analysis package, yielding significantly improved processing times for super-resolution fluorescence microscopy datasets.

Unraveling the Thousand Word Picture: An Introduction to Super-Resolution Data Analysis

Super-resolution microscopy provides direct insight into fundamental biological processes occurring at length scales smaller than light's diffraction limit. The analysis of data at such scales has brought statistical and machine learning methods into the mainstream. Here we provide a survey of data analysis methods starting from an overview of basic statistical techniques underlying the analysis of super-resolution and, more broadly, imaging data. We subsequently break down the analysis of super-resolution data into four problems: the localization problem, the counting problem, the linking problem, and what we've termed the interpretation problem.

Three-Dimensional Localization of Single Molecules for Super-Resolution Imaging and Single-Particle Tracking

Single-molecule super-resolution fluorescence microscopy and single-particle tracking are two imaging modalities that illuminate the properties of cells and materials on spatial scales down to tens of nanometers or with dynamical information about nanoscale particle motion in the millisecond range, respectively. These methods generally use wide-field microscopes and two-dimensional camera detectors to localize molecules to much higher precision than the diffraction limit. Given the limited total photons available from each single-molecule label, both modalities require careful mathematical analysis and image processing. Much more information can be obtained about the system under study by extending to three-dimensional (3D) single-molecule localization: without this capability, visualization of structures or motions extending in the axial direction can easily be missed or confused, compromising scientific understanding. A variety of methods for obtaining both 3D super-resolution images and 3D tracking information have been devised, each with their own strengths and weaknesses. These include imaging of multiple focal planes, point-spread-function engineering, and interferometric detection. These methods may be compared based on their ability to provide accurate and precise position information on single-molecule emitters with limited photons. To successfully apply and further develop these methods, it is essential to consider many practical concerns, including the effects of optical aberrations, field dependence in the imaging system, fluorophore labeling density, and registration between different color channels. Selected examples of 3D super-resolution imaging and tracking are described for illustration from a variety of biological contexts and with a variety of methods, demonstrating the power of 3D localization for understanding complex systems.

Single Particle Tracking: From Theory to Biophysical Applications

After three decades of developments, single particle tracking (SPT) has become a powerful tool to interrogate dynamics in a range of materials including live cells and novel catalytic supports because of its ability to reveal dynamics in the structure-function relationships underlying the heterogeneous nature of such systems. In this review, we summarize the algorithms behind, and practical applications of, SPT. We first cover the theoretical background including particle identification, localization, and trajectory reconstruction. General instrumentation and recent developments to achieve two- and three-dimensional subdiffraction localization and SPT are discussed. We then highlight some applications of SPT to study various biological and synthetic materials systems. Finally, we provide our perspective regarding several directions for future advancements in the theory and application of SPT.

State space approach to single molecule localization in fluorescence microscopy

Single molecule super-resolution microscopy enables imaging at sub-diffraction-limit resolution by producing images of subsets of stochastically photoactivated fluorophores over a sequence of frames. In each frame of the sequence, the fluorophores are accurately localized, and the estimated locations are used to construct a high-resolution image of the cellular structures labeled by the fluorophores. Many methods have been developed for localizing fluorophores from the images. The majority of these methods comprise two separate steps: detection and estimation. In the detection step, fluorophores are identified. In the estimation step, the locations of the identified fluorophores are estimated through an iterative approach. Here, we propose a non-iterative state space-based localization method which combines the detection and estimation steps. We demonstrate that the estimated locations obtained from the proposed method can be used as initial conditions in an estimation routine to potentially obtain improved location estimates. The proposed method models the given image as the frequency response of a multi-order system obtained with a balanced state space realization algorithm based on the singular value decomposition of a Hankel matrix. The locations of the poles of the resulting system determine the peak locations in the frequency domain, and the locations of the most significant peaks correspond to the single molecule locations in the original image. The performance of the method is validated using both simulated and experimental data.

Intensity-based axial localization approaches for multifocal plane microscopy

Multifocal plane microscopy (MUM) can be used to visualize biological samples in three dimensions over large axial depths and provides for the high axial localization accuracy that is needed in applications such as the three-dimensional tracking of single particles and super-resolution microscopy. This report analyzes the performance of intensity-based axial localization approaches as applied to MUM data using Fisher information calculations. In addition, a new non-parametric intensity-based axial location estimation method, Multi-Intensity Lookup Algorithm (MILA), is introduced that, unlike typical intensity-based methods that make use of a single intensity value per data image, utilizes multiple intensity values per data image in determining the axial location of a point source. MILA is shown to be robust against potential bias induced by differences in the sub-pixel location of the imaged point source. The method's effectiveness on experimental data is also evaluated.

Automated tracking of colloidal clusters with sub-pixel accuracy and precision

Quantitative tracking of features from video images is a basic technique employed in many areas of science. Here, we present a method for the tracking of features that partially overlap, in order to be able to track so-called colloidal molecules. Our approach implements two improvements into existing particle tracking algorithms. Firstly, we use the history of previously identified feature locations to successfully find their positions in consecutive frames. Secondly, we present a framework for non-linear least-squares fitting to summed radial model functions and analyze the accuracy (bias) and precision (random error) of the method on artificial data. We find that our tracking algorithm correctly identifies overlapping features with an accuracy below 0.2% of the feature radius and a precision of 0.1 to 0.01 pixels for a typical image of a colloidal cluster. Finally, we use our method to extract the three-dimensional diffusion tensor from the Brownian motion of colloidal dimers.

A state space based approach to localizing single molecules from multi-emitter images

Single molecule super-resolution microscopy is a powerful tool that enables imaging at sub-diffraction-limit resolution. In this technique, subsets of stochastically photoactivated fluorophores are imaged over a sequence of frames and accurately localized, and the estimated locations are used to construct a high-resolution image of the cellular structures labeled by the fluorophores. Available localization methods typically first determine the regions of the image that contain emitting fluorophores through a process referred to as detection. Then, the locations of the fluorophores are estimated accurately in an estimation step. We propose a novel localization method which combines the detection and estimation steps. The method models the given image as the frequency response of a multi-order system obtained with a balanced state space realization algorithm based on the singular value decomposition of a Hankel matrix, and determines the locations of intensity peaks in the image as the pole locations of the resulting system. The locations of the most significant peaks correspond to the locations of single molecules in the original image. Although the accuracy of the location estimates is reasonably good, we demonstrate that, by using the estimates as the initial conditions for a maximum likelihood estimator, refined estimates can be obtained that have a standard deviation close to the Cramér-Rao lower bound-based limit of accuracy. We validate our method using both simulated and experimental multi-emitter images.

Super-resolution Analysis of TCR-Dependent Signaling: Single-Molecule Localization Microscopy

Single-molecule localization microscopy (SMLM) comprises methods that produce super-resolution images from molecular locations of single molecules. These techniques mathematically determine the center of a diffraction-limited spot produced by a fluorescent molecule, which represents the most likely location of the molecule. Only a small cohort of well-separated molecules is visualized in a single image, and then many images are obtained from a single sample. The localizations from all the images are combined to produce a super-resolution picture of the sample. Here we describe the application of two methods, photoactivation localization microscopy (PALM) and direct stochastic optical reconstruction microscopy (dSTORM), to the study of signaling microclusters in T cells.

Multifluorophore localization as a percolation problem: limits to density and precision

We show that the maximum desirable density of activated fluorophores in a superresolution experiment can be determined by treating the overlapping point spread functions as a problem in percolation theory. We derive a bound on the density of activated fluorophores, taking into account the desired localization accuracy and precision, as well as the number of photons emitted. Our bound on density is close to that reported in experimental work, suggesting that further increases in the density of imaged fluorophores will come at the expense of localization accuracy and precision.

Fisher information theory for parameter estimation in single molecule microscopy: tutorial

Estimation of a parameter of interest from image data represents a task that is commonly carried out in single molecule microscopy data analysis. The determination of the positional coordinates of a molecule from its image, for example, forms the basis of standard applications such as single molecule tracking and localization-based super-resolution image reconstruction. Assuming that the estimator used recovers, on average, the true value of the parameter, its accuracy, or standard deviation, is then at best equal to the square root of the Cramér-Rao lower bound. The Cramér-Rao lower bound can therefore be used as a benchmark in the evaluation of the accuracy of an estimator. Additionally, as its value can be computed and assessed for different experimental settings, it is useful as an experimental design tool. This tutorial demonstrates a mathematical framework that has been specifically developed to calculate the Cramér-Rao lower bound for estimation problems in single molecule microscopy and, more broadly, fluorescence microscopy. The material includes a presentation of the photon detection process that underlies all image data, various image data models that describe images acquired with different detector types, and Fisher information expressions that are necessary for the calculation of the lower bound. Throughout the tutorial, examples involving concrete estimation problems are used to illustrate the effects of various factors on the accuracy of parameter estimation and, more generally, to demonstrate the flexibility of the mathematical framework.

Quantitative Analysis of Intracellular Fluorescent Foci in Live Bacteria

Fluorescence microscopy has revolutionized in vivo cellular biology. Through the specific labeling of a protein of interest with a fluorescent protein, one is able to study movement and colocalization, and even count individual proteins in a live cell. Different algorithms exist to quantify the total intensity and position of a fluorescent focus. Although these algorithms have been rigorously studied for in vitro conditions, which are greatly different than the in-homogenous and variable cellular environments, their exact limits and applicability in the context of a live cell have not been thoroughly and systematically evaluated. In this study, we quantitatively characterize the influence of different background subtraction algorithms on several focus analysis algorithms. We use, to our knowledge, a novel approach to assess the sensitivity of the focus analysis algorithms to background removal, in which simulated and experimental data are combined to maintain full control over the sensitivity of a focus within a realistic background of cellular fluorescence. We demonstrate that the choice of algorithm and the corresponding error are dependent on both the brightness of the focus, and the cellular context. Expectedly, focus intensity estimation and localization accuracy suffer in all algorithms at low focus to background ratios, with the bacteroidal background subtraction in combination with the median excess algorithm, and the region of interest background subtraction in combination with a two-dimensional Gaussian fit algorithm, performing the best. We furthermore show that the choice of background subtraction algorithm is dependent on the expression level of the protein under investigation, and that the localization error is dependent on the distance of a focus from the bacterial edge and pole. Our results establish a set of guidelines for what signals can be analyzed to give a targeted spatial and intensity accuracy within a bacterial cell.

Deciphering the Structure and Function of Nuclear Pores Using Single-Molecule Fluorescence Approaches

Due to its central role in macromolecular trafficking and nucleocytoplasmic information transfer, the nuclear pore complex (NPC) has been studied in great detail using a wide spectrum of methods. Consequently, many aspects of its architecture, general function, and role in the life cycle of a cell are well understood. Over the last decade, fluorescence microscopy methods have enabled the real-time visualization of single molecules interacting with and transiting through the NPC, allowing novel questions to be examined with nanometer precision. While initial single-molecule studies focused primarily on import pathways using permeabilized cells, it has recently proven feasible to investigate the export of mRNAs in living cells. Single-molecule assays can address questions that are difficult or impossible to answer by other means, yet the complexity of nucleocytoplasmic transport requires that interpretation be based on a firm genetic, biochemical, and structural foundation. Moreover, conceptually simple single-molecule experiments remain technically challenging, particularly with regard to signal intensity, signal-to-noise ratio, and the analysis of noise, stochasticity, and precision. We discuss nuclear transport issues recently addressed by single-molecule microscopy, evaluate the limits of existing assays and data, and identify open questions for future studies. We expect that single-molecule fluorescence approaches will continue to be applied to outstanding nucleocytoplasmic transport questions, and that the approaches developed for NPC studies are extendable to additional complex systems and pathways within cells.

Investigating the usage of point spread functions in point source and microsphere localization

Using a point spread function (PSF) to localize a point-like object, such as a fluorescent molecule or microsphere, represents a common task in single molecule microscopy image data analysis. The localization may differ in purpose depending on the application or experiment, but a unifying theme is the importance of being able to closely recover the true location of the point-like object with high accuracy. We present two simulation studies, both relating to the performance of object localization via the maximum likelihood fitting of a PSF to the object's image. In the first study, we investigate the integration of the PSF over an image pixel, which represents a critical part of the localization algorithm. Specifically, we explore how the fineness of the integration affects how well a point source can be localized, and find the use of too coarse a step size to produce location estimates that are far from the true location, especially when the images are acquired at relatively low magnifications. We also propose a method for selecting an appropriate step size. In the second study, we investigate the suitability of the common practice of using a PSF to localize a microsphere, despite the mismatch between the microsphere's image and the fitted PSF. Using criteria based on the standard errors of the mean and variance, we find the method suitable for microspheres up to 1 μm and 100 nm in diameter, when the localization is performed, respectively, with and without the simultaneous estimation of the width of the PSF.

A review of progress in single particle tracking: from methods to biophysical insights

Optical microscopy has for centuries been a key tool to study living cells with minimum invasiveness. The advent of single molecule techniques over the past two decades has revolutionized the field of cell biology by providing a more quantitative picture of the complex and highly dynamic organization of living systems. Amongst these techniques, single particle tracking (SPT) has emerged as a powerful approach to study a variety of dynamic processes in life sciences. SPT provides access to single molecule behavior in the natural context of living cells, thereby allowing a complete statistical characterization of the system under study. In this review we describe the foundations of SPT together with novel optical implementations that nowadays allow the investigation of single molecule dynamic events with increasingly high spatiotemporal resolution using molecular densities closer to physiological expression levels. We outline some of the algorithms for the faithful reconstruction of SPT trajectories as well as data analysis, and highlight biological examples where the technique has provided novel insights into the role of diffusion regulating cellular function. The last part of the review concentrates on different theoretical models that describe anomalous transport behavior and ergodicity breaking observed from SPT studies in living cells.