Structured illumination microscopy artefacts caused by illumination scattering

Despite its wide application in live-cell super-resolution (SR) imaging, structured illumination microscopy (SIM) suffers from aberrations caused by various sources. Although artefacts generated from inaccurate reconstruction parameter estimation and noise amplification can be minimized, aberrations due to the scattering of excitation light on samples have rarely been investigated. In this paper, by simulating multiple subcellular structure with the distinct refractive index from water, we study how different thicknesses of this subcellular structure scatter incident light on its optical path of SIM excitation. Because aberrant interference light aggravates with the increase in sample thickness, the reconstruction of the 2D-SIM SR image degraded with the change of focus along the axial axis. Therefore, this work may guide the future development of algorithms to suppress SIM artefacts caused by scattering in thick samples. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 1)'.

Structured illumination microscopy and image scanning microscopy: a review and comparison of imaging properties

Structured illumination microscopy and image scanning microscopy are two microscopical tech- niques, rapidly increasing in practical application, that can result in improvement in transverse spatial resolution, and/or improvement in axial imaging performance. The history and principles of these techniques are reviewed, and the imaging properties of the two methods compared. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 1)'.

Improving axial resolution in Structured Illumination Microscopy using deep learning

Structured Illumination Microscopy (SIM) is a widespread methodology to image live and fixed biological structures smaller than the diffraction limits of conventional optical microscopy. Using recent advances in image up-scaling through deep learning models, we demonstrate a method to reconstruct 3D SIM image stacks with twice the axial resolution attainable through conventional SIM reconstructions. We further demonstrate our method is robust to noise and evaluate it against two-point cases and axial gratings. Finally, we discuss potential adaptions of the method to further improve resolution. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 1)'.

Answers to fundamental questions in superresolution microscopy

This article presents answers to the questions on superresolution and structured illumination microscopy (SIM) as raised in the editorial of this collection of articles (https://doi.org/10.1098/rsta.2020.0143). These answers are based on my personal views on superresolution in light microscopy, supported by reasoning. Discussed are the definition of superresolution, Abbe's resolution limit and the classification of superresolution methods into nonlinear-, prior knowledge- and near-field-based superresolution. A further focus is put on the capabilities and technical aspects of present and future SIM methods. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 1)'.

Extended mechanical force measurements using structured illumination microscopy

Quantifying cell generated mechanical forces is key to furthering our understanding of mechanobiology. Traction force microscopy (TFM) is one of the most broadly applied force probing technologies, but its sensitivity is strictly dependent on the spatio-temporal resolution of the underlying imaging system. In previous works, it was demonstrated that increased sampling densities of cell derived forces permitted by super-resolution fluorescence imaging enhanced the sensitivity of the TFM method. However, these recent advances to TFM based on super-resolution techniques were limited to slow acquisition speeds and high illumination powers. Here, we present three novel TFM approaches that, in combination with total internal reflection, structured illumination microscopy and astigmatism, improve the spatial and temporal performance in either two-dimensional or three-dimensional mechanical force quantification, while maintaining low illumination powers. These three techniques can be straightforwardly implemented on a single optical set-up offering a powerful platform to provide new insights into the physiological force generation in a wide range of biological studies. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 1)'.

mmSIM: an open toolbox for accessible structured illumination microscopy

Since the first practical super-resolution structured illumination fluorescence microscopes (SIM) were demonstrated more than two decades ago, the method has become increasingly popular for a wide range of bioimaging applications. The high cost and relative inflexibility of commercial systems, coupled with the conceptual simplicity of the approach and the desire to exploit and customize existing hardware, have led to the development of a large number of home-built systems. Several detailed hardware designs are available in the scientific literature, complemented by open-source software tools for SIM image validation and reconstruction. However, there remains a lack of simple open-source software to control these systems and manage the synchronization between hardware components, which is critical for effective SIM imaging. This article describes a new suite of software tools based on the popular Micro-Manager package, which enable the keen microscopist to develop and run a SIM system. We use the software to control two custom-built, high-speed, spatial light modulator-based SIM systems, evaluating their performance by imaging a range of fluorescent samples. By simplifying the process of SIM hardware development, we aim to support wider adoption of the technique. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 1)'.

Biobeam-Multiplexed wave-optical simulations of light-sheet microscopy

Sample-induced image-degradation remains an intricate wave-optical problem in light-sheet microscopy. Here we present biobeam, an open-source software package that enables simulation of operational light-sheet microscopes by combining data from 10⁵–10⁶ multiplexed and GPU-accelerated point-spread-function calculations. The wave-optical nature of these simulations leads to the faithful reproduction of spatially varying aberrations, diffraction artifacts, geometric image distortions, adaptive optics, and emergent wave-optical phenomena, and renders image-formation in light-sheet microscopy computationally tractable.

Modern microscopes permit to acquire high quality images of large fields of view, which is the result of a decade-long development of computer aided optical design. However, this high image quality can only be obtained at the very surface of biological specimens: when trying to penetrate deeper into biological tissues, light scattering by cells rapidly leads to severe image blur and computers have so far been unable to model the process by which light forms images in such turbid optical environments. We developed a software that allows one to simulate how microscopes record images deep inside scattering biological samples. Our software reproduces a wide range of optical effects that underlie image blur in tissues. Hence strategies to improve image quality within three-dimensional samples can now be systematically tested by computers. Specifically, our software reproduces intricate wave-optical effects that have recently been proposed as strategies to gain perfect images even in the most turbid environments.This provides the chance for a new generation of microscopes, in which computer models guide the imaging process to enable highest possible resolution even deep inside biological specimens.

Sensitive detection of fluorescence in western blotting by merging images

The western blotting technique is widely used to analyze protein expression levels and protein molecular weight. The chemiluminescence method is mainly used for detection due to its high sensitivity and ease of manipulation, but it is unsuitable for detailed analyses because it cannot be used to detect multiple proteins simultaneously. Recently, more attention has been paid to the fluorescence detection method because it is more quantitative and is suitable for the detection of multiple proteins simultaneously. However, fluorescence detection can be limited by poor image resolution and low detection sensitivity. Here, we describe a method to detect fluorescence in western blots using fluorescence microscopy to obtain high-resolution images. In this method, filters and fluorescent dyes are optimized to enhance detection sensitivity to a level similar to that of the chemiluminescence method.

Analyzing optical imaging of Ca2+ signals via TIRF microscopy: The limits on resolution due to chemical rates and depth of the channels

High resolution total internal reflection (TIRF) microscopy (TIRFM) together with detailed computational modeling provides a powerful approach towards the understanding of a wide range of Ca²⁺ signals mediated by the ubiquitous inositol 1,4,5-trisphosphate (IP₃) receptor (IP₃R) channel. Exploiting this fruitful collaboration further requires close agreement between the models and observations. However, elementary Ca²⁺ release events, puffs, imaged through TIRFM do not show the rapid single-channel openings and closings during and between puffs as are present in simulated puffs using data-driven single channel models. TIRFM also shows a rapid equilibration of 10ms after a channel opens or closes which is not achievable in simulation using standard Ca²⁺ diffusion coefficients and reaction rates between indicator dye and Ca²⁺. Furthermore, TIRFM imaging cannot decipher the depth of the channel with respect to the microscope, which will affect the change in fluorescence that the microscope detects, thereby affecting its sensitivity to fast single-channel activity. Using the widely used Ca²⁺ diffusion coefficients and reaction rates, our simulations show equilibration rates that are eight times slower than TIRFM imaging. We show that to get equilibrium rates consistent with observed values, the diffusion coefficients and reaction rates have to be significantly higher than the values reported in the literature, and predict the channel depth to be 200-250nm. Finally, we show that with the addition of noise, short events due to 1-2ms opening and closing of channels that are observed in computational models can be missed in TIRFM.

Implementation of fluorescence confocal mosaicking microscopy by "early adopter" Mohs surgeons and dermatologists: recent progress

Confocal mosaicking microscopy (CMM) enables rapid imaging of large areas of fresh tissue ex vivo without the processing that is necessary for conventional histology. When performed in fluorescence mode using acridine orange (nuclear specific dye), it enhances nuclei-to-dermis contrast that enables detection of all types of basal cell carcinomas (BCCs), including micronodular and thin strands of infiltrative types. So far, this technique has been mostly validated in research settings for the detection of residual BCC tumor margins with high sensitivity of 89% to 96% and specificity of 99% to 89%. Recently, CMM has advanced to implementation and testing in clinical settings by “early adopter” Mohs surgeons, as an adjunct to frozen section during Mohs surgery. We summarize the development of CMM guided imaging of ex vivo skin tissues from bench to bedside. We also present its current state of application in routine clinical workflow not only for the assessment of residual BCC margins in the Mohs surgical setting but also for some melanocytic lesions and other skin conditions in clinical dermatology settings. Last, we also discuss the potential limitations of this technology as well as future developments. As this technology advances further, it may serve as an adjunct to standard histology and enable rapid surgical pathology of skin cancers at the bedside.

Accurate Detection of Dysmorphic Nuclei Using Dynamic Programming and Supervised Classification

A vast array of pathologies is typified by the presence of nuclei with an abnormal morphology. Dysmorphic nuclear phenotypes feature dramatic size changes or foldings, but also entail much subtler deviations such as nuclear protrusions called blebs. Due to their unpredictable size, shape and intensity, dysmorphic nuclei are often not accurately detected in standard image analysis routines. To enable accurate detection of dysmorphic nuclei in confocal and widefield fluorescence microscopy images, we have developed an automated segmentation algorithm, called Blebbed Nuclei Detector (BleND), which relies on two-pass thresholding for initial nuclear contour detection, and an optimal path finding algorithm, based on dynamic programming, for refining these contours. Using a robust error metric, we show that our method matches manual segmentation in terms of precision and outperforms state-of-the-art nuclear segmentation methods. Its high performance allowed for building and integrating a robust classifier that recognizes dysmorphic nuclei with an accuracy above 95%. The combined segmentation-classification routine is bound to facilitate nucleus-based diagnostics and enable real-time recognition of dysmorphic nuclei in intelligent microscopy workflows.

Model-independent counting of molecules in single-molecule localization microscopy

Most biomolecular processes rely on tightly controlled stoichiometries, from the formation of molecular assemblies to cellular signaling. Single-molecule localization micro-scopy studies of fluorophore blinking offer a promising route to probe oligomeric states. Here we show that the distribution of the number of blinking events assumes a universal functional form, independent of photophysics, under relatively mild assumptions. The number of photophysical states, the kinetics of interconversion, and the fraction of active fluorophores enter as two or three constants. This essentially model-independent formulation allows us to determine molecule counts from fluorophore blinking statistics. The formulas hold even if the fluorophores have many different yet unresolved dark states, as long as there is only a single fluorescent state, or if there are different yet unresolvable fluorescent states, as long as there is only a single dark state. We demonstrate the practical applicability of this approach by quantifying the oligomerization states of membrane proteins tagged with the mEos2 fluorescent protein. We find that the model parameters, obtained by likelihood maximization, are transferable. With the counting statistics being independent of the detailed photophysics and its parameters being transferable, the method should be robust and broadly applicable to counting colocalized molecules in vivo and in vitro.

Comparison of a Visual to a Computer Assisted Technique for Detecting Apoptosis

In many studies, fluorescent dyes (ethidium bromide [EB] and acridine orange [AO]) are used to stain DNA to determine if nuclei are apoptotic. However, there are numerous visual methods for counting these stained DNA that may lead to inaccuracies Measuring apoptosis by the visual counting method may be imprecise because of the variability of individuals’ perception of color. Therefore, the authors compared a visual method of counting chromatin for apoptosis with a method relying on a computer program. They began counting chromatin using the visual method, in which individuals identify the stained DNA using their own visual perception. For comparison, they used a software-based counting method (analySIS software) to determine the color (hue) of the stained DNA. Using the numeric hue values from the software eliminates the variations in human color perception. Intra and interrater reliability of the visual and computerassisted counting methods were evaluated with Spearman’s. The authors found statistical significance in the intrarater reliability (r = 1.0,P = 0.0001 for all chromatin categories) and interrater reliability (r = 0.975,P = 0.005 for both readings) when using the software program. No statistical significance was found for the visual counting method, indicating inaccuracy between and within raters. Thus, the computerassisted counting method of identifying the damaged DNA is more accurate and precise than the individual’s visual perception of color. Based on these data, apoptosis measurements using color staining with EB and AO should be determined using hue values generated by a computer program and not by a researcher’s visual assessment.

Building cell models and simulations from microscope images

The use of fluorescence microscopy has undergone a major revolution over the past twenty years, both with the development of dramatic new technologies and with the widespread adoption of image analysis and machine learning methods. Many open source software tools provide the ability to use these methods in a wide range of studies, and many molecular and cellular phenotypes can now be automatically distinguished. This article presents the next major challenge in microscopy automation, the creation of accurate models of cell organization directly from images, and reviews the progress that has been made towards this challenge.

Maximizing the biochemical resolving power of fluorescence microscopy

Most recent advances in fluorescence microscopy have focused on achieving spatial resolutions below the diffraction limit. However, the inherent capability of fluorescence microscopy to non-invasively resolve different biochemical or physical environments in biological samples has not yet been formally described, because an adequate and general theoretical framework is lacking. Here, we develop a mathematical characterization of the biochemical resolution in fluorescence detection with Fisher information analysis. To improve the precision and the resolution of quantitative imaging methods, we demonstrate strategies for the optimization of fluorescence lifetime, fluorescence anisotropy and hyperspectral detection, as well as different multi-dimensional techniques. We describe optimized imaging protocols, provide optimization algorithms and describe precision and resolving power in biochemical imaging thanks to the analysis of the general properties of Fisher information in fluorescence detection. These strategies enable the optimal use of the information content available within the limited photon-budget typically available in fluorescence microscopy. This theoretical foundation leads to a generalized strategy for the optimization of multi-dimensional optical detection, and demonstrates how the parallel detection of all properties of fluorescence can maximize the biochemical resolving power of fluorescence microscopy, an approach we term Hyper Dimensional Imaging Microscopy (HDIM). Our work provides a theoretical framework for the description of the biochemical resolution in fluorescence microscopy, irrespective of spatial resolution, and for the development of a new class of microscopes that exploit multi-parametric detection systems.

Quantitative Förster resonance energy transfer efficiency measurements using simultaneous spectral unmixing of excitation and emission spectra

Accurate quantification of Förster resonance energy transfer (FRET) using intensity-based methods is difficult due to the overlap of fluorophore excitation and emission spectra. Consequently, mechanisms are required to remove bleedthrough of the donor emission into the acceptor channel and direct excitation of the acceptor when aiming to excite only the donor fluorophores. Methods to circumvent donor bleedthrough using the unmixing of emission spectra have been reported, but these require additional corrections to account for direct excitation of the acceptor. Here we present an alternative method for robust quantification of FRET efficiencies based upon the simultaneous spectral unmixing of both excitation and emission spectra. This has the benefit over existing methodologies in circumventing the issue of donor bleedthrough and acceptor cross excitation without the need for additional corrections. Furthermore, we show that it is applicable with as few as two excitation wavelengths and so can be used for quantifying FRET efficiency in microscope images as easily as for data collected on a spectrofluorometer. We demonstrate the accuracy of the approach by reproducing efficiency values in well characterized FRET standards: HEK cells expressing a variety of linked cerulean and venus fluorescent proteins. Finally we describe simple ImageJ plugins that can be used to calculate and create images of FRET efficiencies from microscope images.

Automated, portable, low-cost bright-field and fluorescence microscope with autofocus and autoscanning capabilities

Optical microscopy is a simple, yet essential, imaging technology. Conventional laboratory-grade optical microscopes are bulky and costly, confining their use to within laboratory settings and restricting their accessibility in regions of limited resources. With the aim of overcoming these limitations, we have realized a portable, low-cost, and highly automated optical microscope that integrates mass-manufactured components, including light-emitting diodes, a web camera, optical disk drives, and a microcontroller. Our implementation is capable of bright-field and fluorescence imaging with micrometer-scale resolution and controlled mechanical actuation of both the lens and sample. We interface the lighting, image capture, and mechanical actuators of the microscope into a single software environment, enabling automation of common microscope operations, such as image focusing and large-area sample visualization. Combination of mechanical actuation and software automation into a compact, low-cost microscope system is an important initial step toward the goal of making optical microscopy universally accessible, portable, and easy to use.

Interactive, graph-based visual analysis of high-dimensional, multi-parameter fluorescence microscopy data in toponomics

In Toponomics, the function protein pattern in cells or tissue (the toponome) is imaged and analyzed for applications in toxicology, new drug development and patient-drug-interaction. The most advanced imaging technique is robot-driven multi-parameter fluorescence microscopy. This technique is capable of co-mapping hundreds of proteins and their distribution and assembly in protein clusters across a cell or tissue sample by running cycles of fluorescence tagging with monoclonal antibodies or other affinity reagents, imaging, and bleaching in situ. The imaging results in complex multi-parameter data composed of one slice or a 3D volume per affinity reagent. Biologists are particularly interested in the localization of co-occurring proteins, the frequency of co-occurrence and the distribution of co-occurring proteins across the cell. We present an interactive visual analysis approach for the evaluation of multi-parameter fluorescence microscopy data in toponomics. Multiple, linked views facilitate the definition of features by brushing multiple dimensions. The feature specification result is linked to all views establishing a focus+context visualization in 3D. In a new attribute view, we integrate techniques from graph visualization. Each node in the graph represents an affinity reagent while each edge represents two co-occurring affinity reagent bindings. The graph visualization is enhanced by glyphs which encode specific properties of the binding. The graph view is equipped with brushing facilities. By brushing in the spatial and attribute domain, the biologist achieves a better understanding of the function protein patterns of a cell. Furthermore, an interactive table view is integrated which summarizes unique fluorescence patterns. We discuss our approach with respect to a cell probe containing lymphocytes and a prostate tissue section.

High-density localization of active molecules using Structured Sparse Model and Bayesian Information Criterion

Localization-based super-resolution microscopy (or called localization microscopy) rely on repeated imaging and localization of active molecules, and the spatial resolution enhancement of localization microscopy is built upon the sacrifice of its temporal resolution. Developing algorithms for high-density localization of active molecules is a promising approach to increase the speed of localization microscopy. Here we present a new algorithm called SSM_BIC for such purpose. The SSM_BIC combines the advantages of the Structured Sparse Model (SSM) and the Bayesian Information Criterion (BIC). Through simulation and experimental studies, we evaluate systematically the performance between the SSM_BIC and the conventional Sparse algorithm in high-density localization of active molecules. We show that the SSM_BIC is superior in processing single molecule images with weak signal embedded in strong background.

Depth-resolved image mapping spectrometer (IMS) with structured illumination

We present a depth-resolved Image Mapping Spectrometer (IMS) which is capable of acquiring 4D (x, y, z, λ) datacubes. Optical sectioning is implemented by structured illumination. The device's spectral imaging performance is demonstrated in a multispectral microsphere and mouse kidney tissue fluorescence imaging experiment. We also compare quantitatively the depth-resolved IMS with a hyperspectral confocal microscope (HCM) in a standard fluorescent bead imaging experiment. The comparison results show that despite the use of a light source with four orders of magnitude lower intensity in the IMS than that in the HCM, the image signal-to-noise ratio acquired by the IMS is 2.6 times higher than that achieved by the equivalent confocal approach.

Adaptive optics for fluorescence wide-field microscopy using spectrally independent guide star and markers

We describe the implementation and use of an adaptive optics loop in the imaging path of a commercial wide field microscope. We show that it is possible to maintain the optical performances of the original microscope when imaging through aberrant biological samples. The sources used for illuminating the adaptive optics loop are spectrally independent, in excitation and emission, from the sample, so they do not appear in the final image, and their use does not contribute to the sample bleaching. Results are compared with equivalent images obtained with an identical microscope devoid of adaptive optics system.

Monte Carlo fluorescence microtomography

Fluorescence microscopy allows real-time monitoring of optical molecular probes for disease characterization, drug development, and tissue regeneration. However, when a biological sample is thicker than 1 mm, intense scattering of light would significantly degrade the spatial resolution of fluorescence microscopy. In this paper, we develop a fluorescence microtomography technique that utilizes the Monte Carlo method to image fluorescence reporters in thick biological samples. This approach is based on an l(0)-regularized tomography model and provides an excellent solution. Our studies on biomimetic tissue scaffolds have demonstrated that the proposed approach is capable of localizing and quantifying the distribution of optical molecular probe accurately and reliably.

A sensitive and specific ES-31 antigen detection based fluorometric assay for confirmation of Mycobacterium tuberculosis in cell culture

Confirmation of presence of M. tuberculosis bacilli on microscopic examination is very important in diagnosis of tuberculosis. The present study was undertaken to find the usefulness of mycobacterial ES-31 serine protease as a marker to detect tuberculosis bacilli using fluorescein isothiocyanate conjugated anti-ES-31 serine protease antibody. This immunofluorescence method was compared with Ziehl-Neelsen and auramine-O staining methods for detection of tuberculosis bacilli. Slides were prepared for each serially diluted tuberculosis H37Ra bacilli (1 x 10(7) bacilli/ml to 5 bacilli/ml). Slides for each dilution group were stained by ZN method, auramine-O and immunostaining methods using fluorescein isothiocyanate conjugated anti-ES-31 serine protease antibody. ZN staining method showed efficacy for detection of M. tuberculosis H37Ra upto 1 x 10(4) bacilli/ml while auramine-O method showed upto 1 x 10(2) bacilli/ml. The presence of bacilli was indicated by green fluorescence on immunostaining using anti-ES-31 antibody conjugate and this method was effective upto 10 bacilli/ml. The slides which were negative for ZN (1 x 10(3) cells/ml) and auramine-O (100 cells/ml) method showed positivity on restaining with immunofluorescent staining method. The results of this preliminary study showed that immunofluorescent staining method using specific anti-ES-31 antibody conjugate was more sensitive for detection of tuberculosis bacilli than ZN and auramine-O methods in samples of laboratory strain. The utility of this method will be studied further in clinical specimens.

Enhancing precision in time-domain fluorescence lifetime imaging

In biological applications of fluorescence lifetime imaging, low signals from samples can be a challenge, causing poor lifetime precision. We demonstrate how optimal signal gating (a method applied to the temporal dimension of a lifetime image) and novel total variation denoising models (a method applied to the spatial dimension of a lifetime image) can be used in time-domain fluorescence lifetime imaging microscopy (FLIM) to improve lifetime precision. In time-gated FLIM, notable fourfold precision improvements were observed in a low-light example. This approach can be employed to improve FLIM data while minimizing sample light exposure and increasing imaging speed.

Development of image mappers for hyperspectral biomedical imaging applications

A new design and fabrication method is presented for creating large-format (>100 mirror facets) image mappers for a snapshot hyperspectral biomedical imaging system called an image mapping spectrometer (IMS). To verify this approach a 250 facet image mapper with 25 multiple-tilt angles is designed for a compact IMS that groups the 25 subpupils in a 5 x 5 matrix residing within a single collecting objective's pupil. The image mapper is fabricated by precision diamond raster fly cutting using surface-shaped tools. The individual mirror facets have minimal edge eating, tilt errors of <1 mrad, and an average roughness of 5.4 nm.

Extended depth-of-focus microscopy via constrained deconvolution

We derive a method for extended depth-of-focus imaging, i.e., a method to render a 2-D image of a thick specimen, such that all the structures within the specimen appear in focus and with greatly increased contrast. We acquire a single image while moving the specimen through focus. The resulting image, which is severely blurred and has very low contrast, is then deconvolved. In the deconvolved image, the entire depth of the specimen is in focus. Because the image is collected continuously while the specimen moves through focus, the acquisition time is short. Likewise, because the deconvolution is done in 2-D, it is done very quickly even with an iterative algorithm.

An automated feedback system with the hybrid model of scoring and classification for solving over-segmentation problems in RNAi high content screening

BACKGROUND

High content screening (HCS) via automated fluorescence microscopy is a powerful technology for generating cellular images that are rich in phenotypic information. RNA interference is a revolutionary approach for silencing gene expression and has become an important method for studying genes through RNA interference-induced cellular phenotype analysis. The convergence of the two technologies has led to large-scale, image-based studies of cellular phenotypes under systematic perturbations of RNA interference. However, existing high content screening image analysis tools are inadequate to extract content regarding cell morphology from the complex images, thus they limit the potential of genome-wide RNA interference high content screening screening for simple marker readouts. In particular, over-segmentation is one of the persistent problems of cell segmentation; this paper describes a new method to alleviate this problem.

METHODS

To solve the issue of over-segmentation, we propose a novel feedback system with a hybrid model for automated cell segmentation of images from high content screening. A Hybrid learning model is developed based on three scoring models to capture specific characteristics of over-segmented cells. Dead nuclei are also removed through a statistical model.

RESULTS

Experimental validation showed that the proposed method had 93.7% sensitivity and 94.23% specificity. When applied to a set of images of F-actin-stained Drosophila cells, 91.3% of over-segmented cells were detected and only 2.8% were under-segmented.

CONCLUSIONS

The proposed feedback system significantly reduces over-segmentation of cell bodies caused by over-segmented nuclei, dead nuclei, and dividing cells. This system can be used in the automated analysis system of high content screening images.

Fluorescence relaxation in 3D from diffraction-limited sources of PAGFP or sinks of EGFP created by multiphoton photoconversion

The relaxation of fluorescence from diffraction-limited sources of photoactivatable green fluorescent protein (PAGFP) or sinks of photobleached enhanced GFP (EGFP) created by multiphoton photo-conversion was measured in solutions of varied viscosity (eta), and in live, spherical Chinese hamster ovary (CHO) cells. Fluorescence relaxation was monitored with the probing laser fixed, or rapidly scanning along a line bisected by the photoconversion site. Novel solutions to several problems that hamper the study of PAGFP diffusion after multiphoton photoconversion are presented. A theoretical model of 3D diffusion in a sphere from a source in the shape of the measured multiphoton point-spread function was applied to the fluorescence data to estimate the apparent diffusion coefficient, D(ap). The model incorporates two novel features that make it of broad utility. First, the model includes the no-flux boundary condition imposed by cell plasma membranes, allowing assessment of potential impact of this boundary on estimates of D(ap). Second, the model uses an inhomogeneous source term that, for the first time, allows analysis of diffusion from sources produced by multiphoton photoconversion pulses of varying duration. For diffusion in aqueous solution, indistinguishable linear relationships between D(ap) and eta(-1) were obtained for the two proteins: for PAGFP, D(aq)= 89 +/- 2.4 microm2 s(-1) (mean +/- 95% confidence interval), and for EGFP D(aq)= 91 +/- 1.8 microm2 s(-1). In CHO cells, the application of the model yielded D(ap)= 20 +/- 3 microm2 s(-1) (PAGFP) and 19 +/- 2 microm2 s(-1) (EGFP). Furthermore, the model quantitatively predicted the decline in baseline fluorescence that accompanied repeated photobleaching cycles in CHO cells expressing EGFP, supporting the hypothesis of fluorophore depletion as an alternative to the oft invoked 'bound fraction' explanation of the deviation of the terminal fluorescence recovery from its pre-bleach baseline level. Nonetheless for their identical diffusive properties, advantages of PAGFP over EGFP were found, including an intrinsically higher signal/noise ratio with 488-nm excitation, and the requirement for approximately 1/200th the cumulative light energy to produce data of comparable signal/noise.

Geometric approach to segmentation and protein localization in cell culture assays

Cell-based fluorescence imaging assays are heterogeneous and require the collection of a large number of images for detailed quantitative analysis. Complexities arise as a result of variation in spatial nonuniformity, shape, overlapping compartments and scale (size). A new technique and methodology has been developed and tested for delineating subcellular morphology and partitioning overlapping compartments at multiple scales. This system is packaged as an integrated software platform for quantifying images that are obtained through fluorescence microscopy. Proposed methods are model based, leveraging geometric shape properties of subcellular compartments and corresponding protein localization. From the morphological perspective, convexity constraint is imposed to delineate and partition nuclear compartments. From the protein localization perspective, radial symmetry is imposed to localize punctate protein events at submicron resolution. Convexity constraint is imposed against boundary information, which are extracted through a combination of zero-crossing and gradient operator. If the convexity constraint fails for the boundary then positive curvature maxima are localized along the contour and the entire blob is partitioned into disjointed convex objects representing individual nuclear compartment, by enforcing geometric constraints. Nuclear compartments provide the context for protein localization, which may be diffuse or punctate. Punctate signal are localized through iterative voting and radial symmetries for improved reliability and robustness. The technique has been tested against 196 images that were generated to study centrosome abnormalities. Corresponding computed representations are compared against manual counts for validation.

Basic building units and properties of a fluorescence single plane illumination microscope

The critical issue of all fluorescence microscopes is the efficient use of the fluorophores, i.e., to detect as many photons from the excited fluorophores as possible, as well as to excite only the fluorophores that are in focus. This issue is addressed in EMBL's implementation of a light sheet based microscope [single plane illumination microscope (SPIM)], which illuminates only the fluorophores in the focal plane of the detection objective lens. The light sheet is a beam that is collimated in one and focused in the other direction. Since no fluorophores are excited outside the detectors' focal plane, the method also provides intrinsic optical sectioning. The total number of observable time points can be improved by several orders of magnitude when compared to a confocal fluorescence microscope. The actual improvement factor depends on the number of planes acquired and required to achieve a certain signal to noise ratio. A SPIM consists of five basic units, which address (1) light detection, (2) illumination of the specimen, (3) generation of an appropriate beam of light, (4) translation and rotation of the specimen, and finally (5) control of different mechanical and electronic parts, data collection, and postprocessing of the data. We first describe the basic building units of EMBL's SPIM and its most relevant properties. We then cover the basic principles underlying this instrument and its unique properties such as the efficient usage of the fluorophores, the reduced photo toxic effects, the true optical sectioning capability, and the excellent axial resolution. We also discuss how an isotropic resolution can be achieved. The optical setup, the control hardware, and the control scheme are explained in detail. We also describe some less obvious refinements of the basic setup that result in an improved performance. The properties of the instrument are demonstrated by images of biological samples that were imaged with one of EMBL's SPIMs.

Bayesian hierarchical model for correcting signal saturation in microarrays using pixel intensities

Pixel saturation occurs when the pixel intensity exceeds the scanner upper threshold of detection and the recorded pixel intensity is then truncated at the threshold. Truncation of the pixel intensity causes the estimates of gene expression (i.e., intensity) to be biased. Microarray experiments are commonly affected by saturated pixels; as a result all higher level analyses are made on these biased gene expression estimates. In this paper, we propose a method for improving the quality of the signal for cDNA microarrays by making use of several scans at varying scanner sensitivities. For each spot, pixel level intensity readings are given as input to a Bayesian hierarchical model. The model uses the pixel intensities of the spot to provide a posterior distribution of the true expression level of the corresponding genes. The parameters of the hierarchical model are estimated jointly with these expression levels, thus performing an integrated analysis of the measurement data. The method improves in all ranges the accuracy with which intensities can be estimated and extends the dynamic range of measured gene expression at the high end. The method is generic and can be applied to data from any organism and for imaging with any scanner. Results from a real data set illustrate an improved precision in the estimation of the expression of genes compared to what can be achieved by applying standard methods and using only a single scan.

Intensity correction of fluorescent confocal laser scanning microscope images by mean-weight filtering

This paper addresses the problem of intensity correction of fluorescent confocal laser scanning microscope images. Confocal laser scanning microscope images are frequently used in medicine for obtaining 3D information about specimen structures by imaging a set of 2D cross sections and performing 3D volume reconstruction afterwards. However, 2D images acquired from fluorescent confocal laser scanning microscope images demonstrate significant intensity heterogeneity, for example, due to photo-bleaching and fluorescent attenuation in depth. We developed an intensity heterogeneity correction technique that (a) adjusts the intensity heterogeneity of 2D images, (b) preserves fine structural details and (c) enhances image contrast, by performing spatially adaptive mean-weight filtering. Our solution is obtained by formulating an optimization problem, followed by filter design and automated selection of filtering parameters. The proposed filtering method is experimentally compared with several existing techniques by using four quality metrics: contrast, intensity heterogeneity (entropy) in a low frequency domain, intensity distortion in a high frequency domain and saturation. Based on our experiments and the four quality metrics, the developed mean-weight filtering outperforms other intensity correction methods by at least a factor of 1.5 when applied to fluorescent confocal laser scanning microscope images.

Sensitive methods to detect epithelial disruption: tests for microhemorrhage in cervicovaginal lavages

The toxic effects of topical microbicides might include epithelial disruption, and if sufficiently severe, may cause visible bleeding. We have developed two sensitive methods to detect hemorrhage far below the visual detection threshold: an enzyme-linked immunosorbent assay for human hemoglobin and a method for staining and enumerating red blood cells by fluorescent microscopy. We have applied these methods to cervicovaginal and introital lavages, and found them to be sensitive and quantitative over a broad range. Comparing the new techniques with a standard heme-detection assay (Hemastix) we found them to be more sensitive, more objective, more fully quantitative, and more likely to be indicative of recent events. In pilot studies we found that the red blood cell and hemoglobin concentrations in samples taken in the luteal phase of the menstrual cycle are low, but still within the detection range of the new assays, providing a quantifiable baseline from which increases in blood from epithelial disruption can be detected and quantified. We propose that these new methods may be used for the safety assessment of microbicides by detecting and quantifying microhemorrhage in cervicovaginal secretions before and after exposure to microbicide.

Single live cell imaging for real-time monitoring of resistance mechanism in Pseudomonas aeruginosa

We have developed and applied single live cell imaging for real-time monitoring of resistance kinetics of Pseudomonas aeruginosa. Real-time images of live cells in the presence of a particular substrate (EtBr) provided the first direct insights of resistance mechanism with both spatial and temporal information and showed that the substrate appeared to be accumulated in cytoplasmic space, but not periplasmic space. Three mutants of P. aeruginosa, PAO4290 (a wild-type expression level of MexAB-OprM), TNP030#1 (nalB-1, MexAB-OprM over expression mutant), and TNP076 (DeltaABM, MexAB-OprM deficient mutant), were used to investigate the roles of these three membrane proteins (MexAB-OprM) in the resistance mechanism. Ethidium bromide (EtBr) was chosen as a fluorescence probe for spectroscopic measurement of bulk cell solution and single cell imaging of bulk cells. Bulk measurement indicated, among three mutants, that nalB-1 accumulated the least EtBr and showed the highest resistance to EtBr, whereas DeltaABM accumulated the most EtBr and showed the lowest resistance to EtBr. This result demonstrated the MexAB-OprM proteins played the roles in resistance mechanism by extruding EtBr out of cells. Unlike the bulk measurement, imaging and analysis of bulk cells at single cell resolution demonstrated individual cell had its distinguished resistance kinetics and offered the direct observation of the regulation of influx and efflux of EtBr with both spatial and temporal resolution. Unlike fluorescent staining assays, live cell imaging provided the real-time kinetic information of transformation of membrane permeability and efflux pump machinery of three mutants. This research constitutes the first direct imaging of resistance mechanism of live bacterial cells at single cell resolution and opens up the new possibility of advancing the understanding of bacteria resistance mechanism.

Fluorescence analysis of biochemical constituents identifies atherosclerotic plaque with a thin fibrous cap

Vulnerable plaque generally contains a thin fibrous cap, lipid pools, and reduced internal plaque collagen. Arterial fluorescence analysis can differentiate atherosclerotic lesions from normal arteries; however, the contribution of the lipid core to atherosclerotic arterial fluorescence remains controversial. This study aimed to identify lipid core fluorophores and to differentiate the lipid core from normal artery and atheroma. The helium-cadmium laser-induced fluorescence spectra of cadaveric arteries and known chemical constituents were recorded. Lipid core fluorescence spectra exhibited marked red shifts and broadening compared with the fluorescence spectra of normal tissue and atheroma. Similar fluorescence spectra were obtained for lipid core and oxidized low density lipoprotein, for atheroma and collagen, and for normal artery and elastin. A classification based on collagen, elastin, and oxidized low density lipoprotein spectral decomposition could discriminate the lipid core (n=29), normal artery (n=74), atheroma (n=73), and preatheroma (n=10) with 86% accuracy. Fibrous cap thickness was correlated with the spectral collagen content index (r=0.65, P<0.0001), especially at a thickness of <200 microm. We conclude that a classification algorithm based on chemical spectral decomposition can accurately classify the fluorescence spectra of normal artery, atheroma, and lipid core and may be useful in identifying vulnerable atheroma in vivo.

Quantitative dosing of surfaces with fluorescent molecules: characterization of fractional monolayer coverages by counting single molecules

Quantitative deposition of dye molecules onto a substrate has been achieved at very low surface concentrations, in the range of 5 x 10(-8) - 1 x 10(-6) monolayer, using the technique of controlled substrate withdrawal from solution. These small surface populations were determined with high (>96%) efficiency by single-molecule counting using an epi-illumination, fluorescence microscope with charge-coupled device detector. The fluorescence imaging resolution (3sigma) is 0.78 microm; over a uniform excitation area of 67 x 67 microm2, a large number (>7,500) of spatially resolved channels are available for counting individual molecules. At low coverages, the number density of fluorescence spots on the surface agrees with the expected surface concentration of molecules, based on the concentration of dye in solution and the solution film thickness predicted from theory. When the surface density of molecules is high enough that fluorescence spot overlap is likely to occur within the optical resolution of the instrument, the observed fewer number of spots can be corrected for overlap through a site occupation model based on Poisson statistics.

Observing secretory granules with a multiangle evanescent wave microscope

In total internal reflection fluorescence microscopy (TIRFM), fluorophores near a surface can be excited with evanescent waves, which decay exponentially with distance from the interface. Penetration depths of evanescent waves from 60 nm to 300 nm were generated by varying the angle of incidence of a laser beam. With a novel telecentric multiangle evanescent wave microscope, we monitored and investigated both single secretory granules and pools of granules in bovine chromaffin cells. By measuring the fluorescence intensity as a function of penetration depth, it is possible through a Laplace transform to obtain the fluorophore distribution as a function of axial position. We discuss the extent to which it is possible to determine distances and diameters of granules with this microscopy technique by modeling the fluorescent volumes of spheres in evanescent fields. The anisotropic near-field detection of fluorophores and the influence of the detection point-spread function are considered. The diameters of isolated granules between 70 nm and 300 nm have been reconstructed, which is clearly beyond the resolution limit of a confocal microscope. Furthermore, the paper demonstrates how evanescent waves propagate along surfaces and scatter at objects with a higher refractive index. TIRFM will have a limited applicability for quantitative measurements when the parameters used to define evanescent waves are not optimally selected.

Global analysis of fluorescence lifetime imaging microscopy data

Global analysis techniques are described for frequency domain fluorescence lifetime imaging microscopy (FLIM) data. These algorithms exploit the prior knowledge that only a limited number of fluorescent molecule species whose lifetimes do not vary spatially are present in the sample. Two approaches to implementing the lifetime invariance constraint are described. In the lifetime invariant fit method, each image in the lifetime image sequence is spatially averaged to obtain an improved signal-to-noise ratio. The lifetime estimations from these averaged data are used to recover the fractional contribution to the steady-state fluorescence on a pixel-by-pixel basis for each species. The second, superior, approach uses a global analysis technique that simultaneously fits the fractional contributions in all pixels and the spatially invariant lifetimes. In frequency domain FLIM the maximum number of lifetimes that can be fit with the global analysis method is twice the number of lifetimes that can be fit with conventional approaches. As a result, it is possible to discern two lifetimes with a single-frequency FLIM setup. The algorithms were tested on simulated data and then applied to separate the cellular distributions of coexpressed green fluorescent proteins in living cells.

Two-dimensional fluorescence intensity distribution analysis: theory and applications

A method of sample analysis is presented which is based on fitting a joint distribution of photon count numbers. In experiments, fluorescence from a microscopic volume containing a fluctuating number of molecules is monitored by two detectors, using a confocal microscope. The two detectors may have different polarizational or spectral responses. Concentrations of fluorescent species together with two specific brightness values per species are determined. The two-dimensional fluorescence intensity distribution analysis (2D-FIDA), if used with a polarization cube, is a tool that is able to distinguish fluorescent species with different specific polarization ratios. As an example of polarization studies by 2D-FIDA, binding of 5'-(6-carboxytetramethylrhodamine) (TAMRA)-labeled theophylline to an anti-theophylline antibody has been studied. Alternatively, if two-color equipment is used, 2D-FIDA can determine concentrations and specific brightness values of fluorescent species corresponding to individual labels alone and their complex. As an example of two-color 2D-FIDA, binding of TAMRA-labeled somatostatin-14 to the human type-2 high-affinity somatostatin receptors present in stained vesicles has been studied. The presented method is unusually accurate among fluorescence fluctuation methods. It is well suited for monitoring a variety of molecular interactions, including receptors and ligands or antibodies and antigens.

Mean and variance of ratio estimators used in fluorescence ratio imaging

BACKGROUND

The ratio of two measured fluorescence signals (called x and y) is used in different applications in fluorescence microscopy. Multiple instances of both signals can be combined in different ways to construct different ratio estimators.

METHODS

The mean and variance of three estimators for the ratio between two random variables, x and y, are discussed. Given n samples of x and y, we can intuitively construct two different estimators: the mean of the ratio of each x and y and the ratio between the mean of x and the mean of y. The former is biased and the latter is only asymptotically unbiased. Using the statistical characteristics of this estimator, a third, unbiased estimator can be constructed.

RESULTS

We tested the three estimators on simulated data, real-world fluorescence test images, and comparative genome hybridization (CGH) data. The results on the simulated and real-world test images confirm the presented theory. The CGH experiments show that our new estimator performs better than the existing estimators.

CONCLUSIONS

We have derived an unbiased ratio estimator that outperforms intuitive ratio estimators.

A theoretical investigation of single-molecule fluorescence detection on thin metallic layers

In the present paper, the excitation and detection of single-molecule fluorescence over thin metallic films is studied theoretically within the framework of classical electrodynamics. The model takes into account the specific conditions of surface plasmon-assisted optical excitation, fluorescence quenching by the metal film, and detection geometry. Extensive numerical results are presented for gold, silver, and aluminum films, showing the detectable fluorescence intensities and their dependence on film thickness and the fluorescent molecule's position under optimal excitation conditions.

Toward objective selection of representative microscope images

Scientists wishing to communicate the essential characteristics of a pattern (such as an immunofluorescence distribution) currently must make a subjective choice of one or two images to publish. We therefore developed methods for objectively choosing a typical image from a set, with emphasis on images from cell biology. The methods involve calculation of numerical features to describe each image, calculation of similarity between images as a distance in feature space, and ranking of images by distance from the center of the feature distribution. Two types of features were explored, image texture measures and Zernike polynomial moments, and various distance measures were utilized. Criteria for evaluating methods for assigning typicality were proposed and applied to sets of images containing more than one pattern. The results indicate the importance of using distance measures that are insensitive to the presence of outliers. For collections of images of the distributions of a lysosomal protein, a Golgi protein, and nuclear DNA, the images chosen as most typical were in good agreement with the conventional understanding of organelle morphologies. The methods described here have been implemented in a web server (http://murphylab.web.cmu.edu/services/TyplC).

Intensity-based energy transfer measurements in digital imaging microscopy

Investigation of protein-protein associations is important in understanding structure and function relationships in living cells. Using Förster-type resonance energy transfer between donor and acceptor labeled monoclonal antibodies we can assess the cell surface topology of membrane proteins against which the antibodies were raised. In our current work we elaborated a quantitative image microscopic technique based on the measurement of fluorescence intensities to calculate the energy transfer efficiency on a pixel-by-pixel basis. We made use of the broad excitation and emission spectrum of cellular autofluorescence for background correction of images. In addition to the reference autofluorescence images (UV background) we recorded three fluorescent images (donor, acceptor and energy transfer signal) of donor-acceptor double labeled samples, and corrected for spectral spillage of the directly excited donor and acceptor fluorescence into the energy transfer image. After careful image registration we were able to calculate the energy transfer efficiency on a pixel-by -pixel basis. In this paper, we also present a critical comparison between results obtained with this method and other approaches (photobleaching and flow cytometric energy transfer measurements).

Site-specific confocal fluorescence imaging of biological microstructures in a turbid medium

Normally transparent biological structures in a turbid medium are imaged using a laser confocal microscope and multiwavelength site-specific fluorescence labelling. The spatial filtering capability of the detector pinhole in the confocal microscope limits the number of scattered fluorescent photons that reach the photodetector. Simultaneous application of different fluorescent markers on the same sample site minimizes photobleaching by reducing the excitation time for each marker. A high-contrast grey-level image is also produced by summing confocal images of the same site taken at different fluorescence wavelengths. Monte Carlo simulations are performed to obtain the quantitative behaviour of confocal fluorescence imaging in turbid media. Confocal images of the following samples were also obtained: (i) 15 microm diameter fluorescent spheres placed 1.16 mm deep beneath an aqueous suspension of 0.0823 microm diameter polystyrene latex spheres, and (ii) hindbrain of a whole-mount mouse embryo (age 10 days) that was stained to fluoresce at 515 nm and 580 nm peak wavelengths. Expression of RNA transcripts of a gene within the embryo hindbrain was detected by a fluorescence-based whole-mount in situ hybridization procedure that we recently tested.

A computer program that periodically monitors the ability to interpret the antinuclear antibody test

Our laboratory has been developing computer programs that help medical technologists improve their performance of the microscope-based immunofluorescence assay for antinuclear antibodies (ANA). This image-based laboratory test has been associated with poor reproducibility. We have previously described our first program, ANA-Tutor, which systematically teaches the ANA test by using approximately 150 processed digital images of ANA test results. The program we describe here, Pattern Plus Auditor, is a logical extension to ANA-Tutor. Pattern Plus Auditor tests the ability of laboratory personnel to interpret the ANA test, and tracks individual and laboratory performance over time. The program consists of image-based questions that test a variety of ANA staining patterns, including homogeneous, speckled, centromere, nucleolar, mixed patterns, and rare patterns. For each question, the program provides correct answers with explanations and color overlays that highlight key image features. By entering the proper password, users gain access to exam results for individuals and for the laboratory as a whole. Results are available for the current exam, any previous exam, or cumulatively on all exams to date. Intralaboratory testing with computer programs such as Pattern Plus Auditor might be a useful part of quality-assurance procedures for many image-based laboratory tests.

Utility of fluorescence microscopy in embryonic/fetal topographical analysis

For topographical analysis of developing embryos, investigators typically rely on scanning electron microscopy (SEM) to provide the surface detail not attainable with light microscopy. SEM is an expensive and time-consuming technique, however, and the preparation procedure may alter morphology and leave the specimen friable. We report that by using a high-resolution compound epifluorescence microscope with inexpensive low-power objectives and the fluorochrome acridine orange, we were able to obtain surface images of fixed or fresh whole rat embryos and fetal palates of considerably greater topographical detail than those obtained using routine light microscopy. Indeed the resulting high-resolution images afford not only superior qualitative documentation of morphological observations, but the capability for detailed morphometry via digitization and computer-assisted image analysis.

Direct detection of Mycobacterium tuberculosis complex in respiratory specimens by a target-amplified test system

A total of 938 respiratory specimens (633 sputa, 249 bronchial and tracheal aspirates, and 56 bronchoalveolar lavages) from 589 patients were tested for direct detection of Mycobacterium tuberculosis complex by the Gen-Probe amplified Mycobacterium tuberculosis direct test (MTD), and the results were compared with those of the conventional methods of fluorescence microscopy and cultivation (solid and radiometric media). One series of specimens (n = 515) was decontaminated with N-acetyl-L-cysteine (NALC)-NaOH: the other one (n = 423) was decontaminated with sodium dodecyl (lauryl) sulfate (SDS)-NaOH. Of the specimens decontaminated with NALC, 39 were MTD and culture positive, 455 were MTD and culture negative, 18 were MTD positive and culture negative, and 3 were MTD negative and culture positive, indicating a sensitivity of 92.9% and a specificity of 96.2% for the MTD. Of the specimens decontaminated with SDS, 35 were MTD and culture positive, 372 were MTD and culture negative, 15 were MTD positive and culture negative, and 1 was MTD negative and culture positive, indicating a sensitivity of 97.2% and a specificity of 96.1% for the MTD. After resolution of discrepant results by review of the patients' clinical data, the sensitivity of the MTD was 93.9%, the specificity was 97.6%, the positive predictive value was 80.7%, and the negative predictive value was 99.3% for the NALC series; the corresponding values were 97.4, 96.9, 76.0, and 99.7%, respectively, for the SDS series. In conclusion, the MTD is a highly sensitive and specific technique for detecting M. tuberculosis complex within hours in both smear-positive and smear-negative respiratory specimens.