Computational framework for simulating fluorescence microscope images with cell populations

Physically-based in silico light sheet microscopy for visualizing fluorescent brain models

Background

We present a physically-based computational model of the light sheet fluorescence microscope (LSFM). Based on Monte Carlo ray tracing and geometric optics, our method simulates the operational aspects and image formation process of the LSFM. This simulated, in silico LSFM creates synthetic images of digital fluorescent specimens that can resemble those generated by a real LSFM, as opposed to established visualization methods producing visually-plausible images. We also propose an accurate fluorescence rendering model which takes into account the intrinsic characteristics of fluorescent dyes to simulate the light interaction with fluorescent biological specimen.

Results

We demonstrate first results of our visualization pipeline to a simplified brain tissue model reconstructed from the somatosensory cortex of a young rat. The modeling aspects of the LSFM units are qualitatively analysed, and the results of the fluorescence model were quantitatively validated against the fluorescence brightness equation and characteristic emission spectra of different fluorescent dyes.

AMS subject classification

Modelling and simulation

Bayesian-based deconvolution fluorescence microscopy using dynamically updated nonstationary expectation estimates

Fluorescence microscopy is widely used for the study of biological specimens. Deconvolution can significantly improve the resolution and contrast of images produced using fluorescence microscopy; in particular, Bayesian-based methods have become very popular in deconvolution fluorescence microscopy. An ongoing challenge with Bayesian-based methods is in dealing with the presence of noise in low SNR imaging conditions. In this study, we present a Bayesian-based method for performing deconvolution using dynamically updated nonstationary expectation estimates that can improve the fluorescence microscopy image quality in the presence of noise, without explicit use of spatial regularization.

Detecting overlapping instances in microscopy images using extremal region trees

In many microscopy applications the images may contain both regions of low and high cell densities corresponding to different tissues or colonies at different stages of growth. This poses a challenge to most previously developed automated cell detection and counting methods, which are designed to handle either the low-density scenario (through cell detection) or the high-density scenario (through density estimation or texture analysis). The objective of this work is to detect all the instances of an object of interest in microscopy images. The instances may be partially overlapping and clustered. To this end we introduce a tree-structured discrete graphical model that is used to select and label a set of non-overlapping regions in the image by a global optimization of a classification score. Each region is labeled with the number of instances it contains - for example regions can be selected that contain two or three object instances, by defining separate classes for tuples of objects in the detection process. We show that this formulation can be learned within the structured output SVM framework and that the inference in such a model can be accomplished using dynamic programming on a tree structured region graph. Furthermore, the learning only requires weak annotations - a dot on each instance. The candidate regions for the selection are obtained as extremal region of a surface computed from the microscopy image, and we show that the performance of the model can be improved by considering a proxy problem for learning the surface that allows better selection of the extremal regions. Furthermore, we consider a number of variations for the loss function used in the structured output learning. The model is applied and evaluated over six quite disparate data sets of images covering: fluorescence microscopy, weak-fluorescence molecular images, phase contrast microscopy and histopathology images, and is shown to exceed the state of the art in performance.

Simulation of bright-field microscopy images depicting pap-smear specimen

As digital imaging is becoming a fundamental part of medical and biomedical research, the demand for computer-based evaluation using advanced image analysis is becoming an integral part of many research projects. A common problem when developing new image analysis algorithms is the need of large datasets with ground truth on which the algorithms can be tested and optimized. Generating such datasets is often tedious and introduces subjectivity and interindividual and intraindividual variations. An alternative to manually created ground-truth data is to generate synthetic images where the ground truth is known. The challenge then is to make the images sufficiently similar to the real ones to be useful in algorithm development. One of the first and most widely studied medical image analysis tasks is to automate screening for cervical cancer through Pap-smear analysis. As part of an effort to develop a new generation cervical cancer screening system, we have developed a framework for the creation of realistic synthetic bright-field microscopy images that can be used for algorithm development and benchmarking. The resulting framework has been assessed through a visual evaluation by experts with extensive experience of Pap-smear images. The results show that images produced using our described methods are realistic enough to be mistaken for real microscopy images. The developed simulation framework is very flexible and can be modified to mimic many other types of bright-field microscopy images. © 2015 The Authors. Published by Wiley Periodicals, Inc. on behalf of ISAC

Feature-based filter design for resolution enhancement of known features in microscopy

We present a new feature-based design concept for filter design in which a filter is designed specifically to image a known feature with dimensions lower than the optical resolution of the system. Unlike the conventional filter design based on focal spot engineering, we use the complete response of the microscope to form a resolution factor and consider minimizing the resolution factor as the design goal. We consider three design goals (i.e., resolution factors) based on the system response and show that a feature matching approach is more suitable in practice. It is shown that a three-bar pattern and grid of square dots of critical dimension, 0.08λ, can be resolved using a simple two-layer feature-based filter designed for radially polarized illumination in aplanatic solid immersion lens microscopy. An example that requires a more complex filter is also shown. Applicability of the concept of feature-based filter design in diverse microscopy scenarios is discussed as well.

Normalized polarization ratios for the analysis of cell polarity

The quantification and analysis of molecular localization in living cells is increasingly important for elucidating biological pathways, and new methods are rapidly emerging. The quantification of cell polarity has generated much interest recently, and ratiometric analysis of fluorescence microscopy images provides one means to quantify cell polarity. However, detection of fluorescence, and the ratiometric measurement, is likely to be sensitive to acquisition settings and image processing parameters. Using imaging of EGFP-expressing cells and computer simulations of variations in fluorescence ratios, we characterized the dependence of ratiometric measurements on processing parameters. This analysis showed that image settings alter polarization measurements; and that clustered localization is more susceptible to artifacts than homogeneous localization. To correct for such inconsistencies, we developed and validated a method for choosing the most appropriate analysis settings, and for incorporating internal controls to ensure fidelity of polarity measurements. This approach is applicable to testing polarity in all cells where the axis of polarity is known.

Shading correction for whole slide image using low rank and sparse decomposition

Many microscopic imaging modalities suffer from the problem of intensity inhomogeneity due to uneven illumination or camera nonlinearity, known as shading artifacts. A typical example of this is the unwanted seam when stitching images to obtain a whole slide image (WSI). Elimination of shading plays an essential role for subsequent image processing such as segmentation, registration, or tracking. In this paper, we propose two new retrospective shading correction algorithms for WSI targeted to two common forms of WSI: multiple image tiles before mosaicking and an already-stitched image. Both methods leverage on recent achievements in matrix rank minimization and sparse signal recovery. We show how the classic shading problem in microscopy can be reformulated as a decomposition problem of low-rank and sparse components, which seeks an optimal separation of the foreground objects of interest and the background illumination field. Additionally, a sparse constraint is introduced in the Fourier domain to ensure the smoothness of the recovered background. Extensive qualitative and quantitative validation on both synthetic and real microscopy images demonstrates superior performance of the proposed methods in shading removal in comparison with a well-established method in ImageJ.

Automatic Cell Detection in Bright-Field Microscope Images Using SIFT, Random Forests, and Hierarchical Clustering

We present a novel machine learning-based system for unstained cell detection in bright-field microscope images. The system is fully automatic since it requires no manual parameter tuning. It is also highly invariant with respect to illumination conditions and to the size and orientation of cells. Images from two adherent cell lines and one suspension cell line were used in the evaluation for a total number of more than 3500 cells. Besides real images, simulated images were also used in the evaluation. The detection error was between approximately zero and 15.5% which is a significantly superior performance compared to baseline approaches.

A flexible and robust approach for segmenting cell nuclei from 2D microscopy images using supervised learning and template matching

We describe a new supervised learning-based template matching approach for segmenting cell nuclei from microscopy images. The method uses examples selected by a user for building a statistical model that captures the texture and shape variations of the nuclear structures from a given dataset to be segmented. Segmentation of subsequent, unlabeled, images is then performed by finding the model instance that best matches (in the normalized cross correlation sense) local neighborhood in the input image. We demonstrate the application of our method to segmenting nuclei from a variety of imaging modalities, and quantitatively compare our results to several other methods. Quantitative results using both simulated and real image data show that, while certain methods may work well for certain imaging modalities, our software is able to obtain high accuracy across several imaging modalities studied. Results also demonstrate that, relative to several existing methods, the template-based method we propose presents increased robustness in the sense of better handling variations in illumination, variations in texture from different imaging modalities, providing more smooth and accurate segmentation borders, as well as handling better cluttered nuclei.

Validation of various adaptive threshold methods of segmentation applied to follicular lymphoma digital images stained with 3,3'-Diaminobenzidine&Haematoxylin

The comparative study of the results of various segmentation methods for the digital images of the follicular lymphoma cancer tissue section is described in this paper. The sensitivity and specificity and some other parameters of the following adaptive threshold methods of segmentation: the Niblack method, the Sauvola method, the White method, the Bernsen method, the Yasuda method and the Palumbo method, are calculated. Methods are applied to three types of images constructed by extraction of the brown colour information from the artificial images synthesized based on counterpart experimentally captured images. This paper presents usefulness of the microscopic image synthesis method in evaluation as well as comparison of the image processing results. The results of thoughtful analysis of broad range of adaptive threshold methods applied to: (1) the blue channel of RGB, (2) the brown colour extracted by deconvolution and (3) the ’brown component’ extracted from RGB allows to select some pairs: method and type of image for which this method is most efficient considering various criteria e.g. accuracy and precision in area detection or accuracy in number of objects detection and so on. The comparison shows that the White, the Bernsen and the Sauvola methods results are better than the results of the rest of the methods for all types of monochromatic images. All three methods segments the immunopositive nuclei with the mean accuracy of 0.9952, 0.9942 and 0.9944 respectively, when treated totally. However the best results are achieved for monochromatic image in which intensity shows brown colour map constructed by colour deconvolution algorithm. The specificity in the cases of the Bernsen and the White methods is 1 and sensitivities are: 0.74 for White and 0.91 for Bernsen methods while the Sauvola method achieves sensitivity value of 0.74 and the specificity value of 0.99. According to Bland-Altman plot the Sauvola method selected objects are segmented without undercutting the area for true positive objects but with extra false positive objects. The Sauvola and the Bernsen methods gives complementary results what will be exploited when the new method of virtual tissue slides segmentation be develop.

Virtual Slides

The virtual slides for this article can be found here: slide 1: http://diagnosticpathology.slidepath.com/dih/webViewer.php?snapshotId=13617947952577 and slide 2: http://diagnosticpathology.slidepath.com/dih/webViewer.php?snapshotId=13617948230017.

On detection of multiple object instances using Hough transforms

Hough transform-based methods for detecting multiple objects use nonmaxima suppression or mode seeking to locate and distinguish peaks in Hough images. Such postprocessing requires the tuning of many parameters and is often fragile, especially when objects are located spatially close to each other. In this paper, we develop a new probabilistic framework for object detection which is related to the Hough transform. It shares the simplicity and wide applicability of the Hough transform but, at the same time, bypasses the problem of multiple peak identification in Hough images and permits detection of multiple objects without invoking nonmaximum suppression heuristics. Our experiments demonstrate that this method results in a significant improvement in detection accuracy both for the classical task of straight line detection and for a more modern category-level (pedestrian) detection problem.

Features for cells and nuclei classification

The performance of automated analysis of cellular images is heavily influenced by the features that characterize cells or cell nuclei. In this paper, an exhaustive set of features including morphological, topological, and texture features are explored to determine the optimal features for classification of cells and cell nuclei. The optimal subset of features are obtained using popular feature selection methods. The results of feature selection indicate that Zernike moment, Daubechies wavelets, and Gabor wavelets give the most important features for the classification of cells or cell nuclei in fluorescent microscopy images.

Analysis of spatial point patterns in nuclear biology

There is considerable interest in cell biology in determining whether, and to what extent, the spatial arrangement of nuclear objects affects nuclear function. A common approach to address this issue involves analyzing a collection of images produced using some form of fluorescence microscopy. We assume that these images have been successfully pre-processed and a spatial point pattern representation of the objects of interest within the nuclear boundary is available. Typically in these scenarios, the number of objects per nucleus is low, which has consequences on the ability of standard analysis procedures to demonstrate the existence of spatial preference in the pattern. There are broadly two common approaches to look for structure in these spatial point patterns. First a spatial point pattern for each image is analyzed individually, or second a simple normalization is performed and the patterns are aggregated. In this paper we demonstrate using synthetic spatial point patterns drawn from predefined point processes how difficult it is to distinguish a pattern from complete spatial randomness using these techniques and hence how easy it is to miss interesting spatial preferences in the arrangement of nuclear objects. The impact of this problem is also illustrated on data related to the configuration of PML nuclear bodies in mammalian fibroblast cells.

Computational pathology: challenges and promises for tissue analysis

The histological assessment of human tissue has emerged as the key challenge for detection and treatment of cancer. A plethora of different data sources ranging from tissue microarray data to gene expression, proteomics or metabolomics data provide a detailed overview of the health status of a patient. Medical doctors need to assess these information sources and they rely on data driven automatic analysis tools. Methods for classification, grouping and segmentation of heterogeneous data sources as well as regression of noisy dependencies and estimation of survival probabilities enter the processing workflow of a pathology diagnosis system at various stages. This paper reports on state-of-the-art of the design and effectiveness of computational pathology workflows and it discusses future research directions in this emergent field of medical informatics and diagnostic machine learning.

Interactive segmentation of clustered cells via geodesic commute distance and constrained density weighted Nyström method

An interactive method is proposed for complex cell segmentation, in particular of clustered cells. This article has two main contributions: First, we explore a hybrid combination of the random walk and the geodesic graph based methods for image segmentation and propose the novel concept of geodesic commute distance to classify pixels. The computation of geodesic commute distance requires an eigenvector decomposition of the weighted Laplacian matrix of a graph constructed from the image to be segmented. Second, by incorporating pairwise constraints from seeds into the algorithm, we present a novel method for eigenvector decomposition, namely a constrained density weighted Nyström method. Both visual and quantitative comparison with other semiautomatic algorithms including Voronoi-based segmentation, grow cut, graph cuts, random walk, and geodesic method are given to evaluate the performance of the proposed method, which is a powerful tool for quantitative analysis of clustered cell images in live cell imaging.

Possible effect from shear stress on maturation of somatic embryos of Norway spruce (Picea abies)

Somatic embryogenesis is the only method with the potential for industrial scale clonal propagation of conifers. Implementation of the method has so far been hampered by the extensive manual labor required for development of the somatic embryos into plants. The utilization of bioreactors is limited since the somatic embryos will not mature and germinate under liquid culture conditions. The negative effect on mature embryo yields from liquid culture conditions has been previously described. We have described the negative effects of shear stress on the development of early stage somatic embryos (proembryogenic masses; PEMs) at shear stresses of 0.086 and 0.14 N/m(2). In the present study, additional flow rates were studied to determine the effects of shear stress at lower rates resembling shear stress in a suspension culture flask. The results showed that shear stress at 0.009, 0.014, and 0.029 N/m(2) inhibited the PEM expansions comparing with the control group without shear stress. This study also provides validation for the cross-correlation method previously developed to show the effect of shear stress on early stage embryo suspensor cell formation and polarization. Furthermore, shear stress was shown to positively affect the uptake of water into the cells. The results indicate that the plasmolyzing effect from macromolecules added to liquid culture medium to stimulate maturation of the embryos are affected by liquid culture conditions and thus can affect the conversion of PEMs into mature somatic embryos.

Communicating subcellular distributions

To build more accurate models of cells and tissues, the ability to incorporate information on the distributions of proteins (and other macromolecules) will become increasingly important. This review describes current progress towards determining and representing protein subcellular patterns so that the information can be used as part of systems biology efforts. Approaches to decomposing an image of the subcellular pattern of a protein give critical information about the fraction of that protein in each of a number of fundamental patterns (e.g., organelles). Methods for learning generative models from images provide a means of capturing the essential properties and variation in those properties of cell shape and organelle patterns. The combination of models of fundamental patterns and vectors specifying the fraction of a protein in each of them provide a much better means of communicating subcellular patterns than the descriptive terms that are currently used. Communicating information about subcellular patterns is important not only for systems biology simulations but also for representing results from microscopy experiments, including high content screening and imaging flow cytometry, in a transportable and generalizable manner.

Cell nuclei and cytoplasm joint segmentation using the sliding band filter

Microscopy cell image analysis is a fundamental tool for biological research. In particular, multivariate fluorescence microscopy is used to observe different aspects of cells in cultures. It is still common practice to perform analysis tasks by visual inspection of individual cells which is time consuming, exhausting and prone to induce subjective bias. This makes automatic cell image analysis essential for large scale, objective studies of cell cultures. Traditionally the task of automatic cell analysis is approached through the use of image segmentation methods for extraction of cells' locations and shapes. Image segmentation, although fundamental, is neither an easy task in computer vision nor is it robust to image quality changes. This makes image segmentation for cell detection semi-automated requiring frequent tuning of parameters. We introduce a new approach for cell detection and shape estimation in multivariate images based on the sliding band filter (SBF). This filter's design makes it adequate to detect overall convex shapes and as such it performs well for cell detection. Furthermore, the parameters involved are intuitive as they are directly related to the expected cell size. Using the SBF filter we detect cells' nucleus and cytoplasm location and shapes. Based on the assumption that each cell has the same approximate shape center in both nuclei and cytoplasm fluorescence channels, we guide cytoplasm shape estimation by the nuclear detections improving performance and reducing errors. Then we validate cell detection by gathering evidence from nuclei and cytoplasm channels. Additionally, we include overlap correction and shape regularization steps which further improve the estimated cell shapes. The approach is evaluated using two datasets with different types of data: a 20 images benchmark set of simulated cell culture images, containing 1000 simulated cells; a 16 images Drosophila melanogaster Kc167 dataset containing 1255 cells, stained for DNA and actin. Both image datasets present a difficult problem due to the high variability of cell shapes and frequent cluster overlap between cells. On the Drosophila dataset our approach achieved a precision/recall of 95%/69% and 82%/90% for nuclei and cytoplasm detection respectively and an overall accuracy of 76%.

Evaluation of methods for detection of fluorescence labeled subcellular objects in microscope images

Background

Several algorithms have been proposed for detecting fluorescently labeled subcellular objects in microscope images. Many of these algorithms have been designed for specific tasks and validated with limited image data. But despite the potential of using extensive comparisons between algorithms to provide useful information to guide method selection and thus more accurate results, relatively few studies have been performed.

Results

To better understand algorithm performance under different conditions, we have carried out a comparative study including eleven spot detection or segmentation algorithms from various application fields. We used microscope images from well plate experiments with a human osteosarcoma cell line and frames from image stacks of yeast cells in different focal planes. These experimentally derived images permit a comparison of method performance in realistic situations where the number of objects varies within image set. We also used simulated microscope images in order to compare the methods and validate them against a ground truth reference result. Our study finds major differences in the performance of different algorithms, in terms of both object counts and segmentation accuracies.

Conclusions

These results suggest that the selection of detection algorithms for image based screens should be done carefully and take into account different conditions, such as the possibility of acquiring empty images or images with very few spots. Our inclusion of methods that have not been used before in this context broadens the set of available detection methods and compares them against the current state-of-the-art methods for subcellular particle detection.

Generation of digital phantoms of cell nuclei and simulation of image formation in 3D image cytometry

Image cytometry still faces the problem of the quality of cell image analysis results. Degradations caused by cell preparation, optics, and electronics considerably affect most 2D and 3D cell image data acquired using optical microscopy. That is why image processing algorithms applied to these data typically offer imprecise and unreliable results. As the ground truth for given image data is not available in most experiments, the outputs of different image analysis methods can be neither verified nor compared to each other. Some papers solve this problem partially with estimates of ground truth by experts in the field (biologists or physicians). However, in many cases, such a ground truth estimate is very subjective and strongly varies between different experts. To overcome these difficulties, we have created a toolbox that can generate 3D digital phantoms of specific cellular components along with their corresponding images degraded by specific optics and electronics. The user can then apply image analysis methods to such simulated image data. The analysis results (such as segmentation or measurement results) can be compared with ground truth derived from input object digital phantoms (or measurements on them). In this way, image analysis methods can be compared with each other and their quality (based on the difference from ground truth) can be computed. We have also evaluated the plausibility of the synthetic images, measured by their similarity to real image data. We have tested several similarity criteria such as visual comparison, intensity histograms, central moments, frequency analysis, entropy, and 3D Haralick features. The results indicate a high degree of similarity between real and simulated image data.

An automated method for cell detection in zebrafish

Quantification of cells is a critical step towards the assessment of cell fate in neurological disease or developmental models. Here, we present a novel cell detection method for the automatic quantification of zebrafish neuronal cells, including primary motor neurons, Rohon-Beard neurons, and retinal cells. Our method consists of four steps. First, a diffused gradient vector field is produced. Subsequently, the orientations and magnitude information of diffused gradients are accumulated, and a response image is computed. In the third step, we perform non-maximum suppression on the response image and identify the detection candidates. In the fourth and final step the detected objects are grouped into clusters based on their color information. Using five different datasets depicting zebrafish cells, we show that our method consistently displays high sensitivity and specificity of over 95%. Our results demonstrate the general applicability of this method to different data samples, including nuclear staining, immunohistochemistry, and cell death detection.