Nonrigid registration of 3-d multichannel microscopy images of cell nuclei

A guide for single-particle chromatin tracking in live cell nuclei

The emergence of labeling strategies and live cell imaging methods enables the imaging of chromatin in living cells at single digit nanometer resolution as well as milliseconds temporal resolution. These technical breakthroughs revolutionize our understanding of chromatin structure, dynamics and functions. Single molecule tracking algorithms are usually preferred to quantify the movement of these intranucleus elements to interpret the spatiotemporal evolution of the chromatin. In this review, we will first summarize the fluorescent labeling strategy of chromatin in live cells which will be followed by a sys-tematic comparison of live cell imaging instrumentation. With the proper microscope, we will discuss the image analysis pipelines to extract the biophysical properties of the chromatin. Finally, we expect to give practical suggestions to broad biologists on how to select methods and link to the model properly according to different investigation pur-poses. This article is protected by copyright. All rights reserved.

A Global Method for Non-Rigid Registration of Cell Nuclei in Live Cell Time-Lapse Images

Non-rigid registration of cell nuclei in time-lapse microscopy images can be achieved through estimating the deformation fields using optical flow methods. In contrast to local optical flow models employed in the existing non-rigid registration methods, we introduce approaches based on a global optical flow model. Our registration model consists of a data fidelity term and a regularization term. We compared different regularizers for the deformation fields and found that a convex quadratic function is more suitable than non-convex ones. To improve the robustness, we propose an adaptive weighting scheme based on the statistics of the noise in fluorescence microscopy images as well as a combined local-global scheme. Moreover, we extend the global method by exploiting high-order image features. The best suitable high-order features are determined through learning two generative image models, namely, fields of experts and convolutional Gaussian restricted Boltzmann machine, whose model formulations are both consistent with the assumption of high-order feature constancy in the registration model. Using multiple data sets of real 2D and 3D live cell microscopy image sequences as well as synthetic image data, we demonstrate that our proposed approach outperforms the previous methods in terms of both registration accuracy and computational efficiency.

Non-Rigid Contour-Based Registration of Cell Nuclei in 2-D Live Cell Microscopy Images Using a Dynamic Elasticity Model

The analysis of the pure motion of subnuclear structures without influence of the cell nucleus motion and deformation is essential in live cell imaging. In this paper, we propose a 2-D contour-based image registration approach for compensation of nucleus motion and deformation in fluorescence microscopy time-lapse sequences. The proposed approach extends our previous approach, which uses a static elasticity model to register cell images. Compared with that scheme, the new approach employs a dynamic elasticity model for the forward simulation of nucleus motion and deformation based on the motion of its contours. The contour matching process is embedded as a constraint into the system of equations describing the elastic behavior of the nucleus. This results in better performance in terms of the registration accuracy. Our approach was successfully applied to real live cell microscopy image sequences of different types of cells including image data that was specifically designed and acquired for evaluation of cell image registration methods. An experimental comparison with the existing contour-based registration methods and an intensity-based registration method has been performed. We also studied the dependence of the results on the choice of method parameters.

ViCAR: An Adaptive and Landmark-Free Registration of Time Lapse Image Data from Microfluidics Experiments

In order to understand gene function in bacterial life cycles, time lapse bioimaging is applied in combination with different marker protocols in so called microfluidics chambers (i.e., a multi-well plate). In one experiment, a series of T images is recorded for one visual field, with a pixel resolution of 60 nm/px. Any (semi-)automatic analysis of the data is hampered by a strong image noise, low contrast and, last but not least, considerable irregular shifts during the acquisition. Image registration corrects such shifts enabling next steps of the analysis (e.g., feature extraction or tracking). Image alignment faces two obstacles in this microscopic context: (a) highly dynamic structural changes in the sample (i.e., colony growth) and (b) an individual data set-specific sample environment which makes the application of landmarks-based alignments almost impossible. We present a computational image registration solution, we refer to as ViCAR: (Vi)sual (C)ues based (A)daptive (R)egistration, for such microfluidics experiments, consisting of (1) the detection of particular polygons (outlined and segmented ones, referred to as visual cues), (2) the adaptive retrieval of three coordinates throughout different sets of frames, and finally (3) an image registration based on the relation of these points correcting both rotation and translation. We tested ViCAR with different data sets and have found that it provides an effective spatial alignment thereby paving the way to extract temporal features pertinent to each resulting bacterial colony. By using ViCAR, we achieved an image registration with 99.9% of image closeness, based on the average rmsd of 4.10⁻² pixels, and superior results compared to a state of the art algorithm.

Diffeomorphic Multi-Frame Non-Rigid Registration of Cell Nuclei in 2D and 3D Live Cell Images

To gain a better understanding of cellular and molecular processes, it is important to quantitatively analyze the motion of subcellular particles in live cell microscopy image sequences. Since, generally, the subcellular particles move and cell nuclei move as well as deform, it is important to decouple the movement of particles from that of the cell nuclei using non-rigid registration methods. We have developed a diffeomorphic multi-frame approach for non-rigid registration of cell nuclei in 2D and 3D live cell fluorescence microscopy images. Our non-rigid registration approach is based on local optic flow estimation, exploits information from multiple consecutive image frames, and determines diffeomorphic transformations in the log-domain, which allows efficient computation of the inverse transformations. To register single images of an image sequence to a reference image, we use a temporally weighted mean image, which is constructed based on inverse transformations and multiple consecutive frames. Using multiple consecutive frames improves the registration accuracy compared to pairwise registration, and using a temporally weighted mean image significantly reduces the computation time compared with previous work. In addition, we use a flow boundary preserving method for regularization of computed deformation vector fields, which prevents from over-smoothing compared to standard Gaussian filtering. Our approach has been successfully applied to 2D and 3D synthetic as well as real live cell microscopy image sequences, and an experimental comparison with non-rigid pairwise, multi-frame, and temporal groupwise registration has been carried out.

Building cell models and simulations from microscope images

The use of fluorescence microscopy has undergone a major revolution over the past twenty years, both with the development of dramatic new technologies and with the widespread adoption of image analysis and machine learning methods. Many open source software tools provide the ability to use these methods in a wide range of studies, and many molecular and cellular phenotypes can now be automatically distinguished. This article presents the next major challenge in microscopy automation, the creation of accurate models of cell organization directly from images, and reviews the progress that has been made towards this challenge.

Multi-modal registration for correlative microscopy using image analogies

Correlative microscopy is a methodology combining the functionality of light microscopy with the high resolution of electron microscopy and other microscopy technologies for the same biological specimen. In this paper, we propose an image registration method for correlative microscopy, which is challenging due to the distinct appearance of biological structures when imaged with different modalities. Our method is based on image analogies and allows to transform images of a given modality into the appearance-space of another modality. Hence, the registration between two different types of microscopy images can be transformed to a mono-modality image registration. We use a sparse representation model to obtain image analogies. The method makes use of corresponding image training patches of two different imaging modalities to learn a dictionary capturing appearance relations. We test our approach on backscattered electron (BSE) scanning electron microscopy (SEM)/confocal and transmission electron microscopy (TEM)/confocal images. We perform rigid, affine, and deformable registration via B-splines and show improvements over direct registration using both mutual information and sum of squared differences similarity measures to account for differences in image appearance.

A motion correction framework for time series sequences in microscopy images

With the advent of in vivo laser scanning fluorescence microscopy techniques, time-series and three-dimensional volumes of living tissue and vessels at micron scales can be acquired to firmly analyze vessel architecture and blood flow. Analysis of a large number of image stacks to extract architecture and track blood flow manually is cumbersome and prone to observer bias. Thus, an automated framework to accomplish these analytical tasks is imperative. The first initiative toward such a framework is to compensate for motion artifacts manifest in these microscopy images. Motion artifacts in in vivo microscopy images are caused by respiratory motion, heart beats, and other motions from the specimen. Consequently, the amount of motion present in these images can be large and hinders further analysis of these images. In this article, an algorithmic framework for the correction of time-series images is presented. The automated algorithm is comprised of a rigid and a nonrigid registration step based on shape contexts. The framework performs considerably well on time-series image sequences of the islets of Langerhans and provides for the pivotal step of motion correction in the further automatic analysis of microscopy images.

RAMTaB: robust alignment of multi-tag bioimages

Background

In recent years, new microscopic imaging techniques have evolved to allow us to visualize several different proteins (or other biomolecules) in a visual field. Analysis of protein co-localization becomes viable because molecules can interact only when they are located close to each other. We present a novel approach to align images in a multi-tag fluorescence image stack. The proposed approach is applicable to multi-tag bioimaging systems which (a) acquire fluorescence images by sequential staining and (b) simultaneously capture a phase contrast image corresponding to each of the fluorescence images. To the best of our knowledge, there is no existing method in the literature, which addresses simultaneous registration of multi-tag bioimages and selection of the reference image in order to maximize the overall overlap between the images.

Methodology/Principal Findings

We employ a block-based method for registration, which yields a confidence measure to indicate the accuracy of our registration results. We derive a shift metric in order to select the Reference Image with Maximal Overlap (RIMO), in turn minimizing the total amount of non-overlapping signal for a given number of tags. Experimental results show that the Robust Alignment of Multi-Tag Bioimages (RAMTaB) framework is robust to variations in contrast and illumination, yields sub-pixel accuracy, and successfully selects the reference image resulting in maximum overlap. The registration results are also shown to significantly improve any follow-up protein co-localization studies.

Conclusions

For the discovery of protein complexes and of functional protein networks within a cell, alignment of the tag images in a multi-tag fluorescence image stack is a key pre-processing step. The proposed framework is shown to produce accurate alignment results on both real and synthetic data. Our future work will use the aligned multi-channel fluorescence image data for normal and diseased tissue specimens to analyze molecular co-expression patterns and functional protein networks.

CellOrganizer: Image-derived models of subcellular organization and protein distribution

This chapter describes approaches for learning models of subcellular organization from images. The primary utility of these models is expected to be from incorporation into complex simulations of cell behaviors. Most current cell simulations do not consider spatial organization of proteins at all, or treat each organelle type as a single, idealized compartment. The ability to build generative models for all proteins in a proteome and use them for spatially accurate simulations is expected to improve the accuracy of models of cell behaviors. A second use, of potentially equal importance, is expected to be in testing and comparing software for analyzing cell images. The complexity and sophistication of algorithms used in cell-image-based screens and assays (variously referred to as high-content screening, high-content analysis, or high-throughput microscopy) is continuously increasing, and generative models can be used to produce images for testing these algorithms in which the expected answer is known.

INTERACTIVE REGISTRATION OF INTRACELLULAR VOLUMES WITH RADIAL BASIS FUNCTIONS

We propose a novel approach to 3D image registration of intracellular volumes. The approach extends a standard image registration framework to the curved cell geometry. An intracellular volume is mapped onto another intracellular domain by using two pairs of point set surfaces approximating their nuclear and plasma membranes. The mapping function consists of the affine transformation, tetrahedral barycentric interpolation, and least-squares formulation of radial basis functions for extracted cell geometry features. An interactive volume registration system is also developed based on our approach. We demonstrate that our approach is capable of creating cell models containing multiple organelles from observed data of living cells.

Nonrigid registration of 2-D and 3-D dynamic cell nuclei images for improved classification of subcellular particle motion

The observed motion of subcellular particles in fluorescence microscopy image sequences of live cells is generally a superposition of the motion and deformation of the cell and the motion of the particles. Decoupling the two types of movements to enable accurate classification of the particle motion requires the application of registration algorithms. We have developed an intensity-based approach for nonrigid registration of multichannel microscopy image sequences of cell nuclei. First, based on 3-D synthetic images we demonstrate that cell nucleus deformations change the observed motion types of particles and that our approach allows to recover the original motion. Second, we have successfully applied our approach to register 2-D and 3-D real microscopy image sequences. A quantitative experimental comparison with previous approaches for nonrigid registration of cell microscopy has also been performed.

Tracking and quantitative analysis of dynamic movements of cells and particles

Understanding complex cellular processes requires investigating the underlying mechanisms within a spatiotemporal context. Although cellular processes are dynamic in nature, most studies in molecular cell biology are based on fixed specimens, for example, using immunocytochemistry or fluorescence in situ hybridization (FISH). However, breakthroughs in fluorescence microscopy imaging techniques, in particular, the discovery of green fluorescent protein (GFP) and its spectral variants, have facilitated the study of a wide range of dynamic processes by allowing nondestructive labeling of target structures in living cells. In addition, the tremendous improvements in spatial and temporal resolution of light microscopes now allow cellular processes to be analyzed in unprecedented detail. These state-of-the-art imaging technologies, however, provide a huge amount of digital image data. To cope with the enormous amount of image data and to extract reproducible as well as quantitative information, computer-based image analysis is required. In this article, we describe methods for computer-based analysis of multidimensional live cell microscopy images and their application to study the dynamics of cells and particles. First, we sketch a general workflow for quantitative analysis of live cell images. Then, we detail computational methods for automatic image analysis comprising image preprocessing, segmentation, registration, tracking, and classification. We conclude with a discussion of quantitative analysis and systems biology.

Automated analysis of time-lapse fluorescence microscopy images: from live cell images to intracellular foci

MOTIVATION

Complete, accurate and reproducible analysis of intracellular foci from fluorescence microscopy image sequences of live cells requires full automation of all processing steps involved: cell segmentation and tracking followed by foci segmentation and pattern analysis. Integrated systems for this purpose are lacking.

RESULTS

Extending our previous work in cell segmentation and tracking, we developed a new system for performing fully automated analysis of fluorescent foci in single cells. The system was validated by applying it to two common tasks: intracellular foci counting (in DNA damage repair experiments) and cell-phase identification based on foci pattern analysis (in DNA replication experiments). Experimental results show that the system performs comparably to expert human observers. Thus, it may replace tedious manual analyses for the considered tasks, and enables high-content screening.

AVAILABILITY AND IMPLEMENTATION

The described system was implemented in MATLAB (The MathWorks, Inc., USA) and compiled to run within the MATLAB environment. The routines together with four sample datasets are available at http://celmia.bigr.nl/. The software is planned for public release, free of charge for non-commercial use, after publication of this article.

Theta rotation and serial registration of light microscopical images using a novel camera rotating device

An electromechanical video camera coupler was developed to rotate a light microscope field of view (FOV) in real time without the need to physically rotate the stage or specimen. The device, referred to as the Camera Thetarotator, rotated microscopical views 240 degrees to assist microscopists to orient specimens within the FOV prior to image capture. The Camera Thetarotator eliminated the effort and artifacts created when rotating photomicrographs using conventional graphics software. The Camera Thetarotator could also be used to semimanually register a dataset of histological sections for three-dimensional (3D) reconstruction by superimposing the transparent, real-time FOV to the previously captured section in the series. When compared to Fourier-based software registration, alignment of serial sections using the Camera Thetarotator was more exact, resulting in more accurate 3D reconstructions with no computer-generated null space. When software-based registration was performed after prealigning sections with the Camera Thetarotator, registration was further enhanced. The Camera Thetarotator expanded microscopical viewing and digital photomicrography and provided a novel, accurate registration method for 3D reconstruction. The Camera Thetarotator would also be useful for performing automated microscopical functions necessary for telemicroscopy, high-throughput image acquisition and analysis, and other light microscopy applications.

Automated image analysis for high-content screening and analysis

The field of high-content screening and analysis consists of a set of methodologies for automated discovery in cell biology and drug development using large amounts of image data. In most cases, imaging is carried out by automated microscopes, often assisted by automated liquid handling and cell culture. Image processing, computer vision, and machine learning are used to automatically process high-dimensional image data into meaningful cell biological results. The key is creating automated analysis pipelines typically consisting of 4 basic steps: (1) image processing (normalization, segmentation, tracing, tracking), (2) spatial transformation to bring images to a common reference frame (registration), (3) computation of image features, and (4) machine learning for modeling and interpretation of data. An overview of these image analysis tools is presented here, along with brief descriptions of a few applications.

Tracking in cell and developmental biology

The past decade has seen an unprecedented data explosion in biology. It has become evident that in order to take full advantage of the potential wealth of information hidden in the data produced by even a single experiment, visual inspection and manual analysis are no longer adequate. To ensure efficiency, consistency, and completeness in data processing and analysis, computational tools are essential. Of particular importance to many modern live-cell imaging experiments is the ability to automatically track and analyze the motion of objects in time-lapse microscopy images. This article surveys the recent literature in this area. Covering all scales of microscopic observation, from cells, down to molecules, and up to entire organisms, it discusses the latest trends and successes in the development and application of computerized tracking methods in cell and developmental biology.