Tracking of migrating cells under phase-contrast video microscopy with combined mean-shift processes

Digital Holographic Microscopy to Assess Cell Behavior

Digital holographic microscopy is an imaging technique particularly well suited to the study of living cells in culture, as no labeling is required and computed phase maps produce high contrast, quantitative pixel information. A full experiment involves instrument calibration, cell culture quality checks, selection and setup of imaging chambers, a sampling plan, image acquisition, phase and amplitude map reconstruction, and parameter map post-processing to extract information about cell morphology and/or motility. Each step is described below, focusing on results from imaging four human cell lines. Several post-processing approaches are detailed, with an aim of tracking individual cells and dynamics of cell populations.

Adaptive tracking algorithm for trajectory analysis of cells and layer-by-layer assessment of motility dynamics

Tracking biological objects such as cells or subcellular components imaged with time-lapse microscopy enables us to understand the molecular principles about the dynamics of cell behaviors. However, automatic object detection, segmentation and extracting trajectories remain as a rate-limiting step due to intrinsic challenges of video processing. This paper presents an adaptive tracking algorithm (Adtari) that automatically finds the optimum search radius and cell linkages to determine trajectories in consecutive frames. A critical assumption in most tracking studies is that displacement remains unchanged throughout the movie and cells in a few frames are usually analyzed to determine its magnitude. Tracking errors and inaccurate association of cells may occur if the user does not correctly evaluate the value or prior knowledge is not present on cell movement. The key novelty of our method is that minimum intercellular distance and maximum displacement of cells between frames are dynamically computed and used to determine the threshold distance. Since the space between cells is highly variable in a given frame, our software recursively alters the magnitude to determine all plausible matches in the trajectory analysis. Our method therefore eliminates a major preprocessing step where a constant distance was used to determine the neighbor cells in tracking methods. Cells having multiple overlaps and splitting events were further evaluated by using the shape attributes including perimeter, area, ellipticity and distance. The features were applied to determine the closest matches by minimizing the difference in their magnitudes. Finally, reporting section of our software were used to generate instant maps by overlaying cell features and trajectories. Adtari was validated by using videos with variable signal-to-noise, contrast ratio and cell density. We compared the adaptive tracking with constant distance and other methods to evaluate performance and its efficiency. Our algorithm yields reduced mismatch ratio, increased ratio of whole cell track, higher frame tracking efficiency and allows layer-by-layer assessment of motility to characterize single-cells. Adaptive tracking provides a reliable, accurate, time efficient and user-friendly open source software that is well suited for analysis of 2D fluorescence microscopy video datasets.

TAIM: Tool for Analyzing Root Images to Calculate the Infection Rate of Arbuscular Mycorrhizal Fungi

Arbuscular mycorrhizal fungi (AMF) infect plant roots and are hypothesized to improve plant growth. Recently, AMF is now available for axenic culture. Therefore, AMF is expected to be used as a microbial fertilizer. To evaluate the usefulness of AMF as a microbial fertilizer, we need to investigate the relationship between the degree of root colonization of AMF and plant growth. The method popularly used for calculation of the degree of root colonization, termed the magnified intersections method, is performed manually and is too labor-intensive to enable an extensive survey to be undertaken. Therefore, we automated the magnified intersections method by developing an application named "Tool for Analyzing root images to calculate the Infection rate of arbuscular Mycorrhizal fungi: TAIM." TAIM is a web-based application that calculates the degree of AMF colonization from images using automated computer vision and pattern recognition techniques. Experimental results showed that TAIM correctly detected sampling areas for calculation of the degree of infection and classified the sampling areas with 87.4% accuracy. TAIM is publicly accessible at http://taim.imlab.jp/.

Deep Reinforcement Learning-Based Progressive Sequence Saliency Discovery Network for Mitosis Detection In Time-Lapse Phase-Contrast Microscopy Images

Mitosis detection plays an important role in the analysis of cell status and behavior and is therefore widely utilized in many biological research and medical applications. In this article, we propose a deep reinforcement learning-based progressive sequence saliency discovery network (PSSD)for mitosis detection in time-lapse phase contrast microscopy images. By discovering the salient frames when cell state changes in the sequence, PSSD can more effectively model the mitosis process for mitosis detection. We formulate the discovery of salient frames as a Markov Decision Process (MDP)that progressively adjusts the selection positions of salient frames in the sequence, and further leverage deep reinforcement learning to learn the policy in the salient frame discovery process. The proposed method consists of two parts: 1)the saliency discovery module that selects the salient frames from the input cell image sequence by progressively adjusting the selection positions of salient frames; 2)the mitosis identification module that takes a sequence of salient frames and performs temporal information fusion for mitotic sequence classification. Since the policy network of the saliency discovery module is trained under the guidance of the mitosis identification module, PSSD can comprehensively explore the salient frames that are beneficial for mitosis detection. To our knowledge, this is the first work to implement deep reinforcement learning to the mitosis detection problem. In the experiment, we evaluate the proposed method on the largest mitosis detection dataset, C2C12-16. Experiment results show that compared with the state-of-the-arts, the proposed method can achieve significant improvement for both mitosis identification and temporal localization on C2C12-16.

An Automated Cell Tracking Approach With Multi-Bernoulli Filtering and Ant Colony Labor Division

In this article, we take as inspiration the labor division into scouts and workers in an ant colony and propose a novel approach for automated cell tracking in the framework of multi-Bernoulli random finite sets. To approximate the Bernoulli parameter sets, we first define an existence probability of an ant colony as well as its discrete density distribution. During foraging, the behavior of scouts is modeled as a chaotic movement to produce a set of potential candidates. Afterwards, a group of workers, i.e., a worker ant colony, is recruited for each candidate, which then embark on gathering heuristic information in a self-organized way. Finally, the pheromone field is formed by the corresponding worker ant colony, from which the Bernoulli parameter is derived and the state of the cell is estimated accordingly to be associated with the existing tracks. Performance comparisons with other previous approaches are conducted on both simulated and real cell image sequences and show the superiority of this algorithm.

A Joint Tracking Approach via Ant Colony Evolution for Quantitative Cell Cycle Analysis

In this paper, we use an ant colony heuristic method to tackle the integration of data association and state estimation in the presence of cell mitosis, morphological change and uncertainty of measurement. Our approach first models the scouting behavior of an unlabeled ant colony as a chaotic process to generate a set of cell candidates in the current frame, then a labeled ant colony foraging process is modeled to construct an interframe matching between previously estimated cell states and current cell candidates through minimizing the optimal sub-pattern assignment metric for track (OSPA-T). The states of cells in the current frame are finally estimated using labeled ant colonies via a multi-Bernoulli parameter set approximated by individual food pheromone fields and heuristic information within the same region of support, the resulting trail pheromone fields over frames constitutes the cell lineage trees of the tracks. A four-stage track recovery strategy is proposed to monitor the history of all established tracks to reconstruct broken tracks in a computationally economic way. The labeling method used in this work is an improvement on previous techniques. The method has been evaluated on publicly available, challenging cell image sequences, and a satisfied performance improvement is achieved in contrast to the state-of-the-art methods.

A Novel Method for Effective Cell Segmentation and Tracking in Phase Contrast Microscopic Images

Cell migration plays an important role in the identification of various diseases and physiological phenomena in living organisms, such as cancer metastasis, nerve development, immune function, wound healing, and embryo formulation and development. The study of cell migration with a real-time microscope generally takes several hours and involves analysis of the movement characteristics by tracking the positions of cells at each time interval in the images of the observed cells. Morphological analysis considers the shapes of the cells, and a phase contrast microscope is used to observe the shape clearly. Therefore, we developed a segmentation and tracking method to perform a kinetic analysis by considering the morphological transformation of cells. The main features of the algorithm are noise reduction using a block-matching 3D filtering method, k-means clustering to mitigate the halo signal that interferes with cell segmentation, and the detection of cell boundaries via active contours, which is an excellent way to detect boundaries. The reliability of the algorithm developed in this study was verified using a comparison with the manual tracking results. In addition, the segmentation results were compared to our method with unsupervised state-of-the-art methods to verify the proposed segmentation process. As a result of the study, the proposed method had a lower error of less than 40% compared to the conventional active contour method.

Spatio-Temporal Mitosis Detection in Time-Lapse Phase-Contrast Microscopy Image Sequences: A Benchmark

In this paper, we report the results of the first international contest on mitosis detection in phase-contrast microscopy image sequences (https://www.iti-tju.org/mitosisdetection), which was held at the workshop of computer vision for microscopy image analysis (CVMI) in CVPR 2019. This contest aims to promote research on spatiotemporal mitosis detection under microscopy images. In this contest, we released a large-scale time-lapse phase-contrast microscopy image dataset (C2C12-16) for the mitosis detection task. Compared with the previous popular datasets (e.g., C2C12, C3H10), C2C12-16 contains more annotated mitotic events and more diverse cell culture environments. A total of ten different mitosis detection methods were submitted in the contest and evaluated on the test sets of four different cell culture environments in C2C12-16. In this benchmark, we describe all methods and conduct a thorough analysis based on their performances and discuss a feasible direction for mitosis detection. To the best of our knowledge, this is the first benchmark for the mitosis detection problem using a time-lapse phase-contrast microscopy spatiotemporal image sequence model.

Faster Mean-shift: GPU-accelerated clustering for cosine embedding-based cell segmentation and tracking

Recently, single-stage embedding based deep learning algorithms gain increasing attention in cell segmentation and tracking. Compared with the traditional "segment-then-associate" two-stage approach, a single-stage algorithm not only simultaneously achieves consistent instance cell segmentation and tracking but also gains superior performance when distinguishing ambiguous pixels on boundaries and overlaps. However, the deployment of an embedding based algorithm is restricted by slow inference speed (e.g., ≈1-2 min per frame). In this study, we propose a novel Faster Mean-shift algorithm, which tackles the computational bottleneck of embedding based cell segmentation and tracking. Different from previous GPU-accelerated fast mean-shift algorithms, a new online seed optimization policy (OSOP) is introduced to adaptively determine the minimal number of seeds, accelerate computation, and save GPU memory. With both embedding simulation and empirical validation via the four cohorts from the ISBI cell tracking challenge, the proposed Faster Mean-shift algorithm achieved 7-10 times speedup compared to the state-of-the-art embedding based cell instance segmentation and tracking algorithm. Our Faster Mean-shift algorithm also achieved the highest computational speed compared to other GPU benchmarks with optimized memory consumption. The Faster Mean-shift is a plug-and-play model, which can be employed on other pixel embedding based clustering inference for medical image analysis. (Plug-and-play model is publicly available: https://github.com/masqm/Faster-Mean-Shift).

Determining the effects of trastuzumab, cetuximab and afatinib by phosphoprotein, gene expression and phenotypic analysis in gastric cancer cell lines

BACKGROUND

Gastric cancer is the fifth most frequently diagnosed cancer and the third leading cause of cancer death worldwide. The molecular mechanisms of action for anti-HER-family drugs in gastric cancer cells are incompletely understood. We compared the molecular effects of trastuzumab and the other HER-family targeting drugs cetuximab and afatinib on phosphoprotein and gene expression level to gain insights into the regulated pathways. Moreover, we intended to identify genes involved in phenotypic effects of anti-HER therapies.

METHODS

A time-resolved analysis of downstream intracellular kinases following EGF, cetuximab, trastuzumab and afatinib treatment was performed by Luminex analysis in the gastric cancer cell lines Hs746T, MKN1, MKN7 and NCI-N87. The changes in gene expression after treatment of the gastric cancer cell lines with EGF, cetuximab, trastuzumab or afatinib for 4 or 24 h were analyzed by RNA sequencing. Significantly enriched pathways and gene ontology terms were identified by functional enrichment analysis. Furthermore, effects of trastuzumab and afatinib on cell motility and apoptosis were analyzed by time-lapse microscopy and western blot for cleaved caspase 3.

RESULTS

The Luminex analysis of kinase activity revealed no effects of trastuzumab, while alterations of AKT1, MAPK3, MEK1 and p70S6K1 activations were observed under cetuximab and afatinib treatment. On gene expression level, cetuximab mainly affected the signaling pathways, whereas afatinib had an effect on both signaling and cell cycle pathways. In contrast, trastuzumab had little effects on gene expression. Afatinib reduced average speed in MKN1 and MKN7 cells and induced apoptosis in NCI-N87 cells. Following treatment with afatinib, a list of 14 genes that might be involved in the decrease of cell motility and a list of 44 genes that might have a potential role in induction of apoptosis was suggested. The importance of one of these genes (HBEGF) as regulator of motility was confirmed by knockdown experiments.

CONCLUSIONS

Taken together, we described the different molecular effects of trastuzumab, cetuximab and afatinib on kinase activity and gene expression. The phenotypic changes following afatinib treatment were reflected by altered biological functions indicated by overrepresentation of gene ontology terms. The importance of identified genes for cell motility was validated in case of HBEGF.

Fully-automatic deep learning-based analysis for determination of the invasiveness of breast cancer cells in an acoustic trap

A single-beam acoustic trapping technique has been shown to be very useful for determining the invasiveness of suspended breast cancer cells in an acoustic trap with a manual calcium analysis method. However, for the rapid translation of the technology into the clinic, the development of an efficient/accurate analytical method is needed. We, therefore, develop a fully-automatic deep learning-based calcium image analysis algorithm for determining the invasiveness of suspended breast cancer cells using a single-beam acoustic trapping system. The algorithm allows to segment cells, find trapped cells, and quantify their calcium changes over time. For better segmentation of calcium fluorescent cells even with vague boundaries, a novel deep learning architecture with multi-scale/multi-channel convolution operations (MM-Net) is devised and constructed by a target inversion training method. The MM-Net outperforms other deep learning models in the cell segmentation. Also, a detection/quantification algorithm is developed and implemented to automatically determine the invasiveness of a trapped cell. For the evaluation of the algorithm, it is applied to quantify the invasiveness of breast cancer cells. The results show that the algorithm offers similar performance to the manual calcium analysis method for determining the invasiveness of cancer cells, suggesting that it may serve as a novel tool to automatically determine the invasiveness of cancer cells with high-efficiency.

MET as resistance factor for afatinib therapy and motility driver in gastric cancer cells

The therapeutic options for advanced gastric cancer are still limited. Several drugs targeting the epidermal growth factor receptor family have been developed. So far, the HER2 antibody trastuzumab is the only drug targeting the HER-family that is available to gastric cancer patients. The pan-HER inhibitor afatinib is currently investigated in clinical trials and shows promising results in cell culture experiments and patient-derived xenograft (PDX) models. However, some cell lines do not respond to afatinib treatment. The determination of resistance factors in these cell lines can help to find the best treatment option for gastric cancer patients. In this study, we analyzed the role of MET as a resistance factor for afatinib therapy in a gastric cancer cell line. MET expression in afatinib-resistant MET-amplified Hs746T cells was reduced by means of siRNA transfection. The effects of MET knockdown on signal transduction, cell proliferation and motility were examined. In addition to the manual assessment of cell motility, a computational motility analysis involving parameters such as (approximate) average speed, displacement entropy or radial effectiveness was realized. Moreover, the impact of afatinib was compared between MET knockdown cells and control cells. MET knockdown in Hs746T cells resulted in impaired signal transduction and reduced cell proliferation and motility. Moreover, the afatinib resistance of Hs746T cells was reversed after MET knockdown. Therefore, the amplification of MET is confirmed as a resistance factor in gastric cancer cells. Whether MET is a useful resistance marker for afatinib therapy or other HER-targeting drugs in patients should be investigated in clinical trials.

Unsupervised Two-Path Neural Network for Cell Event Detection and Classification Using Spatiotemporal Patterns

Automatic event detection in cell videos is essential for monitoring cell populations in biomedicine. Deep learning methods have advantages over traditional approaches for cell event detection due to their ability to capture more discriminative features of cellular processes. Supervised deep learning methods, however, are inherently limited due to the scarcity of annotated data. Unsupervised deep learning methods have shown promise in general (non-cell) videos because they can learn the visual appearance and motion of regularly occurring events. Cell videos, however, can have rapid, irregular changes in cell appearance and motion, such as during cell division and death, which are often the events of most interest. We propose a novel unsupervised two-path input neural network architecture to capture these irregular events with three key elements: 1) a visual encoding path to capture regular spatiotemporal patterns of observed objects with convolutional long short-term memory units; 2) an event detection path to extract information related to irregular events with max-pooling layers; and 3) integration of the hidden states of the two paths to provide a comprehensive representation of the video that is used to simultaneously locate and classify cell events. We evaluated our network in detecting cell division in densely packed stem cells in phase-contrast microscopy videos. Our unsupervised method achieved higher or comparable accuracy to standard and state-of-the-art supervised methods.

Schlieren two-photon microscopy for phase-contrast imaging

While simultaneous phase-contrast and two-photon fluorescence imaging in microscopy can bring abundant biomedical information, it is difficult to retrieve phase information from conventional two-photon microscopes. To realize low-cost, in situ phase-contrast and two-photon fluorescence imaging, we propose Schlieren two-photon microscopy, a method that implements phase-contrast imaging on two-photon microscopes. This method involves spatially modulated fluorescence plates, which are made of two-photon fluorescence dyes or upconversion nanoparticles. We demonstrate that the fluorescence intensity fluctuation reflects the phase gradients of the specimen via theoretical analysis, simulations, and experiments. The proposed method is fully compatible with commercial two-photon microscopes, thus enabling widespread applications in live tissue imaging.

The application of convolution neural network based cell segmentation during cryopreservation

For most of the cells, water permeability and plasma membrane properties play a vital role in the optimal protocol for successful cryopreservation. Measuring the water permeability of cells during subzero temperature is essential. So far, there is no perfect segmentation technique to be used for the image processing task on subzero temperature accurately. The ice formation and variable background during freezing posed a significant challenge for most of the conventional segmentation algorithms. Thus, a robust and accurate segmentation approach that can accurately extract cells from extracellular ice that surrounding the cell boundary is needed. Therefore, we propose a convolutional neural network (CNN) architecture similar to U-Net but differs from those conventionally used in computer vision to extract all the cell boundaries as they shrank in the engulfing ice. The images used was obtained from the cryo-stage microscope, and the data was validated using the Hausdorff distance, means ± standard deviation for different methods of segmentation result using the CNN model. The experimental results prove that the typical CNN model extracts cell borders contour from the background in its subzero state more coherent and effective as compared to other traditional segmentation approaches.

MobilityAnalyser: A novel approach for automatic quantification of cell mobility on periodic patterned substrates using brightfield microscopy images

BACKGROUND AND OBJECTIVE

Surface topography of biomaterials has been shown to have an effect on cells behaviour. Cell-material interactions can be visually characterized by assessing both cell shape and spreading at initial time-points and, its migration patterns, as a response to the underlying topography. Whilst many have reported the study of cell migration and shape with fluorescence labelling, the focus on evaluating cells response to surface topography is to observe, under real-time conditions, interactions between cells and surfaces. In this manuscript we present a novel approach to automatically detect and remove periodic background patterns in brightfield microscopy images in order to perform automatic cell mobility analysis.

METHODS

The developed software, MobilityAnalyser, performs automatic tracking of unmarked cells and allows the user to manually correct any incorrectly detected or tracked cell. Human Mesenchymal Stem Cells (hMSCs) trajectory, migration distance, velocity and persistence were evaluated over line and pillar micropatterned SiO₂ films and on a flat SiO₂ control substrate.

RESULTS

The developed software proved to be effective in automatically removing background patterns of both line and pillar shapes and in performing cell detection and tracking. MobilityAnalyser accurately measured cell mobility in a fraction of the time required for manual analysis and eliminated user subjectivity. The results obtained with the software confirmed how different topographies affect cell trajectory, migration pathways and velocities, with a statistically significant increase for micropatterned surfaces, when compared with the flat control. The persistence parameter also proved the influence of both patterns on the directionality of cell movement.

CONCLUSIONS

MobilityAnalyser is an automatic tool that removes periodic background patterns, detects and tracks cells, and provides cell mobility parameters that characterize the response of cells to different surface topographies. The software is freely available at: https://drive.google.com/open?id=1Fbb321ogLD19SlRjceMETNUqDHgpeBPl.

Microrheology, advances in methods and insights

Microrheology is an emerging technique that probes mechanical response of soft material at micro-scale. Generally, microrheology technique can be divided into active and passive versions. During last two decades, extensive efforts have been paid to improve both the experiment techniques and data analysis methods, especially about how to link consequential particle positions into trajectories. We review the recent advances in microrheology, including improvements in labeling, imaging, data acquiring, data processing and data interpretation. Some of the recent insights in soft matter and living systems gained by using this technique are given. Before these, we also give a very brief description of the basic principles of both active and passive microrheology techniques, and some details about optical particle tracking and DWS.

Multicell migration tracking within angiogenic networks by deep learning-based segmentation and augmented Bayesian filtering

Cell migration is a key feature for living organisms. Image analysis tools are useful in studying cell migration in three-dimensional (3-D) in vitro environments. We consider angiogenic vessels formed in 3-D microfluidic devices (MFDs) and develop an image analysis system to extract cell behaviors from experimental phase-contrast microscopy image sequences. The proposed system initializes tracks with the end-point confocal nuclei coordinates. We apply convolutional neural networks to detect cell candidates and combine backward Kalman filtering with multiple hypothesis tracking to link the cell candidates at each time step. These hypotheses incorporate prior knowledge on vessel formation and cell proliferation rates. The association accuracy reaches 86.4% for the proposed algorithm, indicating that the proposed system is able to associate cells more accurately than existing approaches. Cell culture experiments in 3-D MFDs have shown considerable promise for improving biology research. The proposed system is expected to be a useful quantitative tool for potential microscopy problems of MFDs.

Dancing Styles of Collective Cell Migration: Image-Based Computational Analysis of JRAB/MICAL-L2

Collective cell migration is observed during morphogenesis, angiogenesis, and wound healing, and this type of cell migration also contributes to efficient metastasis in some kinds of cancers. Because collectively migrating cells are much better organized than a random assemblage of individual cells, there seems to be a kind of order in migrating clusters. Extensive research has identified a large number of molecules involved in collective cell migration, and these factors have been analyzed using dramatic advances in imaging technology. To date, however, it remains unclear how myriad cells are integrated as a single unit. Recently, we observed unbalanced collective cell migrations that can be likened to either precision dancing or awa-odori, Japanese traditional dancing similar to the style at Rio Carnival, caused by the impairment of the conformational change of JRAB/MICAL-L2. This review begins with a brief history of image-based computational analyses on cell migration, explains why quantitative analysis of the stylization of collective cell behavior is difficult, and finally introduces our recent work on JRAB/MICAL-L2 as a successful example of the multidisciplinary approach combining cell biology, live imaging, and computational biology. In combination, these methods have enabled quantitative evaluations of the “dancing style” of collective cell migration.

Automated tracking and quantification of angiogenic vessel formation in 3D microfluidic devices

Angiogenesis, the growth of new blood vessels from pre-existing vessels, is a critical step in cancer invasion. Better understanding of the angiogenic mechanisms is required to develop effective antiangiogenic therapies for cancer treatment. We culture angiogenic vessels in 3D microfluidic devices under different Sphingosin-1-phosphate (S1P) conditions and develop an automated vessel formation tracking system (AVFTS) to track the angiogenic vessel formation and extract quantitative vessel information from the experimental time-lapse phase contrast images. The proposed AVFTS first preprocesses the experimental images, then applies a distance transform and an augmented fast marching method in skeletonization, and finally implements the Hungarian method in branch tracking. When applying the AVFTS to our experimental data, we achieve 97.3% precision and 93.9% recall by comparing with the ground truth obtained from manual tracking by visual inspection. This system enables biologists to quantitatively compare the influence of different growth factors. Specifically, we conclude that the positive S1P gradient increases cell migration and vessel elongation, leading to a higher probability for branching to occur. The AVFTS is also applicable to distinguish tip and stalk cells by considering the relative cell locations in a branch. Moreover, we generate a novel type of cell lineage plot, which not only provides cell migration and proliferation histories but also demonstrates cell phenotypic changes and branch information.

T-Time: A data repository of T cell and calcium release-activated calcium channel activation imagery

Background

A fundamental understanding of live-cell dynamics is necessary in order to advance scientific techniques and personalized medicine. For this understanding to be possible, image processing techniques, probes, tracking algorithms and many other methodologies must be improved. Currently there are no large open-source datasets containing live-cell imaging to act as a standard for the community. As a result, researchers cannot evaluate their methodologies on an independent benchmark or leverage such a dataset to formulate scientific questions.

Findings

Here we present T-Time, the largest free and publicly available data set of T cell phase contrast imagery designed with the intention of furthering live-cell dynamics research. T-Time consists of over 40 GB of imagery data, and includes annotations derived from these images using a custom T cell identification and tracking algorithm. The data set contains 71 time-lapse sequences containing T cell movement and calcium release activated calcium channel activation, along with 50 time-lapse sequences of T cell activation and T reg interactions. The database includes a user-friendly web interface, summary information on the time-lapse images, and a mechanism for users to download tailored image datasets for their own research. T-Time is freely available on the web at http://ttime.mlatlab.org.

Conclusions

T-Time is a novel data set of T cell images and associated metadata. It allows users to study T cell interaction and activation.

Trajectory energy minimization for cell growth tracking and genealogy analysis

Cell growth experiments with a microfluidic device produce large-scale time-lapse image data, which contain important information on cell growth and patterns in their genealogy. To extract such information, we propose a scheme to segment and track bacterial cells automatically. In contrast with most published approaches, which often split segmentation and tracking into two independent procedures, we focus on designing an algorithm that describes cell properties evolving between consecutive frames by feeding segmentation and tracking results from one frame to the next one. The cell boundaries are extracted by minimizing the distance regularized level set evolution (DRLSE) model. Each individual cell was identified and tracked by identifying cell septum and membrane as well as developing a trajectory energy minimization function along time-lapse series. Experiments show that by applying this scheme, cell growth and division can be measured automatically. The results show the efficiency of the approach when testing on different datasets while comparing with other existing algorithms. The proposed approach demonstrates great potential for large-scale bacterial cell growth analysis.

Mathematical imaging methods for mitosis analysis in live-cell phase contrast microscopy

In this paper we propose a workflow to detect and track mitotic cells in time-lapse microscopy image sequences. In order to avoid the requirement for cell lines expressing fluorescent markers and the associated phototoxicity, phase contrast microscopy is often preferred over fluorescence microscopy in live-cell imaging. However, common specific image characteristics complicate image processing and impede use of standard methods. Nevertheless, automated analysis is desirable due to manual analysis being subjective, biased and extremely time-consuming for large data sets. Here, we present the following workflow based on mathematical imaging methods. In the first step, mitosis detection is performed by means of the circular Hough transform. The obtained circular contour subsequently serves as an initialisation for the tracking algorithm based on variational methods. It is sub-divided into two parts: in order to determine the beginning of the whole mitosis cycle, a backwards tracking procedure is performed. After that, the cell is tracked forwards in time until the end of mitosis. As a result, the average of mitosis duration and ratios of different cell fates (cell death, no division, division into two or more daughter cells) can be measured and statistics on cell morphologies can be obtained. All of the tools are featured in the user-friendly MATLAB®Graphical User Interface MitosisAnalyser.

A Hybrid Approach for Segmentation and Tracking of Myxococcus Xanthus Swarms

Cell segmentation and motion tracking in time-lapse images are fundamental problems in computer vision, and are also crucial for various biomedical studies. Myxococcus xanthus is a type of rod-like cells with highly coordinated motion. The segmentation and tracking of M. xanthus are challenging, because cells may touch tightly and form dense swarms that are difficult to identify individually in an accurate manner. The known cell tracking approaches mainly fall into two frameworks, detection association and model evolution, each having its own advantages and disadvantages. In this paper, we propose a new hybrid framework combining these two frameworks into one and leveraging their complementary advantages. Also, we propose an active contour model based on the Ribbon Snake, which is seamlessly integrated with our hybrid framework. Evaluated by 10 different datasets, our approach achieves considerable improvement over the state-of-the-art cell tracking algorithms on identifying complete cell trajectories, and higher segmentation accuracy than performing segmentation in individual 2D images.

Deformable Graph Model for Tracking Epithelial Cell Sheets in Fluorescence Microscopy

We propose a novel method for tracking cells that are connected through a visible network of membrane junctions. Tissues of this form are common in epithelial cell sheets and resemble planar graphs where each face corresponds to a cell. We leverage this structure and develop a method to track the entire tissue as a deformable graph. This coupled model in which vertices inform the optimal placement of edges and vice versa captures global relationships between tissue components and leads to accurate and robust cell tracking. We compare the performance of our method with that of four reference tracking algorithms on four data sets that present unique tracking challenges. Our method exhibits consistently superior performance in tracking all cells accurately over all image frames, and is robust over a wide range of image intensity and cell shape profiles. This may be an important tool for characterizing tissues of this type especially in the field of developmental biology where automated cell analysis can help elucidate the mechanisms behind controlled cell-shape changes.

Measuring Blood Glucose Concentrations in Photometric Glucometers Requiring Very Small Sample Volumes

Glucometers present an important self-monitoring tool for diabetes patients and, therefore, must exhibit high accuracy as well as good usability features. Based on an invasive photometric measurement principle that drastically reduces the volume of the blood sample needed from the patient, we present a framework that is capable of dealing with small blood samples, while maintaining the required accuracy. The framework consists of two major parts: 1) image segmentation; and 2) convergence detection. Step 1 is based on iterative mode-seeking methods to estimate the intensity value of the region of interest. We present several variations of these methods and give theoretical proofs of their convergence. Our approach is able to deal with changes in the number and position of clusters without any prior knowledge. Furthermore, we propose a method based on sparse approximation to decrease the computational load, while maintaining accuracy. Step 2 is achieved by employing temporal tracking and prediction, herewith decreasing the measurement time, and, thus, improving usability. Our framework is tested on several real datasets with different characteristics. We show that we are able to estimate the underlying glucose concentration from much smaller blood samples than is currently state of the art with sufficient accuracy according to the most recent ISO standards and reduce measurement time significantly compared to state-of-the-art methods.

Automated tracking of temporal displacements of a red blood cell obtained by time-lapse digital holographic microscopy

Red blood cell (RBC) phase images that are numerically reconstructed by digital holographic microscopy (DHM) can describe the cell structure and dynamics information beneficial for a quantitative analysis of RBCs. However, RBCs investigated with time-lapse DHM undergo temporal displacements when their membranes are loosely attached to the substrate during sedimentation on a glass surface or due to the microscope drift. Therefore, we need to develop a tracking algorithm to localize the same RBC among RBC image sequences and dynamically monitor its biophysical cell parameters; this information is helpful for studies on RBC-related diseases and drug tests. Here, we propose a method, which is a combination of the mean-shift algorithm and Kalman filter, to track a single RBC and demonstrate that the optical path length of the single RBC can be continually extracted from the tracked RBC. The Kalman filter is utilized to predict the target RBC position in the next frame. Then, the mean-shift algorithm starts execution from the predicted location, and a robust kernel, which is adaptive to changes in the RBC scale, shape, and direction, is designed to improve the accuracy of the tracking. Finally, the tracked RBC is segmented and parameters such as the RBC location are extracted to update the Kalman filter and the kernel function for mean-shift tracking; the characteristics of the target RBC are dynamically observed. Experimental results show the feasibility of the proposed algorithm.

Associating approximate paths and temporal sequences of noisy detections: Application to the recovery of spatio-temporal cancer cell trajectories

In this paper we address the problem of recovering spatio-temporal trajectories of cancer cells in phase contrast video-microscopy where the user provides the paths on which the cells are moving. The paths are purely spatial, without temporal information. To recover the temporal information associated to a given path we propose an approach based on automatic cell detection and on a graph-based shortest path search. The nodes in the graph consist of the projections of the cell detections onto the geometrical cell path. The edges relate nodes which correspond to different frames of the sequence and potentially to the same cell and trajectory. In this directed graph we search for the shortest path and use it to define a temporal parametrization of the corresponding geometrical cell path. An evaluation based on 286 paths of 7 phase contrast microscopy videos shows that our algorithm allows to recover 92% of trajectory points with respect to the associated ground truth. We compare our method with a state-of-the-art algorithm for semi-automated cell tracking in phase contrast microscopy which requires interactively placed starting points for the cells to track. The comparison shows that supporting geometrical paths in combination with our algorithm allow us to obtain more reliable cell trajectories.

A computational approach to detect and segment cytoplasm in muscle fiber images

We developed a computational approach to detect and segment cytoplasm in microscopic images of skeletal muscle fibers. The computational approach provides computer-aided analysis of cytoplasm objects in muscle fiber images to facilitate biomedical research. Cytoplasm in muscle fibers plays an important role in maintaining the functioning and health of muscular tissues. Therefore, cytoplasm is often used as a marker in broad applications of musculoskeletal research, including our search on treatment of muscular disorders such as Duchenne muscular dystrophy, a disease that has no available treatment. However, it is often challenging to analyze cytoplasm and quantify it given the large number of images typically generated in experiments and the large number of muscle fibers contained in each image. Manual analysis is not only time consuming but also prone to human errors. In this work we developed a computational approach to detect and segment the longitudinal sections of cytoplasm based on a modified graph cuts technique and iterative splitting method to extract cytoplasm objects from the background. First, cytoplasm objects are extracted from the background using the modified graph cuts technique which is designed to optimize an energy function. Second, an iterative splitting method is designed to separate the touching or adjacent cytoplasm objects from the results of graph cuts. We tested the computational approach on real data from in vitro experiments and found that it can achieve satisfactory performance in terms of precision and recall rates.

Segmentation and tracking of stem cells in time lapse microscopy to quantify dynamic behavioral changes during spheroid formation

Dynamic behavior of stem cells during in vitro development is diverse. Previous cell tracking studies have focused more on cell proliferation than on cell aggregation. However, the enhancement of cell proliferation in association with cell aggregation has been reported. In a previous study, we also demonstrated that the aggregation of adult human mesenchymal stem cells to form three-dimensional (3D) cellular spheroids helped maintain the expression of stemness marker genes in the cells. However, the dynamic behavioral changes triggered by spheroid formation remain to be investigated. A scheme of image processing techniques is proposed to meet this need. A hybrid-thresholding technique was first developed for efficient segmentation of cell clusters, after which a cell tracking method based on pair-matching with topological constraints was designed. Two morphological indices were derived to track the timing of 3D spheroid formation during the cellular aggregation process. Five cell motility indices measured from single cells and 3D spheroids were then compared. After confirmation of more than 90% correspondence between the results obtained by manual tracking and the proposed methods, an analysis of cellular behavior reveals a significant increase in motility in association with spheroid formation, consistent with a previous report that used a gene expression approach. This study proposed a systematic image analysis method to quantify the dynamic behavior of stem cells for stemness evaluation during cell culturing in vitro. Results demonstrated the validity of the developed platform in investigation of the dynamic behavior of cell aggregation in stem cell cultures in vitro.

Classification of 3D Multicellular Organization in Phase Microscopy for High Throughput Screening of Therapeutic Targets

The current trend in high throughput screening is the utilization of more complex model systems that mimic both structural and functional properties of cellular processes in vivo. In this context, 3D cell culture models have emerged as effective systems to study tumor initiation and cancer behavior, where colony organization represents distinct phenotypic signatures that enable differentiation of cancer cells in culture using phase imaging and in the absence of clinical markers. If the colony organization can be classified into different phenotypes, it will enable rapid drug screening using phase microscopy. In this paper, we propose a novel method based on locality-constrained dictionary learning for the discrimination of aberrant colony organization in phase images, which encodes original SIFT (Scale-Invariant Feature Transform) features into high dimensional sparse codes with locality-preserving landmark points on the nonlinear manifold, and summarizes the sparse features at various locations and scales through spatial pyramid matching for robust representation. Experimental results demonstrate the significant improvement of performance, compared to the state-of-art in the field.

A method for automatic segmentation of nuclei in phase-contrast images based on intensity, convexity and texture

This paper presents a method for automatic segmentation of nuclei in phase-contrast images using the intensity, convexity and texture of the nuclei. The proposed method consists of three main stages: preprocessing, h-maxima transformation-based marker controlled watershed segmentation ( h-TMC), and texture analysis. In the preprocessing stage, a top-hat filter is used to increase the contrast and suppress the non-uniform illumination, shading, and other imaging artifacts in the input image. The nuclei segmentation stage consists of a distance transformation, h-maxima transformation and watershed segmentation. These transformations utilize the intensity information and the convexity property of the nucleus for the purpose of detecting a single marker in every nucleus; these markers are then used in the h-TMC watershed algorithm to obtain segments of the nuclei. However, dust particles, imaging artifacts, or prolonged cell cytoplasm may falsely be segmented as nuclei at this stage, and thus may lead to an inaccurate analysis of the cell image. In order to identify and remove these non-nuclei segments, in the third stage a texture analysis is performed, that uses six of the Haralick measures along with the AdaBoost algorithm. The novelty of the proposed method is that it introduces a systematic framework that utilizes intensity, convexity, and texture information to achieve a high accuracy for automatic segmentation of nuclei in the phase-contrast images. Extensive experiments are performed demonstrating the superior performance ( precision = 0.948; recall = 0.924; F1-measure = 0.936; validation based on  ∼ 4850 manually-labeled nuclei) of the proposed method.

A novel framework for cellular tracking and mitosis detection in dense phase contrast microscopy images

The aim of this paper is to detail the development of a novel tracking framework that is able to extract the cell motility indicators and to determine the cellular division (mitosis) events in large time-lapse phase-contrast image sequences. To address the challenges induced by nonstructured (random) motion, cellular agglomeration, and cellular mitosis, the process of automatic (unsupervised) cell tracking is carried out in a sequential manner, where the interframe cell association is achieved by assessing the variation in the local cellular structures in consecutive frames of the image sequence. In our study, a strong emphasis has been placed on the robust use of the topological information in the cellular tracking process and in the development of targeted pattern recognition techniques that were designed to redress the problems caused by segmentation errors, and to precisely identify mitosis using a backward (reversed) tracking strategy. The proposed algorithm has been evaluated on dense phase-contrast cellular data and the experimental results indicate that the proposed algorithm is able to accurately track epithelial and endothelial cells in time-lapse image sequences that are characterized by low contrast and high level of noise. Our algorithm achieved 86.10% overall tracking accuracy and 90.12% mitosis detection accuracy.