Global linking of cell tracks using the Viterbi algorithm

Scale selection and machine learning based cell segmentation and tracking in time lapse microscopy

Monitoring and tracking of cell motion is a key component for understanding disease mechanisms and evaluating the effects of treatments. Time-lapse optical microscopy has been commonly employed for studying cell cycle phases. However, usual manual cell tracking is very time consuming and has poor reproducibility. Automated cell tracking techniques are challenged by variability of cell region intensity distributions and resolution limitations. In this work, we introduce a comprehensive cell segmentation and tracking methodology. A key contribution of this work is that it employs multi-scale space-time interest point detection and characterization for automatic scale selection and cell segmentation. Another contribution is the use of a neural network with class prototype balancing for detection of cell regions. This work also offers a structured mathematical framework that uses graphs for track generation and cell event detection. We evaluated cell segmentation, detection, and tracking performance of our method on time-lapse sequences of the Cell Tracking Challenge (CTC). We also compared our technique to top performing techniques from CTC. Performance evaluation results indicate that the proposed methodology is competitive with these techniques, and that it generalizes very well to diverse cell types and sizes, and multiple imaging techniques. The code of our method is publicly available on https://github.com/smakrogi/CSTQ_Pub/ , (release v.3.2).

A data-driven approach to establishing cell motility patterns as predictors of macrophage subtypes and their relation to cell morphology

The motility of macrophages in response to microenvironment stimuli is a hallmark of innate immunity, where macrophages play pro-inflammatory or pro-reparatory roles depending on their activation status during wound healing. Cell size and shape have been informative in defining macrophage subtypes. Studies show pro and anti-inflammatory macrophages exhibit distinct migratory behaviors, in vitro, in 3D and in vivo but this link has not been rigorously studied. We apply both morphology and motility-based image processing approaches to analyze live cell images consisting of macrophage phenotypes. Macrophage subtypes are differentiated from primary murine bone marrow derived macrophages using a potent lipopolysaccharide (LPS) or cytokine interleukin-4 (IL-4). We show that morphology is tightly linked to motility, which leads to our hypothesis that motility analysis could be used alone or in conjunction with morphological features for improved prediction of macrophage subtypes. We train a support vector machine (SVM) classifier to predict macrophage subtypes based on morphology alone, motility alone, and both morphology and motility combined. We show that motility has comparable predictive capabilities as morphology. However, using both measures can enhance predictive capabilities. While motility and morphological features can be individually ambiguous identifiers, together they provide significantly improved prediction accuracies (75%) from a training dataset of 1000 cells tracked over time using only phase contrast time-lapse microscopy. Thus, the approach combining cell motility and cell morphology information can lead to methods that accurately assess functionally diverse macrophage phenotypes quickly and efficiently. This can support the development of cost efficient and high through-put methods for screening biochemicals targeting macrophage polarization.

Three-dimensional localization and tracking of chromosomal loci throughout the Escherichia coli cell cycle

The intracellular position of genes may impact their expression, but it has not been possible to accurately measure the 3D position of chromosomal loci. In 2D, loci can be tracked using arrays of DNA-binding sites for transcription factors (TFs) fused with fluorescent proteins. However, the same 2D data can result from different 3D trajectories. Here, we have developed a deep learning method for super-resolved astigmatism-based 3D localization of chromosomal loci in live E. coli cells which enables a precision better than 61 nm at a signal-to-background ratio of ~4 on a heterogeneous cell background. Determining the spatial localization of chromosomal loci, we find that some loci are at the periphery of the nucleoid for large parts of the cell cycle. Analyses of individual trajectories reveal that these loci are subdiffusive both longitudinally (x) and radially (r), but that individual loci explore the full radial width on a minute time scale.

Scale Selection and Machine Learning-based Cell Segmentation and Tracking in Time Lapse Microscopy

Monitoring and tracking of cell motion is a key component for understanding disease mechanisms and evaluating the effects of treatments. Time-lapse optical microscopy has been commonly employed for studying cell cycle phases. However, usual manual cell tracking is very time consuming and has poor reproducibility. Automated cell tracking techniques are challenged by variability of cell region intensity distributions and resolution limitations. In this work, we introduce a comprehensive cell segmentation and tracking methodology. A key contribution of this work is that it employs multi-scale space-time interest point detection and characterization for automatic scale selection and cell segmentation. Another contribution is the use of a neural network with class prototype balancing for detection of cell regions. This work also offers a structured mathematical framework that uses graphs for track generation and cell event detection. We evaluated cell segmentation, detection, and tracking performance of our method on time-lapse sequences of the Cell Tracking Challenge (CTC). We also compared our technique to top performing techniques from CTC. Performance evaluation results indicate that the proposed methodology is competitive with these techniques, and that it generalizes very well to diverse cell types and sizes, and multiple imaging techniques.

CMTT-JTracker: a fully test-time adaptive framework serving automated cell lineage construction

Cell tracking is an essential function needed in automated cellular activity monitoring. In practice, processing methods striking a balance between computational efficiency and accuracy as well as demonstrating robust generalizability across diverse cell datasets are highly desired. This paper develops a central-metric fully test-time adaptive framework for cell tracking (CMTT-JTracker). Firstly, a CMTT mechanism is designed for the pre-segmentation of cell images, which enables extracting target information at different resolutions without additional training. Next, a multi-task learning network with the spatial attention scheme is developed to simultaneously realize detection and re-identification tasks based on features extracted by CMTT. Experimental results demonstrate that the CMTT-JTracker exhibits remarkable biological and tracking performance compared with benchmarking tracking methods. It achieves a multiple object tracking accuracy (MOTA) of $0.894$ on Fluo-N2DH-SIM+ and a MOTA of $0.850$ on PhC-C2DL-PSC. Experimental results further confirm that the CMTT applied solely as a segmentation unit outperforms the SOTA segmentation benchmarks on various datasets, particularly excelling in scenarios with dense cells. The Dice coefficients of the CMTT range from a high of $0.928$ to a low of $0.758$ across different datasets.

The Escherichia coli chromosome moves to the replisome

In Escherichia coli, it is debated whether the two replisomes move independently along the two chromosome arms during replication or if they remain spatially confined. Here, we use high-throughput fluorescence microscopy to simultaneously determine the location and short-time-scale (1 s) movement of the replisome and a chromosomal locus throughout the cell cycle. The assay is performed for several loci. We find that (i) the two replisomes are confined to a region of ~250 nm and ~120 nm along the cell's long and short axis, respectively, (ii) the chromosomal loci move to and through this region sequentially based on their distance from the origin of replication, and (iii) when a locus is being replicated, its short time-scale movement slows down. This behavior is the same at different growth rates. In conclusion, our data supports a model with DNA moving towards spatially confined replisomes at replication.

Isolated Random Forest Assisted Spatio-Temporal Ant Colony Evolutionary Algorithm for Cell Tracking in Time-Lapse Sequences

Multi-Object tracking in real world environments is a tough problem, especially for cell morphogenesis with division. Most cell tracking methods are hard to achieve reliable mitosis detection, efficient inter-frame matching, and accurate state estimation simultaneously within a unified tracking framework. In this paper, we propose a novel unified framework that leverages a spatio-temporal ant colony evolutionary algorithm to track cells amidst mitosis under measurement uncertainty. Each Bernoulli ant colony representing a migrating cell is able to capture the occurrence of mitosis through the proposed Isolation Random Forest (IRF)-assisted temporal mitosis detection algorithm with the assumption that mitotic cells exhibit unique spatio-temporal features different from non-mitotic ones. Guided by prediction of a division event, multiple ant colonies evolve between consecutive frames according to an augmented assignment matrix solved by the extended Hungarian method. To handle dense cell populations, an efficient group partition between cells and measurements is exploited, which enables multiple assignment tasks to be executed in parallel with a reduction in matrix dimension. After inter-frame traversing, the ant colony transitions to a foraging stage in which it begins approximating the Bernoulli parameter to estimate cell state by iteratively updating its pheromone field. Experiments on multi-cell tracking in the presence of cell mitosis and morphological changes are conducted, and the results demonstrate that the proposed method outperforms state-of-the-art approaches, striking a balance between accuracy and computational efficiency.

Versatile multiple object tracking in sparse 2D/3D videos via deformable image registration

Tracking body parts in behaving animals, extracting fluorescence signals from cells embedded in deforming tissue, and analyzing cell migration patterns during development all require tracking objects with partially correlated motion. As dataset sizes increase, manual tracking of objects becomes prohibitively inefficient and slow, necessitating automated and semi-automated computational tools. Unfortunately, existing methods for multiple object tracking (MOT) are either developed for specific datasets and hence do not generalize well to other datasets, or require large amounts of training data that are not readily available. This is further exacerbated when tracking fluorescent sources in moving and deforming tissues, where the lack of unique features and sparsely populated images create a challenging environment, especially for modern deep learning techniques. By leveraging technology recently developed for spatial transformer networks, we propose ZephIR, an image registration framework for semi-supervised MOT in 2D and 3D videos. ZephIR can generalize to a wide range of biological systems by incorporating adjustable parameters that encode spatial (sparsity, texture, rigidity) and temporal priors of a given data class. We demonstrate the accuracy and versatility of our approach in a variety of applications, including tracking the body parts of a behaving mouse and neurons in the brain of a freely moving C. elegans. We provide an open-source package along with a web-based graphical user interface that allows users to provide small numbers of annotations to interactively improve tracking results.

Deep learning classification for macrophage subtypes through cell migratory pattern analysis

Macrophages can exhibit pro-inflammatory or pro-reparatory functions, contingent upon their specific activation state. This dynamic behavior empowers macrophages to engage in immune reactions and contribute to tissue homeostasis. Understanding the intricate interplay between macrophage motility and activation status provides valuable insights into the complex mechanisms that govern their diverse functions. In a recent study, we developed a classification method based on morphology, which demonstrated that movement characteristics, including speed and displacement, can serve as distinguishing factors for macrophage subtypes. In this study, we develop a deep learning model to explore the potential of classifying macrophage subtypes based solely on raw trajectory patterns. The classification model relies on the time series of x-y coordinates, as well as the distance traveled and net displacement. We begin by investigating the migratory patterns of macrophages to gain a deeper understanding of their behavior. Although this analysis does not directly inform the deep learning model, it serves to highlight the intricate and distinct dynamics exhibited by different macrophage subtypes, which cannot be easily captured by a finite set of motility metrics. Our study uses cell trajectories to classify three macrophage subtypes: M0, M1, and M2. This advancement holds promising implications for the future, as it suggests the possibility of identifying macrophage subtypes without relying on shape analysis. Consequently, it could potentially eliminate the necessity for high-quality imaging techniques and provide more robust methods for analyzing inherently blurry images.

CellTool: An Open-Source Software Combining Bio-Image Analysis and Mathematical Modeling for the Study of DNA Repair Dynamics

Elucidating the dynamics of DNA repair proteins is essential to understanding the mechanisms that preserve genomic stability and prevent carcinogenesis. However, the measurement and modeling of protein dynamics at DNA lesions via currently available image analysis tools is cumbersome. Therefore, we developed CellTool-a stand-alone open-source software with a graphical user interface for the analysis of time-lapse microscopy images. It combines data management, image processing, mathematical modeling, and graphical presentation of data in a single package. Multiple image filters, segmentation, and particle tracking algorithms, combined with direct visualization of the obtained results, make CellTool an ideal application for the comprehensive analysis of DNA repair protein dynamics. This software enables the fitting of obtained kinetic data to predefined or custom mathematical models. Importantly, CellTool provides a platform for easy implementation of custom image analysis packages written in a variety of programing languages. Using CellTool, we demonstrate that the ALKB homolog 2 (ALKBH2) demethylase is excluded from DNA damage sites despite recruitment of its putative interaction partner proliferating cell nuclear antigen (PCNA). Further, CellTool facilitates the straightforward fluorescence recovery after photobleaching (FRAP) analysis of BRCA1 associated RING domain 1 (BARD1) exchange at complex DNA lesions. In summary, the software presented herein enables the time-efficient analysis of a wide range of time-lapse microscopy experiments through a user-friendly interface.

A Weakly Supervised Learning Method for Cell Detection and Tracking Using Incomplete Initial Annotations

The automatic detection of cells in microscopy image sequences is a significant task in biomedical research. However, routine microscopy images with cells, which are taken during the process whereby constant division and differentiation occur, are notoriously difficult to detect due to changes in their appearance and number. Recently, convolutional neural network (CNN)-based methods have made significant progress in cell detection and tracking. However, these approaches require many manually annotated data for fully supervised training, which is time-consuming and often requires professional researchers. To alleviate such tiresome and labor-intensive costs, we propose a novel weakly supervised learning cell detection and tracking framework that trains the deep neural network using incomplete initial labels. Our approach uses incomplete cell markers obtained from fluorescent images for initial training on the Induced Pluripotent Stem (iPS) cell dataset, which is rarely studied for cell detection and tracking. During training, the incomplete initial labels were updated iteratively by combining detection and tracking results to obtain a model with better robustness. Our method was evaluated using two fields of the iPS cell dataset, along with the cell detection accuracy (DET) evaluation metric from the Cell Tracking Challenge (CTC) initiative, and it achieved 0.862 and 0.924 DET, respectively. The transferability of the developed model was tested using the public dataset FluoN2DH-GOWT1, which was taken from CTC; this contains two datasets with reference annotations. We randomly removed parts of the annotations in each labeled data to simulate the initial annotations on the public dataset. After training the model on the two datasets, with labels that comprise 10% cell markers, the DET improved from 0.130 to 0.903 and 0.116 to 0.877. When trained with labels that comprise 60% cell markers, the performance was better than the model trained using the supervised learning method. This outcome indicates that the model's performance improved as the quality of the labels used for training increased.

Machine learning enhanced cell tracking

Quantifying cell biology in space and time requires computational methods to detect cells, measure their properties, and assemble these into meaningful trajectories. In this aspect, machine learning (ML) is having a transformational effect on bioimage analysis, now enabling robust cell detection in multidimensional image data. However, the task of cell tracking, or constructing accurate multi-generational lineages from imaging data, remains an open challenge. Most cell tracking algorithms are largely based on our prior knowledge of cell behaviors, and as such, are difficult to generalize to new and unseen cell types or datasets. Here, we propose that ML provides the framework to learn aspects of cell behavior using cell tracking as the task to be learned. We suggest that advances in representation learning, cell tracking datasets, metrics, and methods for constructing and evaluating tracking solutions can all form part of an end-to-end ML-enhanced pipeline. These developments will lead the way to new computational methods that can be used to understand complex, time-evolving biological systems.

Ensemble Deep Learning Object Detection Fusion for Cell Tracking, Mitosis, and Lineage

Cell tracking and motility analysis are essential for understanding multicellular processes, automated quantification in biomedical experiments, and medical diagnosis and treatment. However, manual tracking is labor-intensive, tedious, and prone to selection bias and errors. Building upon our previous work, we propose a new deep learning-based method, EDNet, for cell detection, tracking, and motility analysis that is more robust to shape across different cell lines, and models cell lineage and proliferation. EDNet uses an ensemble approach for 2D cell detection that is deep-architecture-agnostic and achieves state-of-the-art performance surpassing single-model YOLO and FasterRCNN convolutional neural networks. EDNet detections are used in our M2Track multiobject tracking algorithm for tracking cells, detecting cell mitosis (cell division) events, and cell lineage graphs. Our methods produce state-of-the-art performance on the Cell Tracking and Mitosis (CTMCv1) dataset with a Multiple Object Tracking Accuracy (MOTA) score of 50.6% and tracking lineage graph edit (TRA) score of 52.5%. Additionally, we compare our detection and tracking methods to human performance on external data in studying the motility of muscle stem cells with different physiological and molecular stimuli. We believe that our method has the potential to improve the accuracy and efficiency of cell tracking and motility analysis. This could lead to significant advances in biomedical research and medical diagnosis. Our code is made publicly available on GitHub.

Regulatory elements coordinating initiation of chromosome replication to the Escherichia coli cell cycle

Escherichia coli coordinates replication and division cycles by initiating replication at a narrow range of cell sizes. By tracking replisomes in individual cells through thousands of division cycles in wild-type and mutant strains, we were able to compare the relative importance of previously described control systems. We found that accurate triggering of initiation does not require synthesis of new DnaA. The initiation size increased only marginally as DnaA was diluted by growth after dnaA expression had been turned off. This suggests that the conversion of DnaA between its active ATP- and inactive ADP-bound states is more important for initiation size control than the total free concentration of DnaA. In addition, we found that the known ATP/ADP converters DARS and datA compensate for each other, although the removal of them makes the initiation size more sensitive to the concentration of DnaA. Only disruption of the regulatory inactivation of DnaA mechanism had a radical impact on replication initiation. This result was corroborated by the finding that termination of one round of replication correlates with the next initiation at intermediate growth rates, as would be the case if RIDA-mediated conversion from DnaA-ATP to DnaA-ADP abruptly stops at termination and DnaA-ATP starts accumulating.

The Cell Tracking Challenge: 10 years of objective benchmarking

The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.

Optogenetic manipulation identifies the roles of ERK and AKT dynamics in controlling mouse embryonic stem cell exit from pluripotency

ERK and AKT signaling control pluripotent cell self-renewal versus differentiation. ERK pathway activity over time (i.e., dynamics) is heterogeneous between individual pluripotent cells, even in response to the same stimuli. To analyze potential functions of ERK and AKT dynamics in controlling mouse embryonic stem cell (ESC) fates, we developed ESC lines and experimental pipelines for the simultaneous long-term manipulation and quantification of ERK or AKT dynamics and cell fates. We show that ERK activity duration or amplitude or the type of ERK dynamics (e.g., transient, sustained, or oscillatory) alone does not influence exit from pluripotency, but the sum of activity over time does. Interestingly, cells retain memory of previous ERK pulses, with duration of memory retention dependent on duration of previous pulse length. FGF receptor/AKT dynamics counteract ERK-induced pluripotency exit. These findings improve our understanding of how cells integrate dynamics from multiple signaling pathways and translate them into cell fate cues.

Embryonic stem cell ERK, AKT, plus STAT3 response dynamics combinatorics are heterogeneous but NANOG state independent

Signaling is central in cell fate regulation, and relevant information is encoded in its activity over time (i.e., dynamics). However, simultaneous dynamics quantification of several pathways in single mammalian stem cells has not yet been accomplished. Here we generate mouse embryonic stem cell (ESC) lines simultaneously expressing fluorescent reporters for ERK, AKT, and STAT3 signaling activity, which all control pluripotency. We quantify their single-cell dynamics combinations in response to different self-renewal stimuli and find striking heterogeneity for all pathways, some dependent on cell cycle but not pluripotency states, even in ESC populations currently assumed to be highly homogeneous. Pathways are mostly independently regulated, but some context-dependent correlations exist. These quantifications reveal surprising single-cell heterogeneity in the important cell fate control layer of signaling dynamics combinations and raise fundamental questions about the role of signaling in (stem) cell fate control.

The time integral of BMP signaling determines fate in a stem cell model for early human development

How paracrine signals are interpreted to yield multiple cell fate decisions in a dynamic context during human development in vivo and in vitro remains poorly understood. Here we report an automated tracking method to follow signaling histories linked to cell fate in large numbers of human pluripotent stem cells (hPSCs). Using an unbiased statistical approach, we discovered that measured BMP signaling history correlates strongly with fate in individual cells. We found that BMP response in hPSCs varies more strongly in the duration of signaling than the level. However, we discovered that both the level and duration of signaling activity control cell fate choices only by changing the time integral of signaling and that duration and level are therefore interchangeable in this context. In a stem cell model for patterning of the human embryo, we showed that signaling histories predict the fate pattern and that the integral model correctly predicts changes in cell fate domains when signaling is perturbed. Using an RNA-seq screen we then found that mechanistically, BMP signaling is integrated by SOX2.

CellTrackVis: interactive browser-based visualization for analyzing cell trajectories and lineages

BACKGROUND

Automatic cell tracking methods enable practitioners to analyze cell behaviors efficiently. Notwithstanding the continuous development of relevant software, user-friendly visualization tools have room for further improvements. Typical visualization mostly comes with main cell tracking tools as a simple plug-in, or relies on specific software/platforms. Although some tools are standalone, limited visual interactivity is provided, or otherwise cell tracking outputs are partially visualized.

RESULTS

This paper proposes a self-reliant visualization system, CellTrackVis, to support quick and easy analysis of cell behaviors. Interconnected views help users discover meaningful patterns of cell motions and divisions in common web browsers. Specifically, cell trajectory, lineage, and quantified information are respectively visualized in a coordinated interface. In particular, immediate interactions among modules enable the study of cell tracking outputs to be more effective, and also each component is highly customizable for various biological tasks.

CONCLUSIONS

CellTrackVis is a standalone browser-based visualization tool. Source codes and data sets are freely available at http://github.com/scbeom/celltrackvis with the tutorial at http://scbeom.github.io/ctv_tutorial .

Segmentation, tracking, and sub-cellular feature extraction in 3D time-lapse images

This paper presents a method for time-lapse 3D cell analysis. Specifically, we consider the problem of accurately localizing and quantitatively analyzing sub-cellular features, and for tracking individual cells from time-lapse 3D confocal cell image stacks. The heterogeneity of cells and the volume of multi-dimensional images presents a major challenge for fully automated analysis of morphogenesis and development of cells. This paper is motivated by the pavement cell growth process, and building a quantitative morphogenesis model. We propose a deep feature based segmentation method to accurately detect and label each cell region. An adjacency graph based method is used to extract sub-cellular features of the segmented cells. Finally, the robust graph based tracking algorithm using multiple cell features is proposed for associating cells at different time instances. We also demonstrate the generality of our tracking method on C. elegans fluorescent nuclei imagery. Extensive experiment results are provided and demonstrate the robustness of the proposed method. The code is available on GitHub and the method is available as a service through the BisQue portal.

Machine learning-based classification of dual fluorescence signals reveals muscle stem cell fate transitions in response to regenerative niche factors

The proper regulation of muscle stem cell (MuSC) fate by cues from the niche is essential for regeneration of skeletal muscle. How pro-regenerative niche factors control the dynamics of MuSC fate decisions remains unknown due to limitations of population-level endpoint assays. To address this knowledge gap, we developed a dual fluorescence imaging time lapse (Dual-FLIT) microscopy approach that leverages machine learning classification strategies to track single cell fate decisions with high temporal resolution. Using two fluorescent reporters that read out maintenance of stemness and myogenic commitment, we constructed detailed lineage trees for individual MuSCs and their progeny, classifying each division event as symmetric self-renewing, asymmetric, or symmetric committed. Our analysis reveals that treatment with the lipid metabolite, prostaglandin E2 (PGE2), accelerates the rate of MuSC proliferation over time, while biasing division events toward symmetric self-renewal. In contrast, the IL6 family member, Oncostatin M (OSM), decreases the proliferation rate after the first generation, while blocking myogenic commitment. These insights into the dynamics of MuSC regulation by niche cues were uniquely enabled by our Dual-FLIT approach. We anticipate that similar binary live cell readouts derived from Dual-FLIT will markedly expand our understanding of how niche factors control tissue regeneration in real time.

BCM3D 2.0: accurate segmentation of single bacterial cells in dense biofilms using computationally generated intermediate image representations

Accurate detection and segmentation of single cells in three-dimensional (3D) fluorescence time-lapse images is essential for observing individual cell behaviors in large bacterial communities called biofilms. Recent progress in machine-learning-based image analysis is providing this capability with ever-increasing accuracy. Leveraging the capabilities of deep convolutional neural networks (CNNs), we recently developed bacterial cell morphometry in 3D (BCM3D), an integrated image analysis pipeline that combines deep learning with conventional image analysis to detect and segment single biofilm-dwelling cells in 3D fluorescence images. While the first release of BCM3D (BCM3D 1.0) achieved state-of-the-art 3D bacterial cell segmentation accuracies, low signal-to-background ratios (SBRs) and images of very dense biofilms remained challenging. Here, we present BCM3D 2.0 to address this challenge. BCM3D 2.0 is entirely complementary to the approach utilized in BCM3D 1.0. Instead of training CNNs to perform voxel classification, we trained CNNs to translate 3D fluorescence images into intermediate 3D image representations that are, when combined appropriately, more amenable to conventional mathematical image processing than a single experimental image. Using this approach, improved segmentation results are obtained even for very low SBRs and/or high cell density biofilm images. The improved cell segmentation accuracies in turn enable improved accuracies of tracking individual cells through 3D space and time. This capability opens the door to investigating time-dependent phenomena in bacterial biofilms at the cellular level.

Adaptive tracking algorithm for trajectory analysis of cells and layer-by-layer assessment of motility dynamics

Tracking biological objects such as cells or subcellular components imaged with time-lapse microscopy enables us to understand the molecular principles about the dynamics of cell behaviors. However, automatic object detection, segmentation and extracting trajectories remain as a rate-limiting step due to intrinsic challenges of video processing. This paper presents an adaptive tracking algorithm (Adtari) that automatically finds the optimum search radius and cell linkages to determine trajectories in consecutive frames. A critical assumption in most tracking studies is that displacement remains unchanged throughout the movie and cells in a few frames are usually analyzed to determine its magnitude. Tracking errors and inaccurate association of cells may occur if the user does not correctly evaluate the value or prior knowledge is not present on cell movement. The key novelty of our method is that minimum intercellular distance and maximum displacement of cells between frames are dynamically computed and used to determine the threshold distance. Since the space between cells is highly variable in a given frame, our software recursively alters the magnitude to determine all plausible matches in the trajectory analysis. Our method therefore eliminates a major preprocessing step where a constant distance was used to determine the neighbor cells in tracking methods. Cells having multiple overlaps and splitting events were further evaluated by using the shape attributes including perimeter, area, ellipticity and distance. The features were applied to determine the closest matches by minimizing the difference in their magnitudes. Finally, reporting section of our software were used to generate instant maps by overlaying cell features and trajectories. Adtari was validated by using videos with variable signal-to-noise, contrast ratio and cell density. We compared the adaptive tracking with constant distance and other methods to evaluate performance and its efficiency. Our algorithm yields reduced mismatch ratio, increased ratio of whole cell track, higher frame tracking efficiency and allows layer-by-layer assessment of motility to characterize single-cells. Adaptive tracking provides a reliable, accurate, time efficient and user-friendly open source software that is well suited for analysis of 2D fluorescence microscopy video datasets.

Rapid antibiotic susceptibility testing and species identification for mixed samples

Antimicrobial resistance is an increasing problem on a global scale. Rapid antibiotic susceptibility testing (AST) is urgently needed in the clinic to enable personalized prescriptions in high-resistance environments and to limit the use of broad-spectrum drugs. Current rapid phenotypic AST methods do not include species identification (ID), leaving time-consuming plating or culturing as the only available option when ID is needed to make the sensitivity call. Here we describe a method to perform phenotypic AST at the single-cell level in a microfluidic chip that allows subsequent genotyping by in situ FISH. By stratifying the phenotypic AST response on the species of individual cells, it is possible to determine the susceptibility profile for each species in a mixed sample in 2 h. In this proof-of-principle study, we demonstrate the operation with four antibiotics and mixed samples with combinations of seven species.

A multi-omic analysis of MCF10A cells provides a resource for integrative assessment of ligand-mediated molecular and phenotypic responses

The phenotype of a cell and its underlying molecular state is strongly influenced by extracellular signals, including growth factors, hormones, and extracellular matrix proteins. While these signals are normally tightly controlled, their dysregulation leads to phenotypic and molecular states associated with diverse diseases. To develop a detailed understanding of the linkage between molecular and phenotypic changes, we generated a comprehensive dataset that catalogs the transcriptional, proteomic, epigenomic and phenotypic responses of MCF10A mammary epithelial cells after exposure to the ligands EGF, HGF, OSM, IFNG, TGFB and BMP2. Systematic assessment of the molecular and cellular phenotypes induced by these ligands comprise the LINCS Microenvironment (ME) perturbation dataset, which has been curated and made publicly available for community-wide analysis and development of novel computational methods ( synapse.org/LINCS_MCF10A ). In illustrative analyses, we demonstrate how this dataset can be used to discover functionally related molecular features linked to specific cellular phenotypes. Beyond these analyses, this dataset will serve as a resource for the broader scientific community to mine for biological insights, to compare signals carried across distinct molecular modalities, and to develop new computational methods for integrative data analysis.

H-EMD: A Hierarchical Earth Mover's Distance Method for Instance Segmentation

Deep learning (DL) based semantic segmentation methods have achieved excellent performance in biomedical image segmentation, producing high quality probability maps to allow extraction of rich instance information to facilitate good instance segmentation. While numerous efforts were put into developing new DL semantic segmentation models, less attention was paid to a key issue of how to effectively explore their probability maps to attain the best possible instance segmentation. We observe that probability maps by DL semantic segmentation models can be used to generate many possible instance candidates, and accurate instance segmentation can be achieved by selecting from them a set of "optimized" candidates as output instances. Further, the generated instance candidates form a well-behaved hierarchical structure (a forest), which allows selecting instances in an optimized manner. Hence, we propose a novel framework, called hierarchical earth mover's distance (H-EMD), for instance segmentation in biomedical 2D+time videos and 3D images, which judiciously incorporates consistent instance selection with semantic-segmentation-generated probability maps. H-EMD contains two main stages: (1) instance candidate generation: capturing instance-structured information in probability maps by generating many instance candidates in a forest structure; (2) instance candidate selection: selecting instances from the candidate set for final instance segmentation. We formulate a key instance selection problem on the instance candidate forest as an optimization problem based on the earth mover's distance (EMD), and solve it by integer linear programming. Extensive experiments on eight biomedical video or 3D datasets demonstrate that H-EMD consistently boosts DL semantic segmentation models and is highly competitive with state-of-the-art methods.

Multiparameter analysis of timelapse imaging reveals kinetics of megakaryocytic erythroid progenitor clonal expansion and differentiation

Single-cell assays have enriched our understanding of hematopoiesis and, more generally, stem and progenitor cell biology. However, these single-end-point approaches provide only a static snapshot of the state of a cell. To observe and measure dynamic changes that may instruct cell fate, we developed an approach for examining hematopoietic progenitor fate specification using long-term (> 7-day) single-cell time-lapse imaging for up to 13 generations with in situ fluorescence staining of primary human hematopoietic progenitors followed by algorithm-assisted lineage tracing. We analyzed progenitor cell dynamics, including the division rate, velocity, viability, and probability of lineage commitment at the single-cell level over time. We applied a Markov probabilistic model to predict progenitor division outcome over each generation in culture. We demonstrated the utility of this methodological pipeline by evaluating the effects of the cytokines thrombopoietin and erythropoietin on the dynamics of self-renewal and lineage specification in primary human bipotent megakaryocytic-erythroid progenitors (MEPs). Our data support the hypothesis that thrombopoietin and erythropoietin support the viability and self-renewal of MEPs, but do not affect fate specification. Thus, single-cell tracking of time-lapse imaged colony-forming unit assays provides a robust method for assessing the dynamics of progenitor self-renewal and lineage commitment.

CancerCellTracker: a brightfield time-lapse microscopy framework for cancer drug sensitivity estimation

MOTIVATION

Time-lapse microscopy is a powerful technique that relies on images of live cells cultured ex vivo that are captured at regular intervals of time to describe and quantify their behavior under certain experimental conditions. This imaging method has great potential in advancing the field of precision oncology by quantifying the response of cancer cells to various therapies and identifying the most efficacious treatment for a given patient. Digital image processing algorithms developed so far require high-resolution images involving very few cells originating from homogeneous cell line populations. We propose a novel framework that tracks cancer cells to capture their behavior and quantify cell viability to inform clinical decisions in a high-throughput manner.

RESULTS

The brightfield microscopy images a large number of patient-derived cells in an ex vivo reconstruction of the tumor microenvironment treated with 31 drugs for up to 6 days. We developed a robust and user-friendly pipeline CancerCellTracker that detects cells in co-culture, tracks these cells across time and identifies cell death events using changes in cell attributes. We validated our computational pipeline by comparing the timing of cell death estimates by CancerCellTracker from brightfield images and a fluorescent channel featuring ethidium homodimer. We benchmarked our results using a state-of-the-art algorithm implemented in ImageJ and previously published in the literature. We highlighted CancerCellTracker's efficiency in estimating the percentage of live cells in the presence of bone marrow stromal cells.

AVAILABILITY AND IMPLEMENTATION

https://github.com/compbiolabucf/CancerCellTracker.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Live single cell imaging assays in glass microwells produced by laser-induced deep etching

Miniaturization of cell culture substrates enables controlled analysis of living cells in confined micro-scale environments. This is particularly suitable for imaging individual cells over time, as they can be monitored without escaping the imaging field-of-view (FoV). Glass materials are ideal for most microscopy applications. However, with current methods used in life sciences, glass microfabrication is limited in terms of either freedom of design, quality, or throughput. In this work, we introduce laser-induced deep etching (LIDE) as a method for producing glass microwell arrays for live single cell imaging assays. We demonstrate novel microwell arrays with deep, high-aspect ratio wells that have rounded, dimpled or flat bottom profiles in either single-layer or double-layer glass chips. The microwells are evaluated for microscopy-based analysis of long-term cell culture, clonal expansion, laterally organized cell seeding, subcellular mechanics during migration and immune cell cytotoxicity assays of both adherent and suspension cells. It is shown that all types of microwells can support viable cell cultures and imaging with single cell resolution, and we highlight specific benefits of each microwell design for different applications. We believe that high-quality glass microwell arrays enabled by LIDE provide a great option for high-content and high-resolution imaging-based live cell assays with a broad range of potential applications within life sciences.

AI-Driven Cell Tracking to Enable High-Throughput Drug Screening Targeting Airway Epithelial Repair for Children with Asthma

The airway epithelium of children with asthma is characterized by aberrant repair that may be therapeutically modifiable. The development of epithelial-targeting therapeutics that enhance airway repair could provide a novel treatment avenue for childhood asthma. Drug discovery efforts utilizing high-throughput live cell imaging of patient-derived airway epithelial culture-based wound repair assays can be used to identify compounds that modulate airway repair in childhood asthma. Manual cell tracking has been used to determine cell trajectories and wound closure rates, but is time consuming, subject to bias, and infeasible for high-throughput experiments. We therefore developed software, EPIC, that automatically tracks low-resolution low-framerate cells using artificial intelligence, analyzes high-throughput drug screening experiments and produces multiple wound repair metrics and publication-ready figures. Additionally, unlike available cell trackers that perform cell segmentation, EPIC tracks cells using bounding boxes and thus has simpler and faster training data generation requirements for researchers working with other cell types. EPIC outperformed publicly available software in our wound repair datasets by achieving human-level cell tracking accuracy in a fraction of the time. We also showed that EPIC is not limited to airway epithelial repair for children with asthma but can be applied in other cellular contexts by outperforming the same software in the Cell Tracking with Mitosis Detection Challenge (CTMC) dataset. The CTMC is the only established cell tracking benchmark dataset that is designed for cell trackers utilizing bounding boxes. We expect our open-source and easy-to-use software to enable high-throughput drug screening targeting airway epithelial repair for children with asthma.

Tracking cell lineages in 3D by incremental deep learning

Deep learning is emerging as a powerful approach for bioimage analysis. Its use in cell tracking is limited by the scarcity of annotated data for the training of deep-learning models. Moreover, annotation, training, prediction, and proofreading currently lack a unified user interface. We present ELEPHANT, an interactive platform for 3D cell tracking that addresses these challenges by taking an incremental approach to deep learning. ELEPHANT provides an interface that seamlessly integrates cell track annotation, deep learning, prediction, and proofreading. This enables users to implement cycles of incremental learning starting from a few annotated nuclei. Successive prediction-validation cycles enrich the training data, leading to rapid improvements in tracking performance. We test the software's performance against state-of-the-art methods and track lineages spanning the entire course of leg regeneration in a crustacean over 1 week (504 timepoints). ELEPHANT yields accurate, fully-validated cell lineages with a modest investment in time and effort.

Automated Deep Lineage Tree Analysis Using a Bayesian Single Cell Tracking Approach

Single-cell methods are beginning to reveal the intrinsic heterogeneity in cell populations, arising from the interplay of deterministic and stochastic processes. However, it remains challenging to quantify single-cell behaviour from time-lapse microscopy data, owing to the difficulty of extracting reliable cell trajectories and lineage information over long time-scales and across several generations. Therefore, we developed a hybrid deep learning and Bayesian cell tracking approach to reconstruct lineage trees from live-cell microscopy data. We implemented a residual U-Net model coupled with a classification CNN to allow accurate instance segmentation of the cell nuclei. To track the cells over time and through cell divisions, we developed a Bayesian cell tracking methodology that uses input features from the images to enable the retrieval of multi-generational lineage information from a corpus of thousands of hours of live-cell imaging data. Using our approach, we extracted 20,000 + fully annotated single-cell trajectories from over 3,500 h of video footage, organised into multi-generational lineage trees spanning up to eight generations and fourth cousin distances. Benchmarking tests, including lineage tree reconstruction assessments, demonstrate that our approach yields high-fidelity results with our data, with minimal requirement for manual curation. To demonstrate the robustness of our minimally supervised cell tracking methodology, we retrieve cell cycle durations and their extended inter- and intra-generational family relationships in 5,000 + fully annotated cell lineages. We observe vanishing cycle duration correlations across ancestral relatives, yet reveal correlated cyclings between cells sharing the same generation in extended lineages. These findings expand the depth and breadth of investigated cell lineage relationships in approximately two orders of magnitude more data than in previous studies of cell cycle heritability, which were reliant on semi-manual lineage data analysis.

A review for Cell and Particle Tracking on Microscopy Images using Algorithms and Deep Learning Technologies

Time-lapse microscopy images generated by biological experiments have been widely used for observing target activities, such as the motion trajectories and survival states. Based on these observations, biologists can conclude experimental results or present new hypotheses for several biological applications, i.e. virus research or drug design. Many methods or tools have been proposed in the past to observe cell and particle activities, which are defined as single cell tracking and single particle tracking problems, by using algorithms and deep learning technologies. In this article, a review for these works is presented in order to summarize the past methods and research topics at first, then points out the problems raised by these works, and finally proposes future research directions. The contributions of this article will help researchers to understand past development trends and further propose innovative technologies.

DMNet: Dual-Stream Marker Guided Deep Network for Dense Cell Segmentation and Lineage Tracking

Accurate segmentation and tracking of cells in microscopy image sequences is extremely beneficial in clinical diagnostic applications and biomedical research. A continuing challenge is the segmentation of dense touching cells and deforming cells with indistinct boundaries, in low signal-to-noise-ratio images. In this paper, we present a dual-stream marker-guided network (DMNet) for segmentation of touching cells in microscopy videos of many cell types. DMNet uses an explicit cell marker-detection stream, with a separate mask-prediction stream using a distance map penalty function, which enables supervised training to focus attention on touching and nearby cells. For multi-object cell tracking we use M2Track tracking-by-detection approach with multi-step data association. Our M2Track with mask overlap includes short term track-to-cell association followed by track-to-track association to re-link tracklets with missing segmentation masks over a short sequence of frames. Our combined detection, segmentation and tracking algorithm has proven its potential on the IEEE ISBI 2021 6th Cell Tracking Challenge (CTC-6) where we achieved multiple top three rankings for diverse cell types. Our team name is MU-Ba-US, and the implementation of DMNet is available at, http://celltrackingchallenge.net/participants/MU-Ba-US/.

A graph-based cell tracking algorithm with few manually tunable parameters and automated segmentation error correction

Automatic cell segmentation and tracking enables to gain quantitative insights into the processes driving cell migration. To investigate new data with minimal manual effort, cell tracking algorithms should be easy to apply and reduce manual curation time by providing automatic correction of segmentation errors. Current cell tracking algorithms, however, are either easy to apply to new data sets but lack automatic segmentation error correction, or have a vast set of parameters that needs either manual tuning or annotated data for parameter tuning. In this work, we propose a tracking algorithm with only few manually tunable parameters and automatic segmentation error correction. Moreover, no training data is needed. We compare the performance of our approach to three well-performing tracking algorithms from the Cell Tracking Challenge on data sets with simulated, degraded segmentation—including false negatives, over- and under-segmentation errors. Our tracking algorithm can correct false negatives, over- and under-segmentation errors as well as a mixture of the aforementioned segmentation errors. On data sets with under-segmentation errors or a mixture of segmentation errors our approach performs best. Moreover, without requiring additional manual tuning, our approach ranks several times in the top 3 on the 6^th edition of the Cell Tracking Challenge.

An Automated Cell Tracking Approach With Multi-Bernoulli Filtering and Ant Colony Labor Division

In this article, we take as inspiration the labor division into scouts and workers in an ant colony and propose a novel approach for automated cell tracking in the framework of multi-Bernoulli random finite sets. To approximate the Bernoulli parameter sets, we first define an existence probability of an ant colony as well as its discrete density distribution. During foraging, the behavior of scouts is modeled as a chaotic movement to produce a set of potential candidates. Afterwards, a group of workers, i.e., a worker ant colony, is recruited for each candidate, which then embark on gathering heuristic information in a self-organized way. Finally, the pheromone field is formed by the corresponding worker ant colony, from which the Bernoulli parameter is derived and the state of the cell is estimated accordingly to be associated with the existing tracks. Performance comparisons with other previous approaches are conducted on both simulated and real cell image sequences and show the superiority of this algorithm.

A Joint Tracking Approach via Ant Colony Evolution for Quantitative Cell Cycle Analysis

In this paper, we use an ant colony heuristic method to tackle the integration of data association and state estimation in the presence of cell mitosis, morphological change and uncertainty of measurement. Our approach first models the scouting behavior of an unlabeled ant colony as a chaotic process to generate a set of cell candidates in the current frame, then a labeled ant colony foraging process is modeled to construct an interframe matching between previously estimated cell states and current cell candidates through minimizing the optimal sub-pattern assignment metric for track (OSPA-T). The states of cells in the current frame are finally estimated using labeled ant colonies via a multi-Bernoulli parameter set approximated by individual food pheromone fields and heuristic information within the same region of support, the resulting trail pheromone fields over frames constitutes the cell lineage trees of the tracks. A four-stage track recovery strategy is proposed to monitor the history of all established tracks to reconstruct broken tracks in a computationally economic way. The labeling method used in this work is an improvement on previous techniques. The method has been evaluated on publicly available, challenging cell image sequences, and a satisfied performance improvement is achieved in contrast to the state-of-the-art methods.

A survey on applications of deep learning in microscopy image analysis

Advanced microscopy enables us to acquire quantities of time-lapse images to visualize the dynamic characteristics of tissues, cells or molecules. Microscopy images typically vary in signal-to-noise ratios and include a wealth of information which require multiple parameters and time-consuming iterative algorithms for processing. Precise analysis and statistical quantification are often needed for the understanding of the biological mechanisms underlying these dynamic image sequences, which has become a big challenge in the field. As deep learning technologies develop quickly, they have been applied in bioimage processing more and more frequently. Novel deep learning models based on convolution neural networks have been developed and illustrated to achieve inspiring outcomes. This review article introduces the applications of deep learning algorithms in microscopy image analysis, which include image classification, region segmentation, object tracking and super-resolution reconstruction. We also discuss the drawbacks of existing deep learning-based methods, especially on the challenges of training datasets acquisition and evaluation, and propose the potential solutions. Furthermore, the latest development of augmented intelligent microscopy that based on deep learning technology may lead to revolution in biomedical research.

EllipTrack: A Global-Local Cell-Tracking Pipeline for 2D Fluorescence Time-Lapse Microscopy

Time-lapse microscopy provides an unprecedented opportunity to monitor single-cell dynamics. However, tracking cells for long periods remains a technical challenge, especially for multi-day, large-scale movies with rapid cell migration, high cell density, and drug treatments that alter cell morphology/behavior. Here, we present EllipTrack, a global-local cell-tracking pipeline optimized for tracking such movies. EllipTrack first implements a global track-linking algorithm to construct tracks that maximize the probability of cell lineages. Tracking mistakes are then corrected with a local track-correction module in which tracks generated by the global algorithm are systematically examined and amended if a more probable alternative can be found. Through benchmarking, we show that EllipTrack outperforms state-of-the-art cell trackers and generates nearly error-free cell lineages for multiple large-scale movies. In addition, EllipTrack can adapt to time- and cell-density-dependent changes in cell migration speeds and requires minimal training datasets. EllipTrack is available at https://github.com/tianchengzhe/EllipTrack.

DeepSeed Local Graph Matching for Densely Packed Cells Tracking

The tracking of densely packed plant cells across microscopy image sequences is very challenging, because their appearance change greatly over time. A local graph matching algorithm was proposed to track such cells by exploiting the tight spatial topology of neighboring cells, and then an iterative searching strategy was used to grow the correspondence from a seed cell pair. Thus, the performance of the existing tracking approach heavily relies on the robustness of finding seed cell pair. However, the existing local graph matching algorithm cannot guarantee the correctness of the seed cell pair, especially in unregistered image sequences or image sequences with large time intervals. In this paper, we propose a DeepSeed local graph matching model to find seed cell pair robustly, by combining local graph matching and CNN-based similarity learning, which uses cells' spatial-temporal contextual information and cell pairs' similarity information. The CNN-based similarity learning is designed to learn cells' deep feature and measure cell pairs' similarity. Compared with the existing plant cell matching methods, the experimental results show that the DeepSeed local graph matching method can track most cells in unregistered image sequences. Moreover, the DeepSeed tracking algorithm can accurately track cells across image sequences with large time intervals.

A Novel Function of TLR2 and MyD88 in the Regulation of Leukocyte Cell Migration Behavior During Wounding in Zebrafish Larvae

Toll-like receptor (TLR) signaling via myeloid differentiation factor 88 protein (MyD88) has been indicated to be involved in the response to wounding. It remains unknown whether the putative role of MyD88 in wounding responses is due to a control of leukocyte cell migration. The aim of this study was to explore in vivo whether TLR2 and MyD88 are involved in modulating neutrophil and macrophage cell migration behavior upon zebrafish larval tail wounding. Live cell imaging of tail-wounded larvae was performed in tlr2 and myd88 mutants and their corresponding wild type siblings. In order to visualize cell migration following tissue damage, we constructed double transgenic lines with fluorescent markers for macrophages and neutrophils in all mutant and sibling zebrafish lines. Three days post fertilization (dpf), tail-wounded larvae were studied using confocal laser scanning microscopy (CLSM) to quantify the number of recruited cells at the wounding area. We found that in both tlr2^-/- and myd88^-/- groups the recruited neutrophil and macrophage numbers are decreased compared to their wild type sibling controls. Through analyses of neutrophil and macrophage migration patterns, we demonstrated that both tlr2 and myd88 control the migration direction of distant neutrophils upon wounding. Furthermore, in both the tlr2 and the myd88 mutants, macrophages migrated more slowly toward the wound edge. Taken together, our findings show that tlr2 and myd88 are involved in responses to tail wounding by regulating the behavior and speed of leukocyte migration in vivo.

Cell segmentation and tracking using CNN-based distance predictions and a graph-based matching strategy

The accurate segmentation and tracking of cells in microscopy image sequences is an important task in biomedical research, e.g., for studying the development of tissues, organs or entire organisms. However, the segmentation of touching cells in images with a low signal-to-noise-ratio is still a challenging problem. In this paper, we present a method for the segmentation of touching cells in microscopy images. By using a novel representation of cell borders, inspired by distance maps, our method is capable to utilize not only touching cells but also close cells in the training process. Furthermore, this representation is notably robust to annotation errors and shows promising results for the segmentation of microscopy images containing in the training data underrepresented or not included cell types. For the prediction of the proposed neighbor distances, an adapted U-Net convolutional neural network (CNN) with two decoder paths is used. In addition, we adapt a graph-based cell tracking algorithm to evaluate our proposed method on the task of cell tracking. The adapted tracking algorithm includes a movement estimation in the cost function to re-link tracks with missing segmentation masks over a short sequence of frames. Our combined tracking by detection method has proven its potential in the IEEE ISBI 2020 Cell Tracking Challenge (http://celltrackingchallenge.net/) where we achieved as team KIT-Sch-GE multiple top three rankings including two top performances using a single segmentation model for the diverse data sets.

Automated Cell Tracking Using Motion Prediction-Based Matching and Event Handling

Automated cell segmentation and tracking enables the quantification of static and dynamic cell characteristics and is significant for disease diagnosis, treatment, drug development, and other biomedical applications. This paper introduces a method for fully automated cell tracking, lineage construction, and quantification. Cell detection is performed in the joint spatio-temporal domain by a motion diffusion-based Partial Differential Equation (PDE) combined with energy minimizing active contours. In the tracking stage, we adopt a variational joint local-global optical flow technique to determine the motion vector field. We utilize the predicted cell motion jointly with spatial cell features to define a maximum likelihood criterion to find inter-frame cell correspondences assuming Markov dependency. We formulate cell tracking and cell event detection as a graph partitioning problem. We propose a solution obtained by minimization of a global cost function defined over the set of all cell tracks. We construct a cell lineage tree that represents the cell tracks and cell events. Finally, we compute morphological, motility, and diffusivity measures and validate cell tracking against manually generated reference standards. The automated tracking method applied to reference segmentation maps produces an average tracking accuracy score ( TRA) of 99 percent, and the fully automated segmentation and tracking system produces an average TRA of 89 percent.

Deep learning for cellular image analysis

Recent advances in computer vision and machine learning underpin a collection of algorithms with an impressive ability to decipher the content of images. These deep learning algorithms are being applied to biological images and are transforming the analysis and interpretation of imaging data. These advances are positioned to render difficult analyses routine and to enable researchers to carry out new, previously impossible experiments. Here we review the intersection between deep learning and cellular image analysis and provide an overview of both the mathematical mechanics and the programming frameworks of deep learning that are pertinent to life scientists. We survey the field's progress in four key applications: image classification, image segmentation, object tracking, and augmented microscopy. Last, we relay our labs' experience with three key aspects of implementing deep learning in the laboratory: annotating training data, selecting and training a range of neural network architectures, and deploying solutions. We also highlight existing datasets and implementations for each surveyed application.

An Ant Colony Inspired Multi-Bernoulli Filter for Cell Tracking in Time-Lapse Microscopy Sequences

The analysis of the dynamic behavior of cells in time-lapse microscopy sequences requires the development of reliable and automatic tracking methods capable of estimating individual cell states and delineating the lineage trees corresponding to the tracks. In this paper, we propose a novel approach, i.e., an ant colony inspired multi-Bernoulli filter, to handle the tracking of a collection of cells within which mitosis, morphological change and erratic dynamics occur. The proposed technique treats each ant colony as an independent one in an ant society, and the existence probability of an ant colony and its density distribution approximation are derived from the individual pheromone field and the corresponding heuristic information for the approximation to the multi-Bernoulli parameters. To effectively guide ant foraging between consecutive frames, a dual prediction mechanism is proposed for the ant colony and its pheromone field. The algorithm performance is tested on challenging datasets with varying population density, frequent cell mitosis and uneven motion over time, demonstrating that the algorithm outperforms recently reported approaches.

Fully unsupervised symmetry-based mitosis detection in time-lapse cell microscopy

MOTIVATION

Cell microscopy datasets have great diversity due to variability in cell types, imaging techniques and protocols. Existing methods are either tailored to specific datasets or are based on supervised learning, which requires comprehensive manual annotations. Using the latter approach, however, poses a significant difficulty due to the imbalance between the number of mitotic cells with respect to the entire cell population in a time-lapse microscopy sequence.

RESULTS

We present a fully unsupervised framework for both mitosis detection and mother-daughters association in fluorescence microscopy data. The proposed method accommodates the difficulty of the different cell appearances and dynamics. Addressing symmetric cell divisions, a key concept is utilizing daughters' similarity. Association is accomplished by defining cell neighborhood via a stochastic version of the Delaunay triangulation and optimization by dynamic programing. Our framework presents promising detection results for a variety of fluorescence microscopy datasets of different sources, including 2D and 3D sequences from the Cell Tracking Challenge.

AVAILABILITY AND IMPLEMENTATION

Code is available in github (github.com/topazgl/mitodix).

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.