A new framework for particle detection in low-SNR fluorescence live-cell images and its application for improved particle tracking

Toward Source-Free Cross Tissues Histopathological Cell Segmentation via Target-Specific Finetuning

Recognition and quantitative analytics of histopathological cells are the golden standard for diagnosing multiple cancers. Despite recent advances in deep learning techniques that have been widely investigated for the automated segmentation of various types of histopathological cells, the heavy dependency on specific histopathological image types with sufficient supervised annotations, as well as the limited access to clinical data in hospitals, still pose significant challenges in the application of computer-aided diagnosis in pathology. In this paper, we focus on the model generalization of cell segmentation towards cross-tissue histopathological images. Remarkably, a novel target-specific finetuning-based self-supervised domain adaptation framework is proposed to transfer the cell segmentation model to unlabeled target datasets, without access to source datasets and annotations. When performed on the target unlabeled histopathological image set, the proposed method only needs to tune very few parameters of the pre-trained model in a self-supervised manner. Considering the morphological properties of pathological cells, we introduce two constraint terms at both local and global levels into this framework to access more reliable predictions. The proposed cross-domain framework is validated on three different types of histopathological tissues, showing promising performance in self-supervised cell segmentation. Additionally, the whole framework can be further applied to clinical tools in pathology without accessing the original training image data. The code and dataset are released at: https://github.com/NeuronXJTU/SFDA-CellSeg.

BNP-Track: A framework for superresolved tracking

Assessing dynamic processes at single molecule scales is key toward capturing life at the level of its molecular actors. Widefield superresolution methods, such as STORM, PALM, and PAINT, provide nanoscale localization accuracy, even when distances between fluorescently labeled single molecules ("emitters") fall below light's diffraction limit. However, as these superresolution methods rely on rare photophysical events to distinguish emitters from both each other and background, they are largely limited to static samples. In contrast, here we leverage spatiotemporal correlations of dynamic widefield imaging data to extend superresolution to simultaneous multiple emitter tracking without relying on photodynamics even as emitter distances from one another fall below the diffraction limit. We simultaneously determine emitter numbers and their tracks (localization and linking) with the same localization accuracy per frame as widefield superresolution does for immobilized emitters under similar imaging conditions (≈50nm). We demonstrate our results for both in cellulo data and, for benchmarking purposes, on synthetic data. To this end, we avoid the existing tracking paradigm relying on completely or partially separating the tasks of emitter number determination, localization of each emitter, and linking emitter positions across frames. Instead, we develop a fully joint posterior distribution over the quantities of interest, including emitter tracks and their total, otherwise unknown, number within the Bayesian nonparametric paradigm. Our posterior quantifies the full uncertainty over emitter numbers and their associated tracks propagated from origins including shot noise and camera artefacts, pixelation, stochastic background, and out-of-focus motion. Finally, it remains accurate in more crowded regimes where alternative tracking tools cannot be applied.

Rapid Temperature-Dependent Rheological Measurements of Non-Newtonian Solutions Using a Machine-Learning Aided Microfluidic Rheometer

Biofluids such as synovial fluid, blood plasma, and saliva contain several proteins which impart non-Newtonian properties to the biofluids. The concentration of such protein macromolecules in biofluids is regarded as an important biomarker for the diagnosis of several health conditions, including cardiovascular disorders, joint quality, and Alzheimer's. Existing technologies for the measurements of macromolecules in biofluids are limited; they require a long turnaround time, or require complex protocols, thus calling for alternative, more suitable, methodologies aimed at such measurements. According to the well-established relations for polymer solutions, the concentration of macromolecules in solutions can also be derived via measurement of rheological properties such as shear-viscosity and the longest relaxation time. We here introduce a microfluidic rheometer for rapid simultaneous measurement of shear viscosity and longest relaxation time of non-Newtonian solutions at different temperatures. At variance with previous technologies, our microfluidic rheometer provides a very short turnaround time of around 2 min or less thanks to the implementation of a machine-learning algorithm. We validated our platform on several aqueous solutions of poly(ethylene oxide). We also performed measurements on hyaluronic acid solutions in the clinical range for joint grade assessment. We observed monotonic behavior with the concentration for both rheological properties, thus speculating on their use as potential rheo-markers, i.e., rheological biomarkers, across several disease states.

A review for Cell and Particle Tracking on Microscopy Images using Algorithms and Deep Learning Technologies

Time-lapse microscopy images generated by biological experiments have been widely used for observing target activities, such as the motion trajectories and survival states. Based on these observations, biologists can conclude experimental results or present new hypotheses for several biological applications, i.e. virus research or drug design. Many methods or tools have been proposed in the past to observe cell and particle activities, which are defined as single cell tracking and single particle tracking problems, by using algorithms and deep learning technologies. In this article, a review for these works is presented in order to summarize the past methods and research topics at first, then points out the problems raised by these works, and finally proposes future research directions. The contributions of this article will help researchers to understand past development trends and further propose innovative technologies.

Deep probabilistic tracking of particles in fluorescence microscopy images

Tracking of particles in temporal fluorescence microscopy image sequences is of fundamental importance to quantify dynamic processes of intracellular structures as well as virus structures. We introduce a probabilistic deep learning approach for fluorescent particle tracking, which is based on a recurrent neural network that mimics classical Bayesian filtering. Compared to previous deep learning methods for particle tracking, our approach takes into account uncertainty, both aleatoric and epistemic uncertainty. Thus, information about the reliability of the computed trajectories is determined. Manual tuning of tracking parameters is not necessary and prior knowledge about the noise statistics is not required. Short and long-term temporal dependencies of individual object dynamics are exploited for state prediction, and assigned detections are used to update the predicted states. For correspondence finding, we introduce a neural network which computes assignment probabilities jointly across multiple detections as well as determines the probabilities of missing detections. Training requires only simulated data and therefore tedious manual annotation of ground truth is not needed. We performed a quantitative performance evaluation based on synthetic and real 2D as well as 3D fluorescence microscopy images. We used image data of the Particle Tracking Challenge as well as real time-lapse fluorescence microscopy images displaying virus structures and chromatin structures. It turned out that our approach yields state-of-the-art results or improves the tracking results compared to previous methods.

A Recurrent Neural Network for Particle Tracking in Microscopy Images Using Future Information, Track Hypotheses, and Multiple Detections

Automatic tracking of particles in time-lapse fluorescence microscopy images is essential for quantifying the dynamic behavior of subcellular structures and virus structures. We introduce a novel particle tracking approach based on a deep recurrent neural network architecture that exploits past and future information in both forward and backward direction. Assignment probabilities are determined jointly across multiple detections, and the probability of missing detections is computed. In addition, existence probabilities are determined by the network to handle track initiation and termination. For correspondence finding, track hypotheses are propagated to future time points so that information at later time points can be used to resolve ambiguities. A handcrafted similarity measure and handcrafted motion features are not necessary. Manually labeled data is not required for network training. We evaluated the performance of our approach using image data of the Particle Tracking Challenge as well as real fluorescence microscopy image sequences of virus structures. It turned out that the proposed approach outperforms previous methods.

Super-Resolution Contrast-Enhanced Ultrasound Methodology for the Identification of In Vivo Vascular Dynamics in 2D

OBJECTIVES

The aim of this study was to provide an ultrasound-based super-resolution methodology that can be implemented using clinical 2-dimensional ultrasound equipment and standard contrast-enhanced ultrasound modes. In addition, the aim is to achieve this for true-to-life patient imaging conditions, including realistic examination times of a few minutes and adequate image penetration depths that can be used to scan entire organs without sacrificing current super-resolution ultrasound imaging performance.

METHODS

Standard contrast-enhanced ultrasound was used along with bolus or infusion injections of SonoVue (Bracco, Geneva, Switzerland) microbubble (MB) suspensions. An image analysis methodology, translated from light microscopy algorithms, was developed for use with ultrasound contrast imaging video data. New features that are tailored for ultrasound contrast image data were developed for MB detection and segmentation, so that the algorithm can deal with single and overlapping MBs. The method was tested initially on synthetic data, then with a simple microvessel phantom, and then with in vivo ultrasound contrast video loops from sheep ovaries. Tracks detailing the vascular structure and corresponding velocity map of the sheep ovary were reconstructed. Images acquired from light microscopy, optical projection tomography, and optical coherence tomography were compared with the vasculature network that was revealed in the ultrasound contrast data. The final method was applied to clinical prostate data as a proof of principle.

RESULTS

Features of the ovary identified in optical modalities mentioned previously were also identified in the ultrasound super-resolution density maps. Follicular areas, follicle wall, vessel diameter, and tissue dimensions were very similar. An approximately 8.5-fold resolution gain was demonstrated in vessel width, as vessels of width down to 60 μm were detected and verified (λ = 514 μm). Best agreement was found between ultrasound measurements and optical coherence tomography with 10% difference in the measured vessel widths, whereas ex vivo microscopy measurements were significantly lower by 43% on average. The results were mostly achieved using video loops of under 2-minute duration that included respiratory motion. A feasibility study on a human prostate showed good agreement between density and velocity ultrasound maps with the histological evaluation of the location of a tumor.

CONCLUSIONS

The feasibility of a 2-dimensional contrast-enhanced ultrasound-based super-resolution method was demonstrated using in vitro, synthetic and in vivo animal data. The method reduces the examination times to a few minutes using state-of-the-art ultrasound equipment and can provide super-resolution maps for an entire prostate with similar resolution to that achieved in other studies.

A novel track initialization algorithm based on random sample consensus in dense clutter

Track initialization in dense clutter environments is an important topic and still a challenging task. Most traditional track initialization techniques firstly consider all possible associated measurement combinations and select the optimal one as an initialized track. Therefore, dense clutter brings great challenges to traditional algorithms. Random sample consensus algorithm, which is different from traditional algorithms, starts from minimum measurements. It samples randomly minimum measurements to establish hypotheses and verifies them through remaining measurements. However, the randomness of sampling influences algorithm performance, especially in dense clutter. A novel track initialization based on random sample consensus, named density-based random sample consensus algorithm, is proposed. It utilizes the fact that sequential measurements originating from the same target are contiguous while clutter is separated in space–time domain. The algorithm defines the density property of measurements to decrease the randomness in sampling procedure and increase the efficiency of track initialization. The experimental results show that the density-based random sample consensus is more superior to random sample consensus, the Hough transform algorithm, and logic-based algorithm.

A new method for quantifying mitochondrial axonal transport

Axonal transport of mitochondria is critical for neuronal survival and function. Automatically quantifying and analyzing mitochondrial movement in a large quantity remain challenging. Here, we report an efficient method for imaging and quantifying axonal mitochondrial transport using microfluidic-chamber-cultured neurons together with a newly developed analysis package named "MitoQuant". This tool-kit consists of an automated program for tracking mitochondrial movement inside live neuronal axons and a transient-velocity analysis program for analyzing dynamic movement patterns of mitochondria. Using this method, we examined axonal mitochondrial movement both in cultured mammalian neurons and in motor neuron axons of Drosophila in vivo. In 3 different paradigms (temperature changes, drug treatment and genetic manipulation) that affect mitochondria, we have shown that this new method is highly efficient and sensitive for detecting changes in mitochondrial movement. The method significantly enhanced our ability to quantitatively analyze axonal mitochondrial movement and allowed us to detect dynamic changes in axonal mitochondrial transport that were not detected by traditional kymographic analyses.

Automated single particle detection and tracking for large microscopy datasets

Recent advances in optical microscopy have enabled the acquisition of very large datasets from living cells with unprecedented spatial and temporal resolutions. Our ability to process these datasets now plays an essential role in order to understand many biological processes. In this paper, we present an automated particle detection algorithm capable of operating in low signal-to-noise fluorescence microscopy environments and handling large datasets. When combined with our particle linking framework, it can provide hitherto intractable quantitative measurements describing the dynamics of large cohorts of cellular components from organelles to single molecules. We begin with validating the performance of our method on synthetic image data, and then extend the validation to include experiment images with ground truth. Finally, we apply the algorithm to two single-particle-tracking photo-activated localization microscopy biological datasets, acquired from living primary cells with very high temporal rates. Our analysis of the dynamics of very large cohorts of 10 000 s of membrane-associated protein molecules show that they behave as if caged in nanodomains. We show that the robustness and efficiency of our method provides a tool for the examination of single-molecule behaviour with unprecedented spatial detail and high acquisition rates.

Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review

Digital pathology and microscopy image analysis is widely used for comprehensive studies of cell morphology or tissue structure. Manual assessment is labor intensive and prone to interobserver variations. Computer-aided methods, which can significantly improve the objectivity and reproducibility, have attracted a great deal of interest in recent literature. Among the pipeline of building a computer-aided diagnosis system, nucleus or cell detection and segmentation play a very important role to describe the molecular morphological information. In the past few decades, many efforts have been devoted to automated nucleus/cell detection and segmentation. In this review, we provide a comprehensive summary of the recent state-of-the-art nucleus/cell segmentation approaches on different types of microscopy images including bright-field, phase-contrast, differential interference contrast, fluorescence, and electron microscopies. In addition, we discuss the challenges for the current methods and the potential future work of nucleus/cell detection and segmentation.

Quantitative comparison of multiframe data association techniques for particle tracking in time-lapse fluorescence microscopy

Biological studies of intracellular dynamic processes commonly require motion analysis of large numbers of particles in live-cell time-lapse fluorescence microscopy imaging data. Many particle tracking methods have been developed in the past years as a first step toward fully automating this task and enabling high-throughput data processing. Two crucial aspects of any particle tracking method are the detection of relevant particles in the image frames and their linking or association from frame to frame to reconstruct the trajectories. The performance of detection techniques as well as specific combinations of detection and linking techniques for particle tracking have been extensively evaluated in recent studies. Comprehensive evaluations of linking techniques per se, on the other hand, are lacking in the literature. Here we present the results of a quantitative comparison of data association techniques for solving the linking problem in biological particle tracking applications. Nine multiframe and two more traditional two-frame techniques are evaluated as a function of the level of missing and spurious detections in various scenarios. The results indicate that linking techniques are generally more negatively affected by missing detections than by spurious detections. If misdetections can be avoided, there appears to be no need to use sophisticated multiframe linking techniques. However, in the practically likely case of imperfect detections, the latter are a safer choice. Our study provides users and developers with novel information to select the right linking technique for their applications, given a detection technique of known quality.

Micro-object motion tracking based on the probability hypothesis density particle tracker

Tracking micro-objects in the noisy microscopy image sequences is important for the analysis of dynamic processes in biological objects. In this paper, an automated tracking framework is proposed to extract the trajectories of micro-objects. This framework uses a probability hypothesis density particle filtering (PF-PHD) tracker to implement a recursive state estimation and trajectories association. In order to increase the efficiency of this approach, an elliptical target model is presented to describe the micro-objects using shape parameters instead of point-like targets which may cause inaccurate tracking. A novel likelihood function, not only covering the spatiotemporal distance but also dealing with geometric shape function based on the Mahalanobis norm, is proposed to improve the accuracy of particle weight in the update process of the PF-PHD tracker. Using this framework, a larger number of tracks are obtained. The experiments are performed on simulated data of microtubule movements and real mouse stem cells. We compare the PF-PHD tracker with the nearest neighbor method and the multiple hypothesis tracking method. Our PF-PHD tracker can simultaneously track hundreds of micro-objects in the microscopy image sequence.

A Novel Multiobject Tracking Approach in the Presence of Collision and Division

This paper aims to develop a general framework for accurately tracking and quantitatively characterizing multiple cells (objects) when collision and division between cells arise. Through introducing three types of interaction events among cells, namely, independence, collision, and division, the corresponding dynamic models are defined and an augmented interacting multiple model particle filter tracking algorithm is first proposed for spatially adjacent cells with varying size. In addition, to reduce the ambiguity of correspondence between frames, both the estimated cell dynamic parameters and cell size are further utilized to identify cells of interest. The experiments have been conducted on two real cell image sequences characterized with cells collision, division, or number variation, and the resulting dynamic parameters such as instant velocity, turn rate were obtained and analyzed.

Multi-Target Tracking With Time-Varying Clutter Rate and Detection Profile: Application to Time-Lapse Cell Microscopy Sequences

Quantitative analysis of the dynamics of tiny cellular and sub-cellular structures, known as particles, in time-lapse cell microscopy sequences requires the development of a reliable multi-target tracking method capable of tracking numerous similar targets in the presence of high levels of noise, high target density, complex motion patterns and intricate interactions. In this paper, we propose a framework for tracking these structures based on the random finite set Bayesian filtering framework. We focus on challenging biological applications where image characteristics such as noise and background intensity change during the acquisition process. Under these conditions, detection methods usually fail to detect all particles and are often followed by missed detections and many spurious measurements with unknown and time-varying rates. To deal with this, we propose a bootstrap filter composed of an estimator and a tracker. The estimator adaptively estimates the required meta parameters for the tracker such as clutter rate and the detection probability of the targets, while the tracker estimates the state of the targets. Our results show that the proposed approach can outperform state-of-the-art particle trackers on both synthetic and real data in this regime.

Tracking multiple particles in fluorescence time-lapse microscopy images via probabilistic data association

Tracking subcellular structures as well as viral structures displayed as 'particles' in fluorescence microscopy images yields quantitative information on the underlying dynamical processes. We have developed an approach for tracking multiple fluorescent particles based on probabilistic data association. The approach combines a localization scheme that uses a bottom-up strategy based on the spot-enhancing filter as well as a top-down strategy based on an ellipsoidal sampling scheme that uses the Gaussian probability distributions computed by a Kalman filter. The localization scheme yields multiple measurements that are incorporated into the Kalman filter via a combined innovation, where the association probabilities are interpreted as weights calculated using an image likelihood. To track objects in close proximity, we compute the support of each image position relative to the neighboring objects of a tracked object and use this support to recalculate the weights. To cope with multiple motion models, we integrated the interacting multiple model algorithm. The approach has been successfully applied to synthetic 2-D and 3-D images as well as to real 2-D and 3-D microscopy images, and the performance has been quantified. In addition, the approach was successfully applied to the 2-D and 3-D image data of the recent Particle Tracking Challenge at the IEEE International Symposium on Biomedical Imaging (ISBI) 2012.

A multiple model probability hypothesis density tracker for time-lapse cell microscopy sequences

Quantitative analysis of the dynamics of tiny cellular and subcellular structures in time-lapse cell microscopy sequences requires the development of a reliable multi-target tracking method capable of tracking numerous similar targets in the presence of high levels of noise, high target density, maneuvering motion patterns and intricate interactions. The linear Gaussian jump Markov system probability hypothesis density (LGJMS-PHD) filter is a recent Bayesian tracking filter that is well-suited for this task. However, the existing recursion equations for this filter do not consider a state-dependent transition probability matrix. As required in many biological applications, we propose a new closed-form recursion that incorporates this assumption and introduce a general framework for particle tracking using the proposed filter. We apply our scheme to multi-target tracking in total internal reflection fluorescence microscopy (TIRFM) sequences and evaluate the performance of our filter against the existing LGJMS-PHD and IMM-JPDA filters.

Secretory vesicles are preferentially targeted to areas of low molecular SNARE density

Intercellular communication is commonly mediated by the regulated fusion, or exocytosis, of vesicles with the cell surface. SNARE (soluble N-ethymaleimide sensitive factor attachment protein receptor) proteins are the catalytic core of the secretory machinery, driving vesicle and plasma membrane merger. Plasma membrane SNAREs (tSNAREs) are proposed to reside in dense clusters containing many molecules, thus providing a concentrated reservoir to promote membrane fusion. However, biophysical experiments suggest that a small number of SNAREs are sufficient to drive a single fusion event. Here we show, using molecular imaging, that the majority of tSNARE molecules are spatially separated from secretory vesicles. Furthermore, the motilities of the individual tSNAREs are constrained in membrane micro-domains, maintaining a non-random molecular distribution and limiting the maximum number of molecules encountered by secretory vesicles. Together our results provide a new model for the molecular mechanism of regulated exocytosis and demonstrate the exquisite organization of the plasma membrane at the level of individual molecular machines.