Red blood cell tracking using optical flow methods

Optical Flow-Based Full-Field Quantitative Blood-Flow Velocimetry Using Temporal Direction Filtering and Peak Interpolation

The quantitative measurement of the microvascular blood-flow velocity is critical to the early diagnosis of microvascular dysfunction, yet there are several challenges with the current quantitative flow velocity imaging techniques for the microvasculature. Optical flow analysis allows for the quantitative imaging of the blood-flow velocity with a high spatial resolution, using the variation in pixel brightness between consecutive frames to trace the motion of red blood cells. However, the traditional optical flow algorithm usually suffers from strong noise from the background tissue, and a significant underestimation of the blood-flow speed in blood vessels, due to the errors in detecting the feature points in optical images. Here, we propose a temporal direction filtering and peak interpolation optical flow method (TPIOF) to suppress the background noise, and improve the accuracy of the blood-flow velocity estimation. In vitro phantom experiments and in vivo animal experiments were performed to validate the improvements in our new method.

Cross-Ray Ultrasound Tomography and Photoacoustic Tomography of Cerebral Hemodynamics in Rodents

Recent advances in functional ultrasound imaging (fUS) and photoacoustic tomography (PAT) offer powerful tools for studying brain function. Complementing each other, fUS and PAT, respectively, measure the cerebral blood flow (CBF) and hemoglobin concentrations, allowing synergistic characterization of cerebral hemodynamics. Here, cross-ray ultrasound tomography (CRUST) and its combination with PAT are presented. CRUST employs a virtual point source from a spherically focused ultrasonic transducer (SFUST) to provide widefield excitation at a 4-kHz pulse repetition frequency. A full-ring-shaped ultrasonic transducer array whose imaging plane is orthogonal to the SFUST's acoustic axis receives scattered ultrasonic waves. Superior to conventional fUS, whose sensitivity to blood flow is angle-dependent and low for perpendicular flow, the crossed transmission and panoramic detection fields of CRUST provide omnidirectional sensitivity to CBF. Using CRUST-PAT, the CBF, oxygen saturation, and hemoglobin concentration changes of the mouse brain during sensory stimulation are measured, with a field of view of ≈7 mm in diameter, spatial resolution of ≈170 µm, and temporal resolution of 200 Hz. The results demonstrate CRUST-PAT as a unique tool for studying cerebral hemodynamics.

Ultrafast two-photon fluorescence imaging of cerebral blood circulation in the mouse brain in vivo

SignificanceCharacterizing blood flow by tracking individual red blood cells as they move through vessels is essential for understanding vascular function. With high spatial resolution, two-photon fluorescence microscopy is the method of choice for imaging blood flow at the cellular level. However, its application is limited to a low flow speed regimen in anesthetized animals by its slow focus scanning mechanism. Using an ultrafast scanning module, we demonstrated two-photon fluorescence imaging of blood flow at 1,000 two-dimensional frames and 1,000,000 one-dimensional line scans per second in the brains of awake mice. These ultrafast measurements enabled us to study hemodynamic and fluid mechanical regimens beyond the reach of conventional methods.

Cell Trafficking at the Intersection of the Tumor-Immune Compartments

Migration is an essential cellular process that regulates human organ development and homeostasis as well as disease initiation and progression. In cancer, immune and tumor cell migration is strongly associated with immune cell infiltration, immune escape, and tumor cell metastasis, which ultimately account for more than 90% of cancer deaths. The biophysics and molecular regulation of the migration of cancer and immune cells have been extensively studied separately. However, accumulating evidence indicates that, in the tumor microenvironment, the motilities of immune and cancer cells are highly interdependent via secreted factors such as cytokines and chemokines. Tumor and immune cells constantly express these soluble factors, which produce a tightly intertwined regulatory network for these cells' respective migration. A mechanistic understanding of the reciprocal regulation of soluble factor-mediated cell migration can provide critical information for the development of new biomarkers of tumor progression and of tumor response to immuno-oncological treatments. We review the biophysical andbiomolecular basis for the migration of immune and tumor cells and their associated reciprocal regulatory network. We also describe ongoing attempts to translate this knowledge into the clinic.

Visual Breast Asymmetry Assessment with Optical-Flow Algorithm

Breast surgery is one of the most important procedures in cosmetic and reconstructive surgery. However, there is no ideal method to assess results. One of the greatest difficulties is the subjective aspect of evaluation. In recent years, several objective computer systems have been proposed and validated as assessment methods, such as BCCT®, OBCS®, GBAI©, etc. In this study, we propose a novel system named VIBA©, that uses an Optical Flow (OF) algorithm which objectively classifies results into symmetrical and asymmetrical categories, with a numerical score. Software was developed in MATLAB (MATLAB and Statistics Toolbox Release 2018b, The MathWorks, Inc., Natick, Massachusetts, USA) called VIBA-Calc© (VIBA stands for VIsual Breast Asymmetry). We compared our OF score with the well-established asymmetry scoring system called Objective Breast Cosmesis Scale (OBCS®). In order to do so, we studied 100 frontal photographs of patients who underwent aesthetic breast surgery between 2017 and 2018, from the senior author's private practice. VIBA-Calc© allows the user to load an image and then draw a rectangle containing both breasts. By simply clicking on a button, the program finds the midline of the rectangle and calculates the final score, as well as the color map of asymmetric regions. Classification into symmetric or asymmetric categories using OBCS and VIBA scores agreed in most cases. Concordance between both classification systems was almost perfect in the group of postoperative cases (k = 0.84; p < 0.001), and substantial in preoperative cases (k = 0.76; p < 0.001). Global Cohen's kappa coefficient was 0.80 (p < 0.001). VIBA© is a useful tool for pre- and post-operative evaluation of breasts, that could be used both in reconstructive and aesthetic surgery.

Temporally coherent cardiac motion tracking from cine MRI: Traditional registration method and modern CNN method

Purpose

Cardiac motion tracking enables quantitative evaluation of myocardial strain, which is clinically interesting in cardiovascular disease research. However, motion tracking is difficult to perform manually. In this paper, we aim to develop and compare two fully automated motion tracking methods for the steady state free precession (SSFP) cine magnetic resonance imaging (MRI), and explore their use in real clinical scenario with different patient groups.

Methods

We proposed two automated cardiac motion tracking method: (a) a traditional registration‐based method, named full cardiac cycle registration, which simultaneously tracks all cine frames within a full cardiac cycle by joint registration of all frames; and (b) a modern convolutional neural network (CNN)‐based method, named Groupwise MotionNet, which enhances the temporal coherence by fusing motion along a continuous time scale. Both methods were evaluated on the healthy volunteer data from the MICCAI 2011 STACOM Challenge, as well as on patient data including hypertrophic cardiomyopathy (HCM) and myocardial infarction (MI).

Results

The full cardiac cycle registration method achieved an average end‐point error (EPE) 2.89 ± 1.57 mm for cardiac motion tracking, with computation time of around 9 min per short‐axis cine MRI (size 128 × 128, 30 cardiac phases). In comparison, the Groupwise MotionNet achieved an average EPE of 0.94 ± 1.59 mm, taking < 1 s for a full cardiac phases. Further experiments showed that registration method had stable performance, independent of patient cohort and MRI machine, while the CNN‐based method relied on the training data to deliver consistently accurate results.

Conclusion

Both registration‐based and CNN‐based method can track the cardiac motion from SSFP cine MRI in a fully automated manner, while taking temporal coherence into account. The registration method is generic, robust, but relatively slow; the CNN‐based method trained with heterogeneous data was able to achieve high tracking accuracy with real‐time performance.

Optical flow analysis reveals that Kinesin-mediated advection impacts the orientation of microtubules in the Drosophila oocyte

The orientation of microtubule (MT) networks is exploited by motors to deliver cargoes to specific intracellular destinations and is thus essential for cell polarity and function. Reconstituted in vitro systems have largely contributed to understanding the molecular framework regulating the behavior of MT filaments. In cells, however, MTs are exposed to various biomechanical forces that might impact on their orientation, but little is known about it. Oocytes, which display forceful cytoplasmic streaming, are excellent model systems to study the impact of motion forces on cytoskeletons in vivo. Here we implement variational optical flow analysis as a new approach to analyze the polarity of MTs in the Drosophila oocyte, a cell that displays distinct Kinesin-dependent streaming. After validating the method as robust for describing MT orientation from confocal movies, we find that increasing the speed of flows results in aberrant plus end growth direction. Furthermore, we find that in oocytes where Kinesin is unable to induce cytoplasmic streaming, the growth direction of MT plus ends is also altered. These findings lead us to propose that cytoplasmic streaming - and thus motion by advection – contributes to the correct orientation of MTs in vivo. Finally, we propose a possible mechanism for a specialized cytoplasmic actin network (the actin mesh) to act as a regulator of flow speeds to counteract the recruitment of Kinesin to MTs.

In-vivo full-field measurement of microcirculatory blood flow velocity based on intelligent object identification

Microcirculation plays a crucial role in delivering oxygen and nutrients to living tissues and in removing metabolic wastes from the human body. Monitoring the velocity of blood flow in microcirculation is essential for assessing various diseases, such as diabetes, cancer, and critical illnesses. Because of the complex morphological pattern of the capillaries, both In-vivo capillary identification and blood flow velocity measurement by conventional optical capillaroscopy are challenging. Thus, we focused on developing an In-vivo optical microscope for capillary imaging, and we propose an In-vivo full-field flow velocity measurement method based on intelligent object identification. The proposed method realizes full-field blood flow velocity measurements in microcirculation by employing a deep neural network to automatically identify and distinguish capillaries from images. In addition, a spatiotemporal diagram analysis is used for flow velocity calculation. In-vivo experiments were conducted, and the images and videos of capillaries were collected for analysis. We demonstrated that the proposed method is highly accurate in performing full-field blood flow velocity measurements in microcirculation. Further, because this method is simple and inexpensive, it can be effectively employed in clinics.

Effects of Cooling on the SNR and Contrast Detection of a Low-Light Event-Based Camera

Johnson-Nyquist noise is the electronic noise generated by the thermal agitation of charge carriers, which increases when the sensor overheats. Current high-speed cameras used in low-light conditions are often cooled down to reduce thermal noise and increase their signal to noise ratio. These sensors, however, record hundreds of frames per second, which takes time, requires energy, and heavy computing power due to the substantial data load. Event-based sensors benefit from a high temporal resolution and record the information in a sparse manner. Based on an asynchronous time-based image sensor, we developed another version of this event-based camera whose pixels were designed for low-light applications and added a Peltier-effect-based cooling system at the back of the sensor in order to reduce thermal noise. We show the benefits from thermal noise reduction and study the improvement of the signal to noise ratio in the estimation of event-based normal flow norm and angle and particle tracking in microscopy.

A Ciliary Motility Index for Activity Measurement in Cell Cultures With Respiratory Syncytial Virus

BACKGROUND

The respiratory epithelium is frequently infected by the respiratory syncytial virus, resulting in inflammation, a reduction in cilia activity and an increase in the production of mucus.

METHODS

In this study, an automatic method has been proposed to characterize the ciliary motility from cell cultures by means of a motility index using a dense optical flow algorithm. This method allows us to determine the ciliary beat frequency (CBF) together with a ciliary motility index of the cells in the cultures. The object of this analysis is to automatically distinguish between normal and infected cells in a culture.

RESULTS

The method was applied in 2 stages. It was concluded from the first stage that the CBF is not a good enough indicator to discriminate between the control and infected cultures. However, the ciliary motility index does succeed in discriminating between the control and infected cultures using the t test with a value t = 6.46 and P < .001. In the second stage, it has been shown that the ciliary motility index did not differ significantly between patients, and the analysis of variance test gives α = 0.05, F = 1.61, P = .20. A threshold for this index has been determined using a receiver operating characteristics analysis that gives an area under the curve of 0.93.

CONCLUSIONS

We have obtained a ciliary motility index that is able to discriminate between control and infected cultures after the eighth postinfection day. After infection, there is a rapid cilia loss of the cells and the measured CBF corresponds to the remaining noninfected cells. This is why the CBF does not discriminate between the control and the infected cells.

Quantitative Analysis of Intracellular Motility Based on Optical Flow Model

Analysis of cell mobility is a key issue for abnormality identification and classification in cell biology research. However, since cell deformation induced by various biological processes is random and cell protrusion is irregular, it is difficult to measure cell morphology and motility in microscopic images. To address this dilemma, we propose an improved variation optical flow model for quantitative analysis of intracellular motility, which not only extracts intracellular motion fields effectively but also deals with optical flow computation problem at the border by taking advantages of the formulation based on L₁ and L₂ norm, respectively. In the energy functional of our proposed optical flow model, the data term is in the form of L₂ norm; the smoothness of the data changes with regional features through an adaptive parameter, using L₁ norm near the edge of the cell and L₂ norm away from the edge. We further extract histograms of oriented optical flow (HOOF) after optical flow field of intracellular motion is computed. Then distances of different HOOFs are calculated as the intracellular motion features to grade the intracellular motion. Experimental results show that the features extracted from HOOFs provide new insights into the relationship between the cell motility and the special pathological conditions.

Pixel-Level and Robust Vibration Source Sensing in High-Frame-Rate Video Analysis

We investigate the effect of appearance variations on the detectability of vibration feature extraction with pixel-level digital filters for high-frame-rate videos. In particular, we consider robust vibrating object tracking, which is clearly different from conventional appearance-based object tracking with spatial pattern recognition in a high-quality image region of a certain size. For 512 × 512 videos of a rotating fan located at different positions and orientations and captured at 2000 frames per second with different lens settings, we verify how many pixels are extracted as vibrating regions with pixel-level digital filters. The effectiveness of dynamics-based vibration features is demonstrated by examining the robustness against changes in aperture size and the focal condition of the camera lens, the apparent size and orientation of the object being tracked, and its rotational frequency, as well as complexities and movements of background scenes. Tracking experiments for a flying multicopter with rotating propellers are also described to verify the robustness of localization under complex imaging conditions in outside scenarios.

In vivo flow mapping in complex vessel networks by single image correlation

We describe a novel method (FLICS, FLow Image Correlation Spectroscopy) to extract flow speeds in complex vessel networks from a single raster-scanned optical xy-image, acquired in vivo by confocal or two-photon excitation microscopy. Fluorescent flowing objects produce diagonal lines in the raster-scanned image superimposed to static morphological details. The flow velocity is obtained by computing the Cross Correlation Function (CCF) of the intensity fluctuations detected in pairs of columns of the image. The analytical expression of the CCF has been derived by applying scanning fluorescence correlation concepts to drifting optically resolved objects and the theoretical framework has been validated in systems of increasing complexity. The power of the technique is revealed by its application to the intricate murine hepatic microcirculatory system where blood flow speed has been mapped simultaneously in several capillaries from a single xy-image and followed in time at high spatial and temporal resolution.