Optical flow: a curve evolution approach

A Variational Approach to Video Registration with Subspace Constraints

This paper addresses the problem of non-rigid video registration, or the computation of optical flow from a reference frame to each of the subsequent images in a sequence, when the camera views deformable objects. We exploit the high correlation between 2D trajectories of different points on the same non-rigid surface by assuming that the displacement of any point throughout the sequence can be expressed in a compact way as a linear combination of a low-rank motion basis. This subspace constraint effectively acts as a trajectory regularization term leading to temporally consistent optical flow. We formulate it as a robust soft constraint within a variational framework by penalizing flow fields that lie outside the low-rank manifold. The resulting energy functional can be decoupled into the optimization of the brightness constancy and spatial regularization terms, leading to an efficient optimization scheme. Additionally, we propose a novel optimization scheme for the case of vector valued images, based on the dualization of the data term. This allows us to extend our approach to deal with colour images which results in significant improvements on the registration results. Finally, we provide a new benchmark dataset, based on motion capture data of a flag waving in the wind, with dense ground truth optical flow for evaluation of multi-frame optical flow algorithms for non-rigid surfaces. Our experiments show that our proposed approach outperforms state of the art optical flow and dense non-rigid registration algorithms.

Pixelwise-adaptive blind optical flow assuming nonstationary statistics

In this paper, we address some of the major issues in optical flow within a new framework assuming nonstationary statistics for the motion field and for the errors. Problems addressed include the preservation of discontinuities, model/data errors, outliers, confidence measures, and performance evaluation. In solving these problems, we assume that the statistics of the motion field and the errors are not only spatially varying, but also unknown. We, thus, derive a blind adaptive technique based on generalized cross validation for estimating an independent regularization parameter for each pixel. Our formulation is pixelwise and combines existing first- and second-order constraints with a new second-order temporal constraint. We derive a new confidence measure for an adaptive rejection of erroneous and outlying motion vectors, and compare our results to other techniques in the literature. A new performance measure is also derived for estimating the signal-to-noise ratio for real sequences when the ground truth is unknown.

Fast two-frame multiscale dense optical flow estimation using discrete wavelet filters

A multiscale algorithm with complexity O(N) (where N is the number of pixels in one image) using wavelet filters is proposed to estimate dense optical flow from two frames. Hierarchical image representation by wavelet decomposition is integrated with differential techniques in a new multiscale framework. It is shown that if a compactly supported wavelet basis with one vanishing moment is carefully selected, hierarchical image, first-order derivative, and corner representations can be obtained from the wavelet decomposition. On the basis of this result, three of the four components of the wave let decomposition are employed to estimate dense optical flow with use of only two frames. This overcomes the "flattening-out" problem in traditional pyramid methods, which produce large errors when low-texture regions become flat at coarse levels as a result of blurring. A two-dimensional affine motion model is used to formulate the optical flow problem as a linear system, with all resolutions simultaneously (i.e., coarse-and-fine) rather than the traditional coarse-to-fine approach, which unavoidably propagates errors from the coarse level. This not only helps to improve the accuracy but also makes the hardware implementation of our algorithm simple. Experiments on different types of image sequences, together with quantitative and qualitative comparisons with several other optical flow methods, are given to demonstrate the effectiveness and the robustness of our algorithm.

Image registration via level-set motion: applications to atlas-based segmentation

Image registration is an often encountered problem in various fields including medical imaging, computer vision and image processing. Numerous algorithms for registering image data have been reported in these areas. In this paper, we present a novel curve evolution approach expressed in a level-set framework to achieve image intensity morphing and a simple non-linear PDE for the corresponding coordinate registration. The key features of the intensity morphing model are that (a) it is very fast and (b) existence and uniqueness of the solution for the evolution model are established in a Sobolev space as opposed to using viscosity methods. The salient features of the coordinate registration model are its simplicity and computational efficiency. The intensity morph is easily achieved via evolving level-sets of one image into the level-sets of the other. To explicitly estimate the coordinate transformation between the images, we derive a non-linear PDE-based motion model which can be solved very efficiently. We demonstrate the performance of our algorithm on a variety of images including synthetic and real data. As an application of the PDE-based motion model, atlas based segmentation of hippocampal shape from several MR brain scans is depicted. In each of these experiments, automated hippocampal shape recovery results are validated via manual "expert" segmentations.

Adaptive, template moderated, spatially varying statistical classification

A novel image segmentation algorithm was developed to allow the automatic segmentation of both normal and abnormal anatomy from medical images. The new algorithm is a form of spatially varying statistical classification, in which an explicit anatomical template is used to moderate the segmentation obtained by statistical classification. The algorithm consists of an iterated sequence of spatially varying classification and nonlinear registration, which forms an adaptive, template moderated (ATM), spatially varying statistical classification (SVC). Classification methods and nonlinear registration methods are often complementary, both in the tasks where they succeed and in the tasks where they fail. By integrating these approaches the new algorithm avoids many of the disadvantages of each approach alone while exploiting the combination. The ATM SVC algorithm was applied to several segmentation problems, involving different image contrast mechanisms and different locations in the body. Segmentation and validation experiments were carried out for problems involving the quantification of normal anatomy (MRI of brains of neonates) and pathology of various types (MRI of patients with multiple sclerosis, MRI of patients with brain tumors, MRI of patients with damaged knee cartilage). In each case, the ATM SVC algorithm provided a better segmentation than statistical classification or elastic matching alone.

Application of the extremum stack to neurological MRI

The extremum stack, as proposed by Koenderink, is a multiresolution image description and segmentation scheme which examines intensity extrema (minima and maxima) as they move and merge through a series of progressively isotropically diffused images known as scale space. Such a data-driven approach is attractive because it is claimed to be a generally applicable and natural method of image segmentation. The performance of the extremum stack is evaluated here using the case of neurological magnetic resonance imaging data as a specific example, and means of improving its performance proposed. It is confirmed experimentally that the extremum stack has the desirable property of being shift-, scale-, and rotation-invariant, and produces natural results for many compact regions of anatomy. It handles elongated objects poorly, however, and subsections of regions may merge prematurely before each region is represented as a single node. It is shown that this premature merging can often be avoided by the application of either a variable conductance-diffusing preprocessing step, or more effectively, the use of an adaptive variable conductance diffusion method within the extremum stack itself in place of the isotropic Gaussian diffusion proposed by Koenderink.