Multiframe temporal estimation of cardiac nonrigid motion

Cardiac motion estimation from medical images: a regularisation framework applied on pairwise image registration displacement fields

Accurate cardiac motion estimation from medical images such as ultrasound is important for clinical evaluation. We present a novel regularisation layer for cardiac motion estimation that will be applied after image registration and demonstrate its effectiveness. The regularisation utilises a spatio-temporal model of motion, b-splines of Fourier, to fit to displacement fields from pairwise image registration. In the process, it enforces spatial and temporal smoothness and consistency, cyclic nature of cardiac motion, and better adherence to the stroke volume of the heart. Flexibility is further given for inclusion of any set of registration displacement fields. The approach gave high accuracy. When applied to human adult Ultrasound data from a Cardiac Motion Analysis Challenge (CMAC), the proposed method is found to have 10% lower tracking error over CMAC participants. Satisfactory cardiac motion estimation is also demonstrated on other data sets, including human fetal echocardiography, chick embryonic heart ultrasound images, and zebrafish embryonic microscope images, with the average Dice coefficient between estimation motion and manual segmentation at 0.82-0.87. The approach of performing regularisation as an add-on layer after the completion of image registration is thus a viable option for cardiac motion estimation that can still have good accuracy. Since motion estimation algorithms are complex, dividing up regularisation and registration can simplify the process and provide flexibility. Further, owing to a large variety of existing registration algorithms, such an approach that is usable on any algorithm may be useful.

Sparse Data-Driven Learning for Effective and Efficient Biomedical Image Segmentation

Sparsity is a powerful concept to exploit for high-dimensional machine learning and associated representational and computational efficiency. Sparsity is well suited for medical image segmentation. We present a selection of techniques that incorporate sparsity, including strategies based on dictionary learning and deep learning, that are aimed at medical image segmentation and related quantification.

Spatiotemporal Strategies for Joint Segmentation and Motion Tracking From Cardiac Image Sequences

Although accurate and robust estimations of the deforming cardiac geometry and kinematics from cine tomographic medical image sequences remain a technical challenge, they have significant clinical value. Traditionally, boundary or volumetric segmentation and motion estimation problems are considered as two sequential steps, even though the order of these processes can be different. In this paper, we present an integrated, spatiotemporal strategy for the simultaneous joint recovery of these two ill-posed problems. We use a mesh-free Galerkin formulation as the representation and computation platform, and adopt iterative procedures to solve the governing equations. Specifically, for each nodal point, the external driving forces are individually constructed through the integration of data-driven edginess measures, prior spatial distributions of myocardial tissues, temporal coherence of image-derived salient features, imaging/image-derived Eulerian velocity information, and cyclic motion model of myocardial behavior. The proposed strategy is accurate and very promising application results are shown from synthetic data, magnetic resonance (MR) phase contrast, tagging image sequences, and gradient echo cine MR image sequences.

Temporally diffeomorphic cardiac motion estimation from three-dimensional echocardiography by minimization of intensity consistency error

PURPOSE

Quantitative analysis of cardiac motion is important for evaluation of heart function. Three dimensional (3D) echocardiography is among the most frequently used imaging modalities for motion estimation because it is convenient, real-time, low-cost, and nonionizing. However, motion estimation from 3D echocardiographic sequences is still a challenging problem due to low image quality and image corruption by noise and artifacts.

METHODS

The authors have developed a temporally diffeomorphic motion estimation approach in which the velocity field instead of the displacement field was optimized. The optimal velocity field optimizes a novel similarity function, which we call the intensity consistency error, defined as multiple consecutive frames evolving to each time point. The optimization problem is solved by using the steepest descent method.

RESULTS

Experiments with simulated datasets, images of anex vivo rabbit phantom, images of in vivo open-chest pig hearts, and healthy human images were used to validate the authors' method. Simulated and real cardiac sequences tests showed that results in the authors' method are more accurate than other competing temporal diffeomorphic methods. Tests with sonomicrometry showed that the tracked crystal positions have good agreement with ground truth and the authors' method has higher accuracy than the temporal diffeomorphic free-form deformation (TDFFD) method. Validation with an open-access human cardiac dataset showed that the authors' method has smaller feature tracking errors than both TDFFD and frame-to-frame methods.

CONCLUSIONS

The authors proposed a diffeomorphic motion estimation method with temporal smoothness by constraining the velocity field to have maximum local intensity consistency within multiple consecutive frames. The estimated motion using the authors' method has good temporal consistency and is more accurate than other temporally diffeomorphic motion estimation methods.

Tracking the motion trajectories of junction structures in 4D CT images of the lung

Respiratory motion poses a major challenge in lung radiotherapy. Based on 4D CT images, a variety of intensity-based deformable registration techniques have been proposed to study the pulmonary motion. However, the accuracy achievable with these approaches can be sub-optimal because the deformation is defined globally in space. Therefore, the accuracy of the alignment of local structures may be compromised. In this work, we propose a novel method to detect a large collection of natural junction structures in the lung and use them as the reliable markers to track the lung motion. Specifically, detection of the junction centers and sizes is achieved by analysis of local shape profiles on one segmented image. To track the temporal trajectory of a junction, the image intensities within a small region of interest surrounding the center are selected as its signature. Under the assumption of the cyclic motion, we describe the trajectory by a closed B-spline curve and search for the control points by maximizing a metric of combined correlation coefficients. Local extrema are suppressed by improving the initial conditions using random walks from pair-wise optimizations. Several descriptors are introduced to analyze the motion trajectories. Our method was applied to 13 real 4D CT images. More than 700 junctions in each case are detected with an average positive predictive value of greater than 90%. The average tracking error between automated and manual tracking is sub-voxel and smaller than the published results using the same set of data.

Cardiac motion and deformation recovery from MRI: a review

Magnetic resonance imaging (MRI) is a highly advanced and sophisticated imaging modality for cardiac motion tracking and analysis, capable of providing 3D analysis of global and regional cardiac function with great accuracy and reproducibility. In the past few years, numerous efforts have been devoted to cardiac motion recovery and deformation analysis from MR image sequences. Many approaches have been proposed for tracking cardiac motion and for computing deformation parameters and mechanical properties of the heart from a variety of cardiac MR imaging techniques. In this paper, an updated and critical review of cardiac motion tracking methods including major references and those proposed in the past ten years is provided. The MR imaging and analysis techniques surveyed are based on cine MRI, tagged MRI, phase contrast MRI, DENSE, and SENC. This paper can serve as a tutorial for new researchers entering the field.

Registration of 4D cardiac CT sequences under trajectory constraints with multichannel diffeomorphic demons

We propose a framework for the nonlinear spatiotemporal registration of 4D time-series of images based on the Diffeomorphic Demons (DD) algorithm. In this framework, the 4D spatiotemporal registration is decoupled into a 4D temporal registration, defined as mapping physiological states, and a 4D spatial registration, defined as mapping trajectories of physical points. Our contribution focuses more specifically on the 4D spatial registration that should be consistent over time as opposed to 3D registration that solely aims at mapping homologous points at a given time-point. First, we estimate in each sequence the motion displacement field, which is a dense representation of the point trajectories we want to register. Then, we perform simultaneously 3D registrations of corresponding time-points with the constraints to map the same physical points over time called the trajectory constraints. Under these constraints, we show that the 4D spatial registration can be formulated as a multichannel registration of 3D images. To solve it, we propose a novel version of the Diffeomorphic Demons (DD) algorithm extended to vector-valued 3D images, the Multichannel Diffeomorphic Demons (MDD). For evaluation, this framework is applied to the registration of 4D cardiac computed tomography (CT) sequences and compared to other standard methods with real patient data and synthetic data simulated from a physiologically realistic electromechanical cardiac model. Results show that the trajectory constraints act as a temporal regularization consistent with motion whereas the multichannel registration acts as a spatial regularization. Finally, using these trajectory constraints with multichannel registration yields the best compromise between registration accuracy, temporal and spatial smoothness, and computation times. A prospective example of application is also presented with the spatiotemporal registration of 4D cardiac CT sequences of the same patient before and after radiofrequency ablation (RFA) in case of atrial fibrillation (AF). The intersequence spatial transformations over a cardiac cycle allow to analyze and quantify the regression of left ventricular hypertrophy and its impact on the cardiac function.

Regional heart motion abnormality detection via information measures and unscented Kalman filtering

This study investigates regional heart motion abnormality detection using various classifier features with Shannon's Differential Entropy (SDE). Rather than relying on elementary measurements or a fixed set of moments, the SDE measures global distribution information and, as such, has more discriminative power in classifying distributions. Based on functional images, which are subject to noise and segmentation inaccuracies, heart wall motion analysis is acknowledged as a difficult problem and, therefore, incorporation of prior knowledge is desirable to enhance the accuracy. Given noisy data and nonlinear dynamic model to describe the myocardial motion, unscented Kalman filter, a recursive nonlinear Bayesian filter, is devised in this study so as to estimate LV cavity points. Subsequently, a naive Bayes classifier algorithm is constructed from the SDEs of different features in order to automatically detect abnormal functional regions of the myocardium. Using 90 x 20 segmented LV cavities of short-axis magnetic resonance images obtained from 30 subjects, the experimental analysis carried over 480 myocardial segments demonstrates that the proposed method perform significantly better than other recent methods, and can lead to a promising diagnostic support tool to assist clinicians.

Estimating Myocardial Motion by 4D Image Warping

A method for spatio-temporally smooth and consistent estimation of cardiac motion from MR cine sequences is proposed. Myocardial motion is estimated within a 4-dimensional (4D) registration framework, in which all 3D images obtained at different cardiac phases are simultaneously registered. This facilitates spatio-temporally consistent estimation of motion as opposed to other registration-based algorithms which estimate the motion by sequentially registering one frame to another. To facilitate image matching, an attribute vector (AV) is constructed for each point in the image, and is intended to serve as a "morphological signature" of that point. The AV includes intensity, boundary, and geometric moment invariants (GMIs). Hierarchical registration of two image sequences is achieved by using the most distinctive points for initial registration of two sequences and gradually adding less-distinctive points to refine the registration. Experimental results on real data demonstrate good performance of the proposed method for cardiac image registration and motion estimation. The motion estimation is validated via comparisons with motion estimates obtained from MR images with myocardial tagging.

Maximum a posteriori strategy for the simultaneous motion and material property estimation of the heart

In addition to its technical merits as a challenging nonrigid motion and structural integrity analysis problem, quantitative estimation of cardiac regional functions and material characteristics has significant physiological and clinical value. We developed a stochastic finite-element framework for the simultaneous recovery of myocardial motion and material parameters from medical image sequences with an extended Kalman filter approach, and we have shown that this simultaneous estimation strategy achieves more accurate and robust results than separated motion and material estimation efforts. In this paper, we present a new computational strategy for the framework based upon the maximum a posteriori estimation principles, realized through the extended Kalman smoother, that produces a sequence of kinematics state and material parameter estimation of the entire myocardium, including the endocardial, epicardial, and midwall tissues. The system dynamics equations of the heart are constructed using a biomechanical model with stochastic parameters, and the tissue material and deformation parameters are jointly estimated from the periodic imaging data. Noise-corrupted synthetic image sequences with known kinematics and material parameters are used to assess the accuracy and robustness of the framework. Experiments with canine magnetic resonance tagging and phase-contrast image sequences have been conducted with very promising results, as validated through comparison to the histological staining of postmortem myocardium.

Four-dimensional image registration for image-guided radiotherapy

PURPOSE

Newly emerged four-dimensional (4D) imaging techniques such as 4D-computed tomography (CT), 4D-cone beam CT, 4D-magnetic resonance imaging, and 4D-positron emission tomography are effective tools to reveal the spatiotemporal details of patients' anatomy. To use the 4D data acquired under different conditions or using different modalities, an algorithm for registering 4D images must be in place. We developed an automated 4D-4D registration method to take advantage of 4D information.

METHODS AND MATERIALS

We used 4D-4D matching to find the appropriate three-dimensional anatomy in the fixed image for each phase of the moving image and spatially register them. A search algorithm was implemented to simultaneously find the best phase and spatial match of two 4D inputs. An interpolation scheme capable of deriving an image set based on temporally adjacent three-dimensional data sets was developed to deal with the situation in which the discrete temporal points of the two inputs do not coincide or correspond.

RESULTS

In a phantom study, our technique was able to reproduce the known "ground truth" with high spatial fidelity. The technique regenerated all deliberately introduced "missing" three-dimensional images at different phases of the input using temporal interpolation. In the registration of gated-magnetic resonance imaging and 4D-CT, the algorithm was able to select the appropriate CT phase. The technique was also able to register 4D-CT with 4D-cone beam CT and two 4D-CT scans acquired at different times. A spatial accuracy of <3 mm was achieved in 98% of voxels in all cases.

CONCLUSION

Automated 4D-4D registration can find the best possible spatiotemporal match between two 4D data sets and is useful for image-guided radiotherapy applications.

State-space analysis of cardiac motion with biomechanical constraints

Quantitative estimation of nonrigid motion from image sequences has important technical and practical significance. State-space analysis provides powerful and convenient ways to construct and incorporate the physically meaningful system dynamics of an object, the image-derived observations, and the process and measurement noise disturbances. In this paper, we present a biomechanical-model constrained state-space analysis framework for the multiframe estimation of the periodic cardiac motion and deformation. The physical constraints take the roles as spatial regulator of the myocardial behavior and spatial filter/interpolator of the data measurements, while techniques from statistical filtering theory impose spatiotemporal constraints to facilitate the incorporation of multiframe information to generate optimal estimates of the heart kinematics. Physiologically meaningful results have been achieved from estimated displacement fields and strain maps using in vivo left ventricular magnetic resonance tagging and phase contrast image sequences, which provide the tag-tag and tag-boundary displacement inputs, and the mid-wall instantaneous velocity information and boundary displacement measures, respectively.

Spatio-temporal nonrigid registration for ultrasound cardiac motion estimation

We propose a new spatio-temporal elastic registration algorithm for motion reconstruction from a series of images. The specific application is to estimate displacement fields from two-dimensional ultrasound sequences of the heart. The basic idea is to find a spatio-temporal deformation field that effectively compensates for the motion by minimizing a difference with respect to a reference frame. The key feature of our method is the use of a semi-local spatio-temporal parametric model for the deformation using splines, and the reformulation of the registration task as a global optimization problem. The scale of the spline model controls the smoothness of the displacement field. Our algorithm uses a multiresolution optimization strategy to obtain a higher speed and robustness. We evaluated the accuracy of our algorithm using a synthetic sequence generated with an ultrasound simulation package, together with a realistic cardiac motion model. We compared our new global multiframe approach with a previous method based on pairwise registration of consecutive frames to demonstrate the benefits of introducing temporal consistency. Finally, we applied the algorithm to the regional analysis of the left ventricle. Displacement and strain parameters were evaluated showing significant differences between the normal and pathological segments, thereby illustrating the clinical applicability of our method.

Consistent Estimation of Cardiac Motions by 4D Image Registration

A 4D image registration method is proposed for consistent estimation of cardiac motion from MR image sequences. Under this 4D registration framework, all 3D cardiac images taken at different time-points are registered simultaneously, and motion estimated is enforced to be spatiotemporally smooth, thereby overcoming potential limitations of some methods that typically estimate cardiac deformation sequentially from one frame to another, instead of treating the entire set of images as a 4D volume. To facilitate our image matching process, an attribute vector is designed for each point in the image to include intensity, boundary and geometric moment invariants (GMIs). Hierarchical registration of two image sequences is achieved by using the most distinctive points for initial registration of two sequences and gradually adding less-distinctive points for refinement of registration. Experimental results on real data demonstrate good performance of the proposed method in registering cardiac images and estimating motions from cardiac image sequences.

Stochastic finite element framework for simultaneous estimation of cardiac kinematic functions and material parameters

A stochastic finite element framework is presented for the simultaneous estimation of the cardiac kinematic functions and material model parameters from periodic medical image sequences. While existing biomechanics studies of the myocardial material constitutive laws have assumed known tissue kinematic measurements, and image analysis efforts on cardiac kinematic functions have relied on fixed constraining models of mathematical or mechanical nature, we illustrate through synthetic data that a probabilistic joint estimation strategy is needed to achieve more robust and accurate analysis of the kinematic functions and material parameters at the same time. For a particular a priori constraining material model with uncertain subject-dependent parameters and a posteriori noisy imaging based observations, our strategy combines the stochastic differential equations of the myocardial dynamics with the finite element method, and the material parameters and the imaging data are treated as random variables with known prior statistics. After the conversion to state space representation, the extended Kalman filtering procedures are adopted to linearize the equations and to provide the joint estimates in an approximate optimal sense. The estimation bias and convergence issues are addressed, and we conclude experimentally that it is possible to adopt this biomechanical model based multiframe estimation approach to achieve converged estimates because of the periodic nature of the cardiac dynamics. The effort is validated using synthetic data sequence with known kinematics and material parameters. Further, under linear elastic material model, estimation results using canine magnetic resonance phase contrast image sequences are presented, which are in very good agreement with histological tissue staining results, the current gold standards.

New Techniques for the Assessment of Regional Left Ventricular Wall Motion

The assessment of regional left ventricular (LV) function has been an important yet unresolved problem since the introduction of echocardiography as a diagnostic tool. Abnormal regional LV wall motion is an early finding in multiple cardiac pathologies and its diagnosis is of critical importance. In the last few years diagnostic procedures based on combined use of existing echocardiographic technologies were geared toward improving the accuracy of detection of baseline and/or induced regional wall motion abnormalities. One of the assumptions is that the combination of reduced LV wall thickening and reduced myocardial velocities can be used to accurately diagnose regional myocardial dysfunction. In this article we will discuss several new techniques for the quantification of regional LV function using Doppler echocardiography.

Biomechanical dynamics of the heart with MRI

Magnetic resonance imaging (MRI) provides a noninvasive way to evaluate the biomechanical dynamics of the heart. MRI can provide spatially registered tomographic images of the heart in different phases of the cardiac cycle, which can be used to assess global cardiac function and regional endocardial surface motion. In addition, MRI can provide detailed information on the patterns of motion within the heart wall, permitting calculation of the evolution of regional strain and related motion variables within the wall. These show consistent patterns of spatial and temporal variation in normal subjects, which are affected by alterations of function due to disease. Although still an evolving technique, MRI shows promise as a new method for research and clinical evaluation of cardiac dynamics.

A review of cardiac image registration methods

In this paper, the current status of cardiac image registration methods is reviewed. The combination of information from multiple cardiac image modalities, such as magnetic resonance imaging, computed tomography, positron emission tomography, single-photon emission computed tomography, and ultrasound, is of increasing interest in the medical community for physiologic understanding and diagnostic purposes. Registration of cardiac images is a more complex problem than brain image registration because the heart is a nonrigid moving organ inside a moving body. Moreover, as compared to the registration of brain images, the heart exhibits much fewer accurate anatomical landmarks. In a clinical context, physicians often mentally integrate image information from different modalities. Automatic registration, based on computer programs, might, however, offer better accuracy and repeatability and save time.