Registration of 4D cardiac CT sequences under trajectory constraints with multichannel diffeomorphic demons

Intra-Cardiac Flow from Geometry Prescribed Computational Fluid Dynamics: Comparison with Ultrasound Vector Flow Imaging

PURPOSE

This paper investigates the accuracy of blood flow velocities simulated from a geometry prescribed computational fluid dynamics (CFD) pipeline by applying it to a dynamic heart phantom. The CFD flow patterns are compared to a direct flow measurement by ultrasound vector flow imaging (VFI). The hypothesis is that the simulated velocity magnitudes are within one standard deviation of the measured velocities.

METHODS

The CFD pipeline uses computed tomography angiography (CTA) images with 20 volumes per cardiac cycle as geometry input. Fluid domain movement is prescribed from volumetric image registration using the CTA image data. Inlet and outlet conditions are defined by the experimental setup. VFI is systematically measured in parallel planes, and compared to the corresponding planes in the simulated time dependent three dimensional fluid velocity field.

RESULTS

The measured VFI and simulated CFD have similar flow patterns when compared qualitatively. A quantitative comparison of the velocity magnitude is also performed at specific regions of interest. These are evaluated at 11 non-overlapping time bins and compared by linear regression giving R2 = 0.809, SD = 0.060 m/s, intercept = - 0.039 m/s, and slope = 1.09. Excluding an outlier at the inlet, the correspondence between CFD and VFI improves to: R2 = 0.823, SD = 0.048 m/s, intercept = -0.030 m/s, and slope = 1.01.

CONCLUSION

The direct comparison of flow patterns shows that the proposed CFD pipeline provide realistic flow patterns in a well-controlled experimental setup. The demanded accuracy is obtained close to the inlet and outlet, but not in locations far from these.

Projection-based dynamic tomography

Objective. This paper proposes a 4D dynamic tomography framework that allows a moving sample to be imaged in a tomograph as well as the associated space-time kinematics to be measured with nothing more than a single conventional x-ray scan.Approach. The method exploits the consistency of the projection/reconstruction operations through a multi-scale procedure. The procedure is composed of two main parts solved alternatively: a motion-compensated reconstruction algorithm and a projection-based measurement procedure that estimates the displacement field directly on each projection.Main results. The method is applied to two studies: a numerical simulation of breathing from chest computed tomography (4D-CT) and a clinical cone-beam CT of a breathing patient acquired for image guidance of radiotherapy. The reconstructed volume, initially blurred by the motion, is cleaned from motion artifacts.Significance. Applying the proposed approach results in an improved reconstruction quality showing sharper edges and finer details.

Learning a Generative Motion Model From Image Sequences Based on a Latent Motion Matrix

We propose to learn a probabilistic motion model from a sequence of images for spatio-temporal registration. Our model encodes motion in a low-dimensional probabilistic space - the motion matrix - which enables various motion analysis tasks such as simulation and interpolation of realistic motion patterns allowing for faster data acquisition and data augmentation. More precisely, the motion matrix allows to transport the recovered motion from one subject to another simulating for example a pathological motion in a healthy subject without the need for inter-subject registration. The method is based on a conditional latent variable model that is trained using amortized variational inference. This unsupervised generative model follows a novel multivariate Gaussian process prior and is applied within a temporal convolutional network which leads to a diffeomorphic motion model. Temporal consistency and generalizability is further improved by applying a temporal dropout training scheme. Applied to cardiac cine-MRI sequences, we show improved registration accuracy and spatio-temporally smoother deformations compared to three state-of-the-art registration algorithms. Besides, we demonstrate the model's applicability for motion analysis, simulation and super-resolution by an improved motion reconstruction from sequences with missing frames compared to linear and cubic interpolation.

A deep learning based framework for the registration of three dimensional multi-modal medical images of the head

Image registration is a fundamental task in image analysis in which the transform that moves the coordinate system of one image to another is calculated. Registration of multi-modal medical images has important implications for clinical diagnosis, treatment planning, and image-guided surgery as it provides the means of bringing together complimentary information obtained from different image modalities. However, since different image modalities have different properties due to their different acquisition methods, it remains a challenging task to find a fast and accurate match between multi-modal images. Furthermore, due to reasons such as ethical issues and need for human expert intervention, it is difficult to collect a large database of labelled multi-modal medical images. In addition, manual input is required to determine the fixed and moving images as input to registration algorithms. In this paper, we address these issues and introduce a registration framework that (1) creates synthetic data to augment existing datasets, (2) generates ground truth data to be used in the training and testing of algorithms, (3) registers (using a combination of deep learning and conventional machine learning methods) multi-modal images in an accurate and fast manner, and (4) automatically classifies the image modality so that the process of registration can be fully automated. We validate the performance of the proposed framework on CT and MRI images of the head obtained from a publicly available registration database.

Balanced multi-image demons for non-rigid registration of magnetic resonance images

A new approach is introduced for non-rigid registration of a pair of magnetic resonance images (MRI). It is a generalization of the demons algorithm with low computational cost, based on local information augmentation (by integrating multiple images) and balanced implementation. Specifically, a single deformation that best registers more pairs of images is estimated. All these images are extracted by applying different operators to the two original ones, processing local neighbors of each pixel. The following five images were found to be appropriate for MRI registration: the raw image and those obtained by contrast-limited adaptive histogram equalization, local median, local entropy and phase symmetry. Thus, each local point in the images is supplemented by augmented information coming by processing its neighbor. Moreover, image pairs are processed in alternation for each iteration of the algorithm (in a balanced way), computing both a forward and a backward registration. The new method (called balanced multi-image demons) is tested on sagittal MRIs from 10 patients, both in simulated and experimental conditions, improving the performances over the classical demons approach with minimal increase of the computational cost (processing time around twice that of standard demons). Specifically, a simulated deformation was applied to the MRIs (either original or corrupted by additive Gaussian or speckle noises). In all tested cases, the new algorithm improved the estimation of the simulated deformation (squared estimation error decreased by about 65% in the average). Moreover, statistically significant improvements were obtained in experimental tests, in which different brain regions (i.e., brain, posterior fossa and cerebellum) were identified by the atlas approach and compared to those manually delineated (in the average, Dice coefficient increased of about 6%). The conclusion is that a balanced method applied to multiple information extracted from neighboring pixels is a low cost approach to improve registration of MRIs.

One-Shot Learning for Deformable Medical Image Registration and Periodic Motion Tracking

Deformable image registration is a very important field of research in medical imaging. Recently multiple deep learning approaches were published in this area showing promising results. However, drawbacks of deep learning methods are the need for a large amount of training datasets and their inability to register unseen images different from the training datasets. One shot learning comes without the need of large training datasets and has already been proven to be applicable to 3D data. In this work we present a one shot registration approach for periodic motion tracking in 3D and 4D datasets. When applied to a 3D dataset the algorithm calculates the inverse of the registration vector field simultaneously. For registration we employed a U-Net combined with a coarse to fine approach and a differential spatial transformer module. The algorithm was thoroughly tested with multiple 4D and 3D datasets publicly available. The results show that the presented approach is able to track periodic motion and to yield a competitive registration accuracy. Possible applications are the use as a stand-alone algorithm for 3D and 4D motion tracking or in the beginning of studies until enough datasets for a separate training phase are available.

A resting state fMRI analysis pipeline for pooling inference across diverse cohorts: an ENIGMA rs-fMRI protocol

Large-scale consortium efforts such as Enhancing NeuroImaging Genetics through Meta-Analysis (ENIGMA) and other collaborative efforts show that combining statistical data from multiple independent studies can boost statistical power and achieve more accurate estimates of effect sizes, contributing to more reliable and reproducible research. A meta- analysis would pool effects from studies conducted in a similar manner, yet to date, no such harmonized protocol exists for resting state fMRI (rsfMRI) data. Here, we propose an initial pipeline for multi-site rsfMRI analysis to allow research groups around the world to analyze scans in a harmonized way, and to perform coordinated statistical tests. The challenge lies in the fact that resting state fMRI measurements collected by researchers over the last decade vary widely, with variable quality and differing spatial or temporal signal-to-noise ratio (tSNR). An effective harmonization must provide optimal measures for all quality data. Here we used rsfMRI data from twenty-two independent studies with approximately fifty corresponding T1-weighted and rsfMRI datasets each, to (A) review and aggregate the state of existing rsfMRI data, (B) demonstrate utility of principal component analysis (PCA)-based denoising and (C) develop a deformable ENIGMA EPI template based on the representative anatomy that incorporates spatial distortion patterns from various protocols and populations.

Machine Learning Approaches for Myocardial Motion and Deformation Analysis

Information about myocardial motion and deformation is key to differentiate normal and abnormal conditions. With the advent of approaches relying on data rather than pre-conceived models, machine learning could either improve the robustness of motion quantification or reveal patterns of motion and deformation (rather than single parameters) that differentiate pathologies. We review machine learning strategies for extracting motion-related descriptors and analyzing such features among populations, keeping in mind constraints specific to the cardiac application.

Atlas-Based Computational Analysis of Heart Shape and Function in Congenital Heart Disease

Approximately 1% of all babies are born with some form of congenital heart defect. Many serious forms of CHD can now be surgically corrected after birth, which has led to improved survival into adulthood. However, many patients require serial monitoring to evaluate progression of heart failure and determine timing of interventions. Accurate multidimensional quantification of regional heart shape and function is required for characterizing these patients. A computational atlas of single ventricle and biventricular heart shape and function enables quantification of remodeling in terms of z scores in relation to specific reference populations. Progression of disease can then be monitored effectively by longitudinal evaluation of z scores. A biomechanical analysis of cardiac function in relation to population variation enables investigation of the underlying mechanisms for developing pathology. Here, we summarize recent progress in this field, with examples in single ventricle and biventricular congenital pathologies.

Phase Vector Incompressible Registration Algorithm for Motion Estimation From Tagged Magnetic Resonance Images

Tagged magnetic resonance imaging has been used for decades to observe and quantify motion and strain of deforming tissue. It is challenging to obtain 3-D motion estimates due to a tradeoff between image slice density and acquisition time. Typically, interpolation methods are used either to combine 2-D motion extracted from sparse slice acquisitions into 3-D motion or to construct a dense volume from sparse acquisitions before image registration methods are applied. This paper proposes a new phase-based 3-D motion estimation technique that first computes harmonic phase volumes from interpolated tagged slices and then matches them using an image registration framework. The approach uses several concepts from diffeomorphic image registration with a key novelty that defines a symmetric similarity metric on harmonic phase volumes from multiple orientations. The material property of harmonic phase solves the aperture problem of optical flow and intensity-based methods and is robust to tag fading. A harmonic magnitude volume is used in enforcing incompressibility in the tissue regions. The estimated motion fields are dense, incompressible, diffeomorphic, and inverse-consistent at a 3-D voxel level. The method was evaluated using simulated phantoms, human brain data in mild head accelerations, human tongue data during speech, and an open cardiac data set. The method shows comparable accuracy to three existing methods while demonstrating low computation time and robustness to tag fading and noise.

Longitudinal Analysis of Pre-Term Neonatal Cerebral Ventricles From 3D Ultrasound Images Using Spatial-Temporal Deformable Registration

Preterm neonates with a very low birth weight of less than 1,500 grams are at increased risk for developing intraventricular hemorrhage (IVH), which is a major cause of brain injury in preterm neonates. Quantitative measurements of ventricular dilatation or shrinkage play an important role in monitoring patients and evaluating treatment options. 3D ultrasound (US) has been developed to monitor ventricle volume as a biomarker for ventricular changes. However, ventricle volume as a global indicator does not allow for precise analysis of local ventricular changes, which could be linked to specific neurological problems often seen in the patient population later in life. In this work, a 3D+t spatial-temporal deformable registration approachis proposed, which is applied to the analysis of the detailed local changes of preterm IVH neonatal ventricles from 3D US images. In particular, a novel sequential convex/dual optimization algorithm is introduced to extract the optimal 3D+t spatial-temporal deformable field, which simultaneously optimizes the sequence of 3D deformation fieldswhile enjoying both efficiencyand simplicity in numerics. The developed registration technique was evaluated by comparing two manually extracted ventricle surfaces from the baseline and the registered follow-up images using the metrics of Dice similarity coefficient (DSC), mean absolute surface distance (MAD), and maximum absolute surface distance (MAXD). The performed experiments using 14 patients with 5 time-point images per patient show that the proposed 3D+t registration approach accurately recovered the longitudinal deformation of ventricle surfaces from 3D US images. The proposed approach may be potentially used to analyse the change pattern of cerebral ventricles of IVH patients, their response to different treatment options, and to elucidate the deficiencies that a patient could have later in life. To the best of our knowledge, this paper reports the first study on the longitudinalanalysis of neonatal ventricular system from 3D US images.

Nonrigid registration with corresponding points constraint for automatic segmentation of cardiac DSCT images

Background

Dual-source computed tomography (DSCT) is a very effective way for diagnosis and treatment of heart disease. The quantitative information of spatiotemporal DSCT images can be important for the evaluation of cardiac function. To avoid the shortcoming of manual delineation, it is imperative to develop an automatic segmentation technique for 4D cardiac images.

Methods

In this paper, we implement the heart segmentation-propagation framework based on nonrigid registration. The corresponding points of anatomical substructures are extracted by using the extension of n-dimensional scale invariant feature transform method. They are considered as a constraint term of nonrigid registration using the free-form deformation, in order to restrain the large variations and boundary ambiguity between subjects.

Results

We validate our method on 15 patients at ten time phases. Atlases are constructed by the training dataset from ten patients. On the remaining data the median overlap is shown to improve significantly compared to original mutual information, in particular from 0.4703 to 0.5015 (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ p = 5.0 \times 10^{ - 4} $$\end{document}p=5.0×10-4) for left ventricle myocardium and from 0.6307 to 0.6519 (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ p = 6.0 \times 10^{ - 4} $$\end{document}p=6.0×10-4) for right atrium.

Conclusions

The proposed method outperforms standard mutual information of intensity only. The segmentation errors had been significantly reduced at the left ventricle myocardium and the right atrium. The mean surface distance of using our framework is around 1.73 mm for the whole heart.

Cross contrast multi-channel image registration using image synthesis for MR brain images

Multi-modal deformable registration is important for many medical image analysis tasks such as atlas alignment, image fusion, and distortion correction. Whereas a conventional method would register images with different modalities using modality independent features or information theoretic metrics such as mutual information, this paper presents a new framework that addresses the problem using a two-channel registration algorithm capable of using mono-modal similarity measures such as sum of squared differences or cross-correlation. To make it possible to use these same-modality measures, image synthesis is used to create proxy images for the opposite modality as well as intensity-normalized images from each of the two available images. The new deformable registration framework was evaluated by performing intra-subject deformation recovery, intra-subject boundary alignment, and inter-subject label transfer experiments using multi-contrast magnetic resonance brain imaging data. Three different multi-channel registration algorithms were evaluated, revealing that the framework is robust to the multi-channel deformable registration algorithm that is used. With a single exception, all results demonstrated improvements when compared against single channel registrations using the same algorithm with mutual information.

Comprehensive Modeling and Visualization of Cardiac Anatomy and Physiology from CT Imaging and Computer Simulations

In clinical cardiology, both anatomy and physiology are needed to diagnose cardiac pathologies. CT imaging and computer simulations provide valuable and complementary data for this purpose. However, it remains challenging to gain useful information from the large amount of high-dimensional diverse data. The current tools are not adequately integrated to visualize anatomic and physiologic data from a complete yet focused perspective. We introduce a new computer-aided diagnosis framework, which allows for comprehensive modeling and visualization of cardiac anatomy and physiology from CT imaging data and computer simulations, with a primary focus on ischemic heart disease. The following visual information is presented: (1) Anatomy from CT imaging: geometric modeling and visualization of cardiac anatomy, including four heart chambers, left and right ventricular outflow tracts, and coronary arteries; (2) Function from CT imaging: motion modeling, strain calculation, and visualization of four heart chambers; (3) Physiology from CT imaging: quantification and visualization of myocardial perfusion and contextual integration with coronary artery anatomy; (4) Physiology from computer simulation: computation and visualization of hemodynamics (e.g., coronary blood velocity, pressure, shear stress, and fluid forces on the vessel wall). Substantially, feedback from cardiologists have confirmed the practical utility of integrating these features for the purpose of computer-aided diagnosis of ischemic heart disease.

A Parallel Nonrigid Registration Algorithm Based on B-Spline for Medical Images

The nonrigid registration algorithm based on B-spline Free-Form Deformation (FFD) plays a key role and is widely applied in medical image processing due to the good flexibility and robustness. However, it requires a tremendous amount of computing time to obtain more accurate registration results especially for a large amount of medical image data. To address the issue, a parallel nonrigid registration algorithm based on B-spline is proposed in this paper. First, the Logarithm Squared Difference (LSD) is considered as the similarity metric in the B-spline registration algorithm to improve registration precision. After that, we create a parallel computing strategy and lookup tables (LUTs) to reduce the complexity of the B-spline registration algorithm. As a result, the computing time of three time-consuming steps including B-splines interpolation, LSD computation, and the analytic gradient computation of LSD, is efficiently reduced, for the B-spline registration algorithm employs the Nonlinear Conjugate Gradient (NCG) optimization method. Experimental results of registration quality and execution efficiency on the large amount of medical images show that our algorithm achieves a better registration accuracy in terms of the differences between the best deformation fields and ground truth and a speedup of 17 times over the single-threaded CPU implementation due to the powerful parallel computing ability of Graphics Processing Unit (GPU).

Patient-specific coronary blood supply territories for quantitative perfusion analysis

Myocardial perfusion imaging, coupled with quantitative perfusion analysis, provides an important diagnostic tool for the identification of ischaemic heart disease caused by coronary stenoses. The accurate mapping between coronary anatomy and under-perfused areas of the myocardium is important for diagnosis and treatment. However, in the absence of the actual coronary anatomy during the reporting of perfusion images, areas of ischaemia are allocated to a coronary territory based on a population-derived 17-segment (American Heart Association) AHA model of coronary blood supply. This work presents a solution for the fusion of 2D Magnetic Resonance (MR) myocardial perfusion images and 3D MR angiography data with the aim to improve the detection of ischaemic heart disease. The key contribution of this work is a novel method for the mediated spatiotemporal registration of perfusion and angiography data and a novel method for the calculation of patient-specific coronary supply territories. The registration method uses 4D cardiac MR cine series spanning the complete cardiac cycle in order to overcome the under-constrained nature of non-rigid slice-to-volume perfusion-to-angiography registration. This is achieved by separating out the deformable registration problem and solving it through phase-to-phase registration of the cine series. The use of patient-specific blood supply territories in quantitative perfusion analysis (instead of the population-based model of coronary blood supply) has the potential of increasing the accuracy of perfusion analysis. Quantitative perfusion analysis diagnostic accuracy evaluation with patient-specific territories against the AHA model demonstrates the value of the mediated spatiotemporal registration in the context of ischaemic heart disease diagnosis.

Scalable High-Performance Image Registration Framework by Unsupervised Deep Feature Representations Learning

Feature selection is a critical step in deformable image registration. In particular, selecting the most discriminative features that accurately and concisely describe complex morphological patterns in image patches improves correspondence detection, which in turn improves image registration accuracy. Furthermore, since more and more imaging modalities are being invented to better identify morphological changes in medical imaging data, the development of deformable image registration method that scales well to new image modalities or new image applications with little to no human intervention would have a significant impact on the medical image analysis community. To address these concerns, a learning-based image registration framework is proposed that uses deep learning to discover compact and highly discriminative features upon observed imaging data. Specifically, the proposed feature selection method uses a convolutional stacked autoencoder to identify intrinsic deep feature representations in image patches. Since deep learning is an unsupervised learning method, no ground truth label knowledge is required. This makes the proposed feature selection method more flexible to new imaging modalities since feature representations can be directly learned from the observed imaging data in a very short amount of time. Using the LONI and ADNI imaging datasets, image registration performance was compared to two existing state-of-the-art deformable image registration methods that use handcrafted features. To demonstrate the scalability of the proposed image registration framework, image registration experiments were conducted on 7.0-T brain MR images. In all experiments, the results showed that the new image registration framework consistently demonstrated more accurate registration results when compared to state of the art.

Scalable High-Performance Image Registration Framework by Unsupervised Deep Feature Representations Learning

Feature selection is a critical step in deformable image registration. In particular, selecting the most discriminative features that accurately and concisely describe complex morphological patterns in image patches improves correspondence detection, which in turn improves image registration accuracy. Furthermore, since more and more imaging modalities are being invented to better identify morphological changes in medical imaging data, the development of deformable image registration method that scales well to new image modalities or new image applications with little to no human intervention would have a significant impact on the medical image analysis community. To address these concerns, a learning-based image registration framework is proposed that uses deep learning to discover compact and highly discriminative features upon observed imaging data. Specifically, the proposed feature selection method uses a convolutional stacked autoencoder to identify intrinsic deep feature representations in image patches. Since deep learning is an unsupervised learning method, no ground truth label knowledge is required. This makes the proposed feature selection method more flexible to new imaging modalities since feature representations can be directly learned from the observed imaging data in a very short amount of time. Using the LONI and ADNI imaging datasets, image registration performance was compared to two existing state-of-the-art deformable image registration methods that use handcrafted features. To demonstrate the scalability of the proposed image registration framework, image registration experiments were conducted on 7.0-T brain MR images. In all experiments, the results showed that the new image registration framework consistently demonstrated more accurate registration results when compared to state of the art.

Longitudinal modeling of appearance and shape and its potential for clinical use

Clinical assessment routinely uses terms such as development, growth trajectory, degeneration, disease progression, recovery or prediction. This terminology inherently carries the aspect of dynamic processes, suggesting that single measurements in time and cross-sectional comparison may not sufficiently describe spatiotemporal changes. In view of medical imaging, such tasks encourage subject-specific longitudinal imaging. Whereas follow-up, monitoring and prediction are natural tasks in clinical diagnosis of disease progression and of assessment of therapeutic intervention, translation of methodologies for calculation of temporal profiles from longitudinal data to clinical routine still requires significant research and development efforts. Rapid advances in image acquisition technology with significantly reduced acquisition times and with increase of patient comfort favor repeated imaging over the observation period. In view of serial imaging ranging over multiple years, image acquisition faces the challenging issue of scanner standardization and calibration which is crucial for successful spatiotemporal analysis. Longitudinal 3D data, represented as 4D images, capture time-varying anatomy and function. Such data benefits from dedicated analysis methods and tools that make use of the inherent correlation and causality of repeated acquisitions of the same subject. Availability of such data spawned progress in the development of advanced 4D image analysis methodologies that carry the notion of linear and nonlinear regression, now applied to complex, high-dimensional data such as images, image-derived shapes and structures, or a combination thereof. This paper provides examples of recently developed analysis methodologies for 4D image data, primarily focusing on progress in areas of core expertise of the authors. These include spatiotemporal shape modeling and growth trajectories of white matter fiber tracts demonstrated with examples from ongoing longitudinal clinical neuroimaging studies such as analysis of early brain growth in subjects at risk for mental illness and neurodegeneration in Huntington's disease (HD). We will discuss broader aspects of current limitations and need for future research in view of data consistency and analysis methodologies.

Synchronization and Registration of Cine Magnetic Resonance and Dynamic Computed Tomography Images of the Heart

The synchronization and registration of dynamic computed tomography (CT) and magnetic resonance images (MRI) of the heart is required to perform a combined analysis of their complementary information. We propose a novel method that synchronizes and registers intrapatient dynamic CT and cine-MRI short axis view (SAX). For the synchronization step, a normalized cross-correlation curve is computed from each image sequence to describe the global cardiac dynamics. The time axes of these curves are then warped using an adapted dynamic time warping (DTW) procedure. The adaptation constrains the time deformation to obtain a coherent warping function. The registration step then computes the rigid transformation that maximizes the multiimage normalized mutual information of DTW-synchronized images. The DTW synchronization and the multiimage registration were evaluated using dynamic CT and cine-SAX acquisitions from nine patients undergoing cardiac resynchronization therapy. The distance between the end-systolic phases after DTW was used to evaluate the synchronization. Mean errors, expressed as a percentage of the RR-intervals, were 3.9% and 3.7% after adapted DTW synchronization against 10.8% and 11.3% after linear synchronization, for dynamic CT and cine-SAX, respectively. This suggests that the adapted DTW synchronization leads to a coherent warping of cardiac dynamics. The multiimage registration was evaluated using fiducial points. Compared to a monoimage and a two-image registration, the multiimage registration of DTW-synchronized images obtained the lowest mean fiducial error showing that the use of dynamic voxel intensity information improves the registration.

Deep Adaptive Log-Demons: Diffeomorphic Image Registration with Very Large Deformations

This paper proposes a new framework for capturing large and complex deformation in image registration. Traditionally, this challenging problem relies firstly on a preregistration, usually an affine matrix containing rotation, scale, and translation and afterwards on a nonrigid transformation. According to preregistration, the directly calculated affine matrix, which is obtained by limited pixel information, may misregistrate when large biases exist, thus misleading following registration subversively. To address this problem, for two-dimensional (2D) images, the two-layer deep adaptive registration framework proposed in this paper firstly accurately classifies the rotation parameter through multilayer convolutional neural networks (CNNs) and then identifies scale and translation parameters separately. For three-dimensional (3D) images, affine matrix is located through feature correspondences by a triplanar 2D CNNs. Then deformation removal is done iteratively through preregistration and demons registration. By comparison with the state-of-the-art registration framework, our method gains more accurate registration results on both synthetic and real datasets. Besides, principal component analysis (PCA) is combined with correlation like Pearson and Spearman to form new similarity standards in 2D and 3D registration. Experiment results also show faster convergence speed.

An implicit sliding-motion preserving regularisation via bilateral filtering for deformable image registration

Several biomedical applications require accurate image registration that can cope effectively with complex organ deformations. This paper addresses this problem by introducing a generic deformable registration algorithm with a new regularization scheme, which is performed through bilateral filtering of the deformation field. The proposed approach is primarily designed to handle smooth deformations both between and within body structures, and also more challenging deformation discontinuities exhibited by sliding organs. The conventional Gaussian smoothing of deformation fields is replaced by a bilateral filtering procedure, which compromises between the spatial smoothness and local intensity similarity kernels, and is further supported by a deformation field similarity kernel. Moreover, the presented framework does not require any explicit prior knowledge about the organ motion properties (e.g. segmentation) and therefore forms a fully automated registration technique. Validation was performed using synthetic phantom data and publicly available clinical 4D CT lung data sets. In both cases, the quantitative analysis shows improved accuracy when compared to conventional Gaussian smoothing. In addition, we provide experimental evidence that masking the lungs in order to avoid the problem of sliding motion during registration performs similarly in terms of the target registration error when compared to the proposed approach, however it requires accurate lung segmentation. Finally, quantification of the level and location of detected sliding motion yields visually plausible results by demonstrating noticeable sliding at the pleural cavity boundaries.

Multispectral image registration based on local canonical correlation analysis

Medical scans are today routinely acquired using multiple sequences or contrast settings, resulting in multispectral data. For the automatic analysis of this data, the evaluation of multispectral similarity is essential. So far, few concepts have been proposed to deal in a principled way with images containing multiple channels. Here, we present a new approach based on a well known statistical technique: canonical correlation analysis (CCA). CCA finds a mapping of two multidimensional variables into two new bases, which best represent the true underlying relations of the signals. In contrast to previously used metrics, it is therefore able to find new correlations based on linear combinations of multiple channels. We extend this concept to efficiently model local canonical correlation (LCCA) between image patches. This novel, more general similarity metric can be applied to images with an arbitrary number of channels. The most important property of LCCA is its invariance to affine transformations of variables. When used on local histograms, LCCA can also deal with multimodal similarity. We demonstrate the performance of our concept on challenging clinical multispectral datasets.

Non-rigid registration of serial dedicated breast CT, longitudinal dedicated breast CT and PET/CT images using the diffeomorphic demons method

RATIONALE AND OBJECTIVES

Dedicated breast CT and PET/CT scanners provide detailed 3D anatomical and functional imaging data sets and are currently being investigated for applications in breast cancer management such as diagnosis, monitoring response to therapy and radiation therapy planning. Our objective was to evaluate the performance of the diffeomorphic demons (DD) non-rigid image registration method to spatially align 3D serial (pre- and post-contrast) dedicated breast computed tomography (CT), and longitudinally-acquired dedicated 3D breast CT and positron emission tomography (PET)/CT images.

METHODS

The algorithmic parameters of the DD method were optimized for the alignment of dedicated breast CT images using training data and fixed. The performance of the method for image alignment was quantitatively evaluated using three separate data sets; (1) serial breast CT pre- and post-contrast images of 20 women, (2) breast CT images of 20 women acquired before and after repositioning the subject on the scanner, and (3) dedicated breast PET/CT images of 7 women undergoing neo-adjuvant chemotherapy acquired pre-treatment and after 1 cycle of therapy.

RESULTS

The DD registration method outperformed no registration (p < 0.001) and conventional affine registration (p ≤ 0.002) for serial and longitudinal breast CT and PET/CT image alignment. In spite of the large size of the imaging data, the computational cost of the DD method was found to be reasonable (3-5 min).

CONCLUSIONS

Co-registration of dedicated breast CT and PET/CT images can be performed rapidly and reliably using the DD method. This is the first study evaluating the DD registration method for the alignment of dedicated breast CT and PET/CT images.

Temporally diffeomorphic cardiac motion estimation from three-dimensional echocardiography by minimization of intensity consistency error

PURPOSE

Quantitative analysis of cardiac motion is important for evaluation of heart function. Three dimensional (3D) echocardiography is among the most frequently used imaging modalities for motion estimation because it is convenient, real-time, low-cost, and nonionizing. However, motion estimation from 3D echocardiographic sequences is still a challenging problem due to low image quality and image corruption by noise and artifacts.

METHODS

The authors have developed a temporally diffeomorphic motion estimation approach in which the velocity field instead of the displacement field was optimized. The optimal velocity field optimizes a novel similarity function, which we call the intensity consistency error, defined as multiple consecutive frames evolving to each time point. The optimization problem is solved by using the steepest descent method.

RESULTS

Experiments with simulated datasets, images of anex vivo rabbit phantom, images of in vivo open-chest pig hearts, and healthy human images were used to validate the authors' method. Simulated and real cardiac sequences tests showed that results in the authors' method are more accurate than other competing temporal diffeomorphic methods. Tests with sonomicrometry showed that the tracked crystal positions have good agreement with ground truth and the authors' method has higher accuracy than the temporal diffeomorphic free-form deformation (TDFFD) method. Validation with an open-access human cardiac dataset showed that the authors' method has smaller feature tracking errors than both TDFFD and frame-to-frame methods.

CONCLUSIONS

The authors proposed a diffeomorphic motion estimation method with temporal smoothness by constraining the velocity field to have maximum local intensity consistency within multiple consecutive frames. The estimated motion using the authors' method has good temporal consistency and is more accurate than other temporally diffeomorphic motion estimation methods.

The Insight ToolKit image registration framework

Publicly available scientific resources help establish evaluation standards, provide a platform for teaching and improve reproducibility. Version 4 of the Insight ToolKit (ITK⁴) seeks to establish new standards in publicly available image registration methodology. ITK⁴ makes several advances in comparison to previous versions of ITK. ITK⁴ supports both multivariate images and objective functions; it also unifies high-dimensional (deformation field) and low-dimensional (affine) transformations with metrics that are reusable across transform types and with composite transforms that allow arbitrary series of geometric mappings to be chained together seamlessly. Metrics and optimizers take advantage of multi-core resources, when available. Furthermore, ITK⁴ reduces the parameter optimization burden via principled heuristics that automatically set scaling across disparate parameter types (rotations vs. translations). A related approach also constrains steps sizes for gradient-based optimizers. The result is that tuning for different metrics and/or image pairs is rarely necessary allowing the researcher to more easily focus on design/comparison of registration strategies. In total, the ITK⁴ contribution is intended as a structure to support reproducible research practices, will provide a more extensive foundation against which to evaluate new work in image registration and also enable application level programmers a broad suite of tools on which to build. Finally, we contextualize this work with a reference registration evaluation study with application to pediatric brain labeling.¹

Artifact-resistant motion estimation with a patient-specific artifact model for motion-compensated cone-beam CT

PURPOSE

In image-guided radiation therapy (IGRT) valuable information for patient positioning, dose verification, and adaptive treatment planning is provided by an additional kV imaging unit. However, due to the limited gantry rotation speed during treatment the typical acquisition time is quite long. Tomographic images of the thorax suffer from motion blurring or, if a gated 4D reconstruction is performed, from significant streak artifacts. Our purpose is to provide a method that reliably estimates respiratory motion in presence of severe artifacts. The estimated motion vector fields are then used for motion-compensated image reconstruction to provide high quality respiratory-correlated 4D volumes for on-board cone-beam CT (CBCT) scans.

METHODS

The proposed motion estimation method consists of a model that explicitly addresses image artifacts because in presence of severe artifacts state-of-the-art registration methods tend to register artifacts rather than anatomy. Our artifact model, e.g., generates streak artifacts very similar to those included in the gated 4D CBCT images, but it does not include respiratory motion. In combination with a registration strategy, the model gives an error estimate that is used to compensate the corresponding errors of the motion vector fields that are estimated from the gated 4D CBCT images. The algorithm is tested in combination with a cyclic registration approach using temporal constraints and with a standard 3D-3D registration approach. A qualitative and quantitative evaluation of the motion-compensated results was performed using simulated rawdata created on basis of clinical CT data. Further evaluation includes patient data which were scanned with an on-board CBCT system.

RESULTS

The model-based motion estimation method is nearly insensitive to image artifacts of gated 4D reconstructions as they are caused by angular undersampling. The motion is accurately estimated and our motion-compensated image reconstruction algorithm can correct for it. Motion artifacts of 3D standard reconstruction are significantly reduced, while almost no new artifacts are introduced.

CONCLUSIONS

Using the artifact model allows to accurately estimate and compensate for patient motion, even if the initial reconstructions are of very low image quality. Using our approach together with a cyclic registration algorithm yields a combination which shows almost no sensitivity to sparse-view artifacts and thus ensures both high spatial and high temporal resolution.

Unsupervised deep feature learning for deformable registration of MR brain images

Establishing accurate anatomical correspondences is critical for medical image registration. Although many hand-engineered features have been proposed for correspondence detection in various registration applications, no features are general enough to work well for all image data. Although many learning-based methods have been developed to help selection of best features for guiding correspondence detection across subjects with large anatomical variations, they are often limited by requiring the known correspondences (often presumably estimated by certain registration methods) as the ground truth for training. To address this limitation, we propose using an unsupervised deep learning approach to directly learn the basis filters that can effectively represent all observed image patches. Then, the coefficients by these learnt basis filters in representing the particular image patch can be regarded as the morphological signature for correspondence detection during image registration. Specifically, a stacked two-layer convolutional network is constructed to seek for the hierarchical representations for each image patch, where the high-level features are inferred from the responses of the low-level network. By replacing the hand-engineered features with our learnt data-adaptive features for image registration, we achieve promising registration results, which demonstrates that a general approach can be built to improve image registration by using data-adaptive features through unsupervised deep learning.

Fast Geodesic Active Fields for Image Registration Based on Splitting and Augmented Lagrangian Approaches

In this paper, we present an efficient numerical scheme for the recently introduced geodesic active fields (GAF) framework for geometric image registration. This framework considers the registration task as a weighted minimal surface problem. Hence, the data-term and the regularization-term are combined through multiplication in a single, parametrization invariant and geometric cost functional. The multiplicative coupling provides an intrinsic, spatially varying and data-dependent tuning of the regularization strength, and the parametrization invariance allows working with images of nonflat geometry, generally defined on any smoothly parametrizable manifold. The resulting energy-minimizing flow, however, has poor numerical properties. Here, we provide an efficient numerical scheme that uses a splitting approach; data and regularity terms are optimized over two distinct deformation fields that are constrained to be equal via an augmented Lagrangian approach. Our approach is more flexible than standard Gaussian regularization, since one can interpolate freely between isotropic Gaussian and anisotropic TV-like smoothing. In this paper, we compare the geodesic active fields method with the popular Demons method and three more recent state-of-the-art algorithms: NL-optical flow, MRF image registration, and landmark-enhanced large displacement optical flow. Thus, we can show the advantages of the proposed FastGAF method. It compares favorably against Demons, both in terms of registration speed and quality. Over the range of example applications, it also consistently produces results not far from more dedicated state-of-the-art methods, illustrating the flexibility of the proposed framework.

Medical image registration by combining global and local information: a chain-type diffeomorphic demons algorithm

The demons algorithm is a popular algorithm for non-rigid image registration because of its computational efficiency and simple implementation. The deformation forces of the classic demons algorithm were derived from image gradients by considering the deformation to decrease the intensity dissimilarity between images. However, the methods using the difference of image intensity for medical image registration are easily affected by image artifacts, such as image noise, non-uniform imaging and partial volume effects. The gradient magnitude image is constructed from the local information of an image, so the difference in a gradient magnitude image can be regarded as more reliable and robust for these artifacts. Then, registering medical images by considering the differences in both image intensity and gradient magnitude is a straightforward selection. In this paper, based on a diffeomorphic demons algorithm, we propose a chain-type diffeomorphic demons algorithm by combining the differences in both image intensity and gradient magnitude for medical image registration. Previous work had shown that the classic demons algorithm can be considered as an approximation of a second order gradient descent on the sum of the squared intensity differences. By optimizing the new dissimilarity criteria, we also present a set of new demons forces which were derived from the gradients of the image and gradient magnitude image. We show that, in controlled experiments, this advantage is confirmed, and yields a fast convergence.

Accurate semiautomatic assessment of ligament length variations from MRI data

PURPOSE

A semiautomatic method for the assessment of ligament length variations during different joint positions based on MRI data is proposed.

METHODS

Ligament lengths are represented as distances between points marking characteristic locations in the ligament insertion regions on the bones. These points are defined manually for one single reference joint position and for all other joint positions they are automatically mapped with high accuracy to the correct locations using image registration methods. The methodology is validated using data from 16 volunteers depicting the coracoclavicular ligaments in the left shoulder during different arm abductions.

RESULTS

The method yielded a superior reproducibility of the point locations over different joint positions compared to manual point marking. Significant ligament length variations were found for different abductions which was not possible with manual measurements. Acquisition related geometric distortions and inaccuracies during the registration and segmentation process were small.

CONCLUSIONS

The proposed method provides superior accuracy for the in vivo analysis of ligament dynamics compared to manual measurements. This permits a better understanding of the ligament behavior during joint motion and offers new possibilities for presurgical planning which to date has not been possible with manual data analysis.

A statistical framework for inter-group image registration

Groupwise image registration plays an important role in medical image analysis. The principle of groupwise image registration is to align a given set of images to a hidden template space in an iteratively manner without explicitly selecting any individual image as the template. Although many approaches have been proposed to address the groupwise image registration problem for registering a single group of images, few attentions and efforts have been paid to the registration problem between two or more different groups of images. In this paper, we propose a statistical framework to address the registration problems between two different image groups. The main contributions of this paper lie in the following aspects: (1) In this paper, we demonstrate that directly registering the group mean images estimated from two different image groups is not sufficient to establish the reliable transformation from one image group to the other image group. (2) A novel statistical framework is proposed to extract anatomical features from the white matter, gray matter and cerebrospinal fluid tissue maps of all aligned images as morphological signatures for each voxel. The extracted features provide much richer anatomical information than the voxel intensity of the group mean image, and can be integrated with the multi-channel Demons registration algorithm to perform the registration process. (3) The proposed method has been extensively evaluated on two publicly available brain MRI databases: the LONI LPBA40 and the IXI databases, and it is also compared with a conventional inter-group image registration approach which directly performs deformable registration between the group mean images of two image groups. Experimental results show that the proposed method consistently achieves higher registration accuracy than the method under comparison.

Atlas construction for dynamic (4D) PET using diffeomorphic transformations

A novel dynamic (4D) PET to PET image registration procedure is proposed and applied to multiple PET scans acquired with the high resolution research tomograph (HRRT), the highest resolution human brain PET scanner available in the world. By extending the recent diffeomorphic log-demons (DLD) method and applying it to multiple dynamic [11C]raclopride scans from the HRRT, an important step towards construction of a PET atlas of unprecedented quality for [11C]raclopride imaging of the human brain has been achieved. Accounting for the temporal dimension in PET data improves registration accuracy when compared to registration of 3D to 3D time-averaged PET images. The DLD approach was chosen for its ease in providing both an intensity and shape template, through iterative sequential pair-wise registrations with fast convergence. The proposed method is applicable to any PET radiotracer, providing 4D atlases with useful applications in high accuracy PET data simulations and automated PET image analysis.

Recognition of finger flexion motion from ultrasound image: a feasibility study

Muscle contraction results in structural and morphologic changes of the related muscle. Therefore, finger flexion can be monitored from measurements of these morphologic changes. We used ultrasound imaging to record muscle activities during finger flexion and extracted features to discriminate different fingers' flexions using a support vector machine (SVM). Registration of ultrasound images before and after finger flexion was performed to generate a deformation field, from which angle features and wavelet-based features were extracted. The SVM was then used to classify the motions of different fingers. The experimental results showed that the overall mean recognition accuracy was 94.05% ± 4.10%, with the highest for the thumb (97%) and the lowest for the ring finger (92%) and the mean F value was 0.94 ± 0.02, indicating high accuracy and reliability of this method. The results suggest that the proposed method has the potential to be used as an alternative method of surface electromyography in differentiating the motions of different fingers.

Registration-based segmentation of murine 4D cardiac micro-CT data using symmetric normalization

Micro-CT can play an important role in preclinical studies of cardiovascular disease because of its high spatial and temporal resolution. Quantitative analysis of 4D cardiac images requires segmentation of the cardiac chambers at each time point, an extremely time consuming process if done manually. To improve throughput this study proposes a pipeline for registration-based segmentation and functional analysis of 4D cardiac micro-CT data in the mouse. Following optimization and validation using simulations, the pipeline was applied to in vivo cardiac micro-CT data corresponding to ten cardiac phases acquired in C57BL/6 mice (n = 5). After edge-preserving smoothing with a novel adaptation of 4D bilateral filtration, one phase within each cardiac sequence was manually segmented. Deformable registration was used to propagate these labels to all other cardiac phases for segmentation. The volumes of each cardiac chamber were calculated and used to derive stroke volume, ejection fraction, cardiac output, and cardiac index. Dice coefficients and volume accuracies were used to compare manual segmentations of two additional phases with their corresponding propagated labels. Both measures were, on average, >0.90 for the left ventricle and >0.80 for the myocardium, the right ventricle, and the right atrium, consistent with trends in inter- and intra-segmenter variability. Segmentation of the left atrium was less reliable. On average, the functional metrics of interest were underestimated by 6.76% or more due to systematic label propagation errors around atrioventricular valves; however, execution of the pipeline was 80% faster than performing analogous manual segmentation of each phase.

Tracking the motion trajectories of junction structures in 4D CT images of the lung

Respiratory motion poses a major challenge in lung radiotherapy. Based on 4D CT images, a variety of intensity-based deformable registration techniques have been proposed to study the pulmonary motion. However, the accuracy achievable with these approaches can be sub-optimal because the deformation is defined globally in space. Therefore, the accuracy of the alignment of local structures may be compromised. In this work, we propose a novel method to detect a large collection of natural junction structures in the lung and use them as the reliable markers to track the lung motion. Specifically, detection of the junction centers and sizes is achieved by analysis of local shape profiles on one segmented image. To track the temporal trajectory of a junction, the image intensities within a small region of interest surrounding the center are selected as its signature. Under the assumption of the cyclic motion, we describe the trajectory by a closed B-spline curve and search for the control points by maximizing a metric of combined correlation coefficients. Local extrema are suppressed by improving the initial conditions using random walks from pair-wise optimizations. Several descriptors are introduced to analyze the motion trajectories. Our method was applied to 13 real 4D CT images. More than 700 junctions in each case are detected with an average positive predictive value of greater than 90%. The average tracking error between automated and manual tracking is sub-voxel and smaller than the published results using the same set of data.

Regression-based cardiac motion prediction from single-phase CTA

State of the art cardiac computed tomography (CT) enables the acquisition of imaging data of the heart over the entire cardiac cycle at concurrent high spatial and temporal resolution. However, in clinical practice, acquisition is increasingly limited to 3-D images. Estimating the shape of the cardiac structures throughout the entire cardiac cycle from a 3-D image is therefore useful in applications such as the alignment of preoperative computed tomography angiography (CTA) to intra-operative X-ray images for improved guidance in coronary interventions. We hypothesize that the motion of the heart is partially explained by its shape and therefore investigate the use of three regression methods for motion estimation from single-phase shape information. Quantitative evaluation on 150 4-D CTA images showed a small, but statistically significant, increase in the accuracy of the predicted shape sequences when using any of the regression methods, compared to shape-independent motion prediction by application of the mean motion. The best results were achieved using principal component regression resulting in point-to-point errors of 2.3±0.5 mm, compared to values of 2.7±0.6 mm for shape-independent motion estimation. Finally, we showed that this significant difference withstands small variations in important parameter settings of the landmarking procedure.

Medical image registration: a review

This paper presents a review of automated image registration methodologies that have been used in the medical field. The aim of this paper is to be an introduction to the field, provide knowledge on the work that has been developed and to be a suitable reference for those who are looking for registration methods for a specific application. The registration methodologies under review are classified into intensity or feature based. The main steps of these methodologies, the common geometric transformations, the similarity measures and accuracy assessment techniques are introduced and described.

Building Spatiotemporal Anatomical Models using Joint 4-D Segmentation, Registration, and Subject-Specific Atlas Estimation

Longitudinal analysis of anatomical changes is a vital component in many personalized-medicine applications for predicting disease onset, determining growth/atrophy patterns, evaluating disease progression, and monitoring recovery. Estimating anatomical changes in longitudinal studies, especially through magnetic resonance (MR) images, is challenging because of temporal variability in shape (e.g. from growth/atrophy) and appearance (e.g. due to imaging parameters and tissue properties affecting intensity contrast, or from scanner calibration). This paper proposes a novel mathematical framework for constructing subject-specific longitudinal anatomical models. The proposed method solves a generalized problem of joint segmentation, registration, and subject-specific atlas building, which involves not just two images, but an entire longitudinal image sequence. The proposed framework describes a novel approach that integrates fundamental principles that underpin methods for image segmentation, image registration, and atlas construction. This paper presents evaluation on simulated longitudinal data and on clinical longitudinal brain MRI data. The results demonstrate that the proposed framework effectively integrates information from 4-D spatiotemporal data to generate spatiotemporal models that allow analysis of anatomical changes over time.

A spatiotemporal statistical atlas of motion for the quantification of abnormal myocardial tissue velocities

In this paper, we present a new method for the automatic comparison of myocardial motion patterns and the characterization of their degree of abnormality, based on a statistical atlas of motion built from a reference healthy population. Our main contribution is the computation of atlas-based indexes that quantify the abnormality in the motion of a given subject against a reference population, at every location in time and space. The critical computational cost inherent to the construction of an atlas is highly reduced by the definition of myocardial velocities under a small displacements hypothesis. The indexes we propose are of notable interest for the assessment of anomalies in cardiac mobility and synchronicity when applied, for instance, to candidate selection for cardiac resynchronization therapy (CRT). We built an atlas of normality using 2D ultrasound cardiac sequences from 21 healthy volunteers, to which we compared 14 CRT candidates with left ventricular dyssynchrony (LVDYS). We illustrate the potential of our approach in characterizing septal flash, a specific motion pattern related to LVDYS and recently introduced as a very good predictor of response to CRT.

Group-wise diffeomorphic diffusion tensor image registration

We propose an unbiased group-wise diffeomorphic registration technique to normalize a group of diffusion tensor (DT) images. Our method uses an implicit reference group-wise registration framework to avoid bias caused by reference selection. Log-Euclidean metrics on diffusion tensors are used for the tensor interpolation and computation of the similarity cost functions. The overall energy function is constructed by a diffeomorphic demons approach. The tensor reorientation is performed and implicitly optimized during the registration procedure. The performance of the proposed method is compared with reference-based diffusion tensor imaging (DTI) registration methods. The registered DTI images have smaller shape differences in terms of reduced variance of the fractional anisotropy maps and more consistent tensor orientations. We demonstrate that fiber tract atlas construction can benefit from the group-wise registration by producing fiber bundles with higher overlaps.