A nonconservative Lagrangian framework for statistical fluid registration-SAFIRA

Learning Deformable Image Registration From Optimization: Perspective, Modules, Bilevel Training and Beyond

Conventional deformable registration methods aim at solving an optimization model carefully designed on image pairs and their computational costs are exceptionally high. In contrast, recent deep learning-based approaches can provide fast deformation estimation. These heuristic network architectures are fully data-driven and thus lack explicit geometric constraints which are indispensable to generate plausible deformations, e.g., topology-preserving. Moreover, these learning-based approaches typically pose hyper-parameter learning as a black-box problem and require considerable computational and human effort to perform many training runs. To tackle the aforementioned problems, we propose a new learning-based framework to optimize a diffeomorphic model via multi-scale propagation. Specifically, we introduce a generic optimization model to formulate diffeomorphic registration and develop a series of learnable architectures to obtain propagative updating in the coarse-to-fine feature space. Further, we propose a new bilevel self-tuned training strategy, allowing efficient search of task-specific hyper-parameters. This training strategy increases the flexibility to various types of data while reduces computational and human burdens. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data. Extensive results demonstrate the state-of-the-art performance of the proposed method with diffeomorphic guarantee and extreme efficiency. We also apply our framework to challenging multi-modal image registration, and investigate how our registration to support the down-streaming tasks for medical image analysis including multi-modal fusion and image segmentation.

Non-Rigid Multi-Modal 3D Medical Image Registration Based on Foveated Modality Independent Neighborhood Descriptor

The non-rigid multi-modal three-dimensional (3D) medical image registration is highly challenging due to the difficulty in the construction of similarity measure and the solution of non-rigid transformation parameters. A novel structural representation based registration method is proposed to address these problems. Firstly, an improved modality independent neighborhood descriptor (MIND) that is based on the foveated nonlocal self-similarity is designed for the effective structural representations of 3D medical images to transform multi-modal image registration into mono-modal one. The sum of absolute differences between structural representations is computed as the similarity measure. Subsequently, the foveated MIND based spatial constraint is introduced into the Markov random field (MRF) optimization to reduce the number of transformation parameters and restrict the calculation of the energy function in the image region involving non-rigid deformation. Finally, the accurate and efficient 3D medical image registration is realized by minimizing the similarity measure based MRF energy function. Extensive experiments on 3D positron emission tomography (PET), computed tomography (CT), T1, T2, and (proton density) PD weighted magnetic resonance (MR) images with synthetic deformation demonstrate that the proposed method has higher computational efficiency and registration accuracy in terms of target registration error (TRE) than the registration methods that are based on the hybrid L-BFGS-B and cat swarm optimization (HLCSO), the sum of squared differences on entropy images, the MIND, and the self-similarity context (SSC) descriptor, except that it provides slightly bigger TRE than the HLCSO for CT-PET image registration. Experiments on real MR and ultrasound images with unknown deformation have also be done to demonstrate the practicality and superiority of the proposed method.

A momentum-based diffeomorphic demons framework for deformable MR-CT image registration

Neuro-navigated procedures require a high degree of geometric accuracy but are subject to geometric error from complex deformation in the deep brain-e.g. regions about the ventricles due to egress of cerebrospinal fluid (CSF) upon neuroendoscopic approach or placement of a ventricular shunt. We report a multi-modality, diffeomorphic, deformable registration method using momentum-based acceleration of the Demons algorithm to solve the transformation relating preoperative MRI and intraoperative CT as a basis for high-precision guidance. The registration method (pMI-Demons) extends the mono-modality, diffeomorphic form of the Demons algorithm to multi-modality registration using pointwise mutual information (pMI) as a similarity metric. The method incorporates a preprocessing step to nonlinearly stretch CT image values and incorporates a momentum-based approach to accelerate convergence. Registration performance was evaluated in phantom and patient images: first, the sensitivity of performance to algorithm parameter selection (including update and displacement field smoothing, histogram stretch, and the momentum term) was analyzed in a phantom study over a range of simulated deformations; and second, the algorithm was applied to registration of MR and CT images for four patients undergoing minimally invasive neurosurgery. Performance was compared to two previously reported methods (free-form deformation using mutual information (MI-FFD) and symmetric normalization using mutual information (MI-SyN)) in terms of target registration error (TRE), Jacobian determinant (J), and runtime. The phantom study identified optimal or nominal settings of algorithm parameters for translation to clinical studies. In the phantom study, the pMI-Demons method achieved comparable registration accuracy to the reference methods and strongly reduced outliers in TRE (p [Formula: see text] 0.001 in Kolmogorov-Smirnov test). Similarly, in the clinical study: median TRE  =  1.54 mm (0.83-1.66 mm interquartile range, IQR) for pMI-Demons compared to 1.40 mm (1.02-1.67 mm IQR) for MI-FFD and 1.64 mm (0.90-1.92 mm IQR) for MI-SyN. The pMI-Demons and MI-SyN methods yielded diffeomorphic transformations (J  >  0) that preserved topology, whereas MI-FFD yielded unrealistic (J  <  0) deformations subject to tissue folding and tearing. Momentum-based acceleration gave a ~35% speedup of the pMI-Demons method, providing registration runtime of 10.5 min (reduced to 2.2 min on GPU), compared to 15.5 min for MI-FFD and 34.7 min for MI-SyN. The pMI-Demons method achieved registration accuracy comparable to MI-FFD and MI-SyN, maintained diffeomorphic transformation similar to MI-SyN, and accelerated runtime in a manner that facilitates translation to image-guided neurosurgery.

Architecture-Driven Level Set Optimization: From Clustering to Subpixel Image Segmentation

Thanks to their effectiveness, active contour models (ACMs) are of great interest for computer vision scientists. The level set methods (LSMs) refer to the class of geometric active contours. Comparing with the other ACMs, in addition to subpixel accuracy, it has the intrinsic ability to automatically handle topological changes. Nevertheless, the LSMs are computationally expensive. A solution for their time consumption problem can be hardware acceleration using some massively parallel devices such as graphics processing units (GPUs). But the question is: which accuracy can we reach while still maintaining an adequate algorithm to massively parallel architecture? In this paper, we attempt to push back the compromise between, speed and accuracy, efficiency and effectiveness, to a higher level, comparing with state-of-the-art methods. To this end, we designed a novel architecture-aware hybrid central processing unit (CPU)-GPU LSM for image segmentation. The initialization step, using the well-known k -means algorithm, is fast although executed on a CPU, while the evolution equation of the active contour is inherently local and therefore suitable for GPU-based acceleration. The incorporation of local statistics in the level set evolution allowed our model to detect new boundaries which are not extracted by the used clustering algorithm. Comparing with some cutting-edge LSMs, the introduced model is faster, more accurate, less subject to giving local minima, and therefore suitable for automatic systems. Furthermore, it allows two-phase clustering algorithms to benefit from the numerous LSM advantages such as the ability to achieve robust and subpixel accurate segmentation results with smooth and closed contours. Intensive experiments demonstrate, objectively and subjectively, the good performance of the introduced framework both in terms of speed and accuracy.

Simultaneous Longitudinal Registration with Group-Wise Similarity Prior

Here we present an algorithm for the simultaneous registration of N longitudinal image pairs such that information acquired by each pair is used to constrain the registration of each other pair. More specifically, in the geodesic shooting setting for Large Deformation Diffeomorphic Metric Mappings (LDDMM) an average of the initial momenta characterizing the N transformations is maintained throughout and updates to individual momenta are constrained to be similar to this average. In this way, the N registrations are coupled and explore the space of diffeomorphisms as a group, the variance of which is constrained to be small. Our approach is motivated by the observation that transformations learned from images in the same diagnostic category share characteristics. The group-wise consistency prior serves to strengthen the contribution of the common signal among the N image pairs to the transformation for a specific pair, relative to features particular to that pair. We tested the algorithm on 57 longitudinal image pairs of Alzheimer's Disease patients from the Alzheimer's Disease Neuroimaging Initiative and evaluated the ability of the algorithm to produce momenta that better represent the long-term biological processes occurring in the underlying anatomy. We found that for many image pairs, momenta learned with the group-wise prior better predict a third time point image unobserved in the registration.

GPU accelerated edge-region based level set evolution constrained by 2D gray-scale histogram

Due to its intrinsic nature which allows to easily handle complex shapes and topological changes, the level set method (LSM) has been widely used in image segmentation. Nevertheless, LSM is computationally expensive, which limits its applications in real-time systems. For this purpose, we propose a new level set algorithm, which uses simultaneously edge, region, and 2D histogram information in order to efficiently segment objects of interest in a given scene. The computational complexity of the proposed LSM is greatly reduced by using the highly parallelizable lattice Boltzmann method (LBM) with a body force to solve the level set equation (LSE). The body force is the link with image data and is defined from the proposed LSE. The proposed LSM is then implemented using an NVIDIA graphics processing units to fully take advantage of the LBM local nature. The new algorithm is effective, robust against noise, independent to the initial contour, fast, and highly parallelizable. The edge and region information enable to detect objects with and without edges, and the 2D histogram information enable the effectiveness of the method in a noisy environment. Experimental results on synthetic and real images demonstrate subjectively and objectively the performance of the proposed method.

Deformable medical image registration: a survey

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: 1) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; 2) longitudinal studies, where temporal structural or anatomical changes are investigated; and 3) population modeling and statistical atlases used to study normal anatomical variability. In this paper, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this paper is to provide an extensive account of registration techniques in a systematic manner.

Spatial-temporal atlas of human fetal brain development during the early second trimester

During the second trimester, the human fetal brain undergoes numerous changes that lead to substantial variation in the neonatal in terms of its morphology and tissue types. As fetal MRI is more and more widely used for studying the human brain development during this period, a spatiotemporal atlas becomes necessary for characterizing the dynamic structural changes. In this study, 34 postmortem human fetal brains with gestational ages ranging from 15 to 22 weeks were scanned using 7.0 T MR. We used automated morphometrics, tensor-based morphometry and surface modeling techniques to analyze the data. Spatiotemporal atlases of each week and the overall atlas covering the whole period with high resolution and contrast were created. These atlases were used for the analysis of age-specific shape changes during this period, including development of the cerebral wall, lateral ventricles, Sylvian fissure, and growth direction based on local surface measurements. Our findings indicate that growth of the subplate zone is especially striking and is the main cause for the lamination pattern changes. Changes in the cortex around Sylvian fissure demonstrate that cortical growth may be one of the mechanisms for gyration. Surface deformation mapping, revealed by local shape analysis, indicates that there is global anterior-posterior growth pattern, with frontal and temporal lobes developing relatively quickly during this period. Our results are valuable for understanding the normal brain development trajectories and anatomical characteristics. These week-by-week fetal brain atlases can be used as reference in in vivo studies, and may facilitate the quantification of fetal brain development across space and time.

Genetic and environmental influences on neuroimaging phenotypes: a meta-analytical perspective on twin imaging studies

Because brain structure and function are affected in neurological and psychiatric disorders, it is important to disentangle the sources of variation in these phenotypes. Over the past 15 years, twin studies have found evidence for both genetic and environmental influences on neuroimaging phenotypes, but considerable variation across studies makes it difficult to draw clear conclusions about the relative magnitude of these influences. Here we performed the first meta-analysis of structural MRI data from 48 studies on >1,250 twin pairs, and diffusion tensor imaging data from 10 studies on 444 twin pairs. The proportion of total variance accounted for by genes (A), shared environment (C), and unshared environment (E), was calculated by averaging A, C, and E estimates across studies from independent twin cohorts and weighting by sample size. The results indicated that additive genetic estimates were significantly different from zero for all meta-analyzed phenotypes, with the exception of fractional anisotropy (FA) of the callosal splenium, and cortical thickness (CT) of the uncus, left parahippocampal gyrus, and insula. For many phenotypes there was also a significant influence of C. We now have good estimates of heritability for many regional and lobar CT measures, in addition to the global volumes. Confidence intervals are wide and number of individuals small for many of the other phenotypes. In conclusion, while our meta-analysis shows that imaging measures are strongly influenced by genes, and that novel phenotypes such as CT measures, FA measures, and brain activation measures look especially promising, replication across independent samples and demographic groups is necessary.

COMPARISON OF VOLUMETRIC REGISTRATION ALGORITHMS FOR TENSOR-BASED MORPHOMETRY

Nonlinear registration of brain MRI scans is often used to quantify morphological differences associated with disease or genetic factors. Recently, surface-guided fully 3D volumetric registrations have been developed that combine intensity-guided volume registrations with cortical surface constraints. In this paper, we compare one such algorithm to two popular high-dimensional volumetric registration methods: large-deformation viscous fluid registration, formulated in a Riemannian framework, and the diffeomorphic "Demons" algorithm. We performed an objective morphometric comparison, by using a large MRI dataset from 340 young adult twin subjects to examine 3D patterns of correlations in anatomical volumes. Surface-constrained volume registration gave greater effect sizes for detecting morphometric associations near the cortex, while the other two approaches gave greater effects sizes subcortically. These findings suggest novel ways to combine the advantages of multiple methods in the future.