Deformable medical image registration: a survey

Physical Constraint Finite Element Model for Medical Image Registration

Due to being derived from linear assumption, most elastic body based non-rigid image registration algorithms are facing challenges for soft tissues with complex nonlinear behavior and with large deformations. To take into account the geometric nonlinearity of soft tissues, we propose a registration algorithm on the basis of Newtonian differential equation. The material behavior of soft tissues is modeled as St. Venant-Kirchhoff elasticity, and the nonlinearity of the continuum represents the quadratic term of the deformation gradient under the Green- St.Venant strain. In our algorithm, the elastic force is formulated as the derivative of the deformation energy with respect to the nodal displacement vectors of the finite element; the external force is determined by the registration similarity gradient flow which drives the floating image deforming to the equilibrium condition. We compared our approach to three other models: 1) the conventional linear elastic finite element model (FEM); 2) the dynamic elastic FEM; 3) the robust block matching (RBM) method. The registration accuracy was measured using three similarities: MSD (Mean Square Difference), NC (Normalized Correlation) and NMI (Normalized Mutual Information), and was also measured using the mean and max distance between the ground seeds and corresponding ones after registration. We validated our method on 60 image pairs including 30 medical image pairs with artificial deformation and 30 clinical image pairs for both the chest chemotherapy treatment in different periods and brain MRI normalization. Our method achieved a distance error of 0.320±0.138 mm in x direction and 0.326±0.111 mm in y direction, MSD of 41.96±13.74, NC of 0.9958±0.0019, NMI of 1.2962±0.0114 for images with large artificial deformations; and average NC of 0.9622±0.008 and NMI of 1.2764±0.0089 for the real clinical cases. Student’s t-test demonstrated that our model statistically outperformed the other methods in comparison (p-values <0.05).

State-of-the-art in retinal optical coherence tomography image analysis

Optical coherence tomography (OCT) is an emerging imaging modality that has been widely used in the field of biomedical imaging. In the recent past, it has found uses as a diagnostic tool in dermatology, cardiology, and ophthalmology. In this paper we focus on its applications in the field of ophthalmology and retinal imaging. OCT is able to non-invasively produce cross-sectional volumetric images of the tissues which can be used for analysis of tissue structure and properties. Due to the underlying physics, OCT images suffer from a granular pattern, called speckle noise, which restricts the process of interpretation. This requires specialized noise reduction techniques to eliminate the noise while preserving image details. Another major step in OCT image analysis involves the use of segmentation techniques for distinguishing between different structures, especially in retinal OCT volumes. The outcome of this step is usually thickness maps of different retinal layers which are very useful in study of normal/diseased subjects. Lastly, movements of the tissue under imaging as well as the progression of disease in the tissue affect the quality and the proper interpretation of the acquired images which require the use of different image registration techniques. This paper reviews various techniques that are currently used to process raw image data into a form that can be clearly interpreted by clinicians.

A method of surface marker location optimization for tumor motion estimation in lung stereotactic body radiation therapy

PURPOSE

Accurately localizing lung tumor localization is essential for high-precision radiation therapy techniques such as stereotactic body radiation therapy (SBRT). Since direct monitoring of tumor motion is not always achievable due to the limitation of imaging modalities for treatment guidance, placement of fiducial markers on the patient's body surface to act as a surrogate for tumor position prediction is a practical alternative for tracking lung tumor motion during SBRT treatments. In this work, the authors propose an innovative and robust model to solve the multimarker position optimization problem. The model is able to overcome the major drawbacks of the sparse optimization approach (SOA) model.

METHODS

The principle-component-analysis (PCA) method was employed as the framework to build the authors' statistical prediction model. The method can be divided into two stages. The first stage is to build the surrogate tumor matrix and calculate its eigenvalues and associated eigenvectors. The second stage is to determine the "best represented" columns of the eigenvector matrix obtained from stage one and subsequently acquire the optimal marker positions as well as numbers. Using 4-dimensional CT (4 DCT) and breath hold CT imaging data, the PCA method was compared to the SOA method with respect to calculation time, average prediction accuracy, prediction stability, noise resistance, marker position consistency, and marker distribution.

RESULTS

The PCA and SOA methods which were both tested were on all 11 patients for a total of 130 cases including 4 DCT and breath-hold CT scenarios. The maximum calculation time for the PCA method was less than 1 s with 64 752 surface points, whereas the average calculation time for the SOA method was over 12 min with 400 surface points. Overall, the tumor center position prediction errors were comparable between the two methods, and all were less than 1.5 mm. However, for the extreme scenarios (breath hold), the prediction errors for the PCA method were not only smaller, but were also more stable than for the SOA method. Results obtained by imposing a series of random noises to the surrogates indicated that the PCA method was much more noise resistant than the SOA method. The marker position consistency tests using various combinations of 4 DCT phases to construct the surrogates suggested that the marker position predictions of the PCA method were more consistent than those of the SOA method, in spite of surrogate construction. Marker distribution tests indicated that greater than 80% of the calculated marker positions fell into the high cross correlation and high motion magnitude regions for both of the algorithms.

CONCLUSIONS

The PCA model is an accurate, efficient, robust, and practical model for solving the multimarker position optimization problem to predict lung tumor motion during SBRT treatments. Due to its generality, PCA model can also be applied to other imaging guidance system whichever using surface motion as the surrogates.

An automated workflow for the anatomo-functional mapping of the barrel cortex

BACKGROUND

The rodent barrel cortex is a widely used model to study the cortical processing of tactile sensory information. It is notable by the cytoarchitecture of its layer IV, which contains distinguishable structural units called barrels that can be considered as anatomical landmarks of the functional columnar organization of the cerebral cortex. To study sensory integration in the barrel cortex it is therefore essential to map recorded functional data onto the underlying barrel topography, which can be reconstructed from the post hoc alignment of tangential brain slices stained for cytochrome oxidase.

NEW METHOD

This article presents an automated workflow to perform the registration of histological slices of the barrel cortex followed by the 2-D reconstruction of the barrel map from the registered slices. The registration of two successive slices is obtained by computing a rigid transformation to align sets of detected blood vessel cross-sections. This is achieved by using a robust variant of the classical iterative closest point method. A single fused image of the barrel field is then generated by computing a nonlinear merging of the gradients from the registered images.

COMPARISON WITH EXISTING METHODS

This novel anatomo-functional mapping tool leads to a substantial gain in time and precision compared to conventional manual methods. It provides a flexible interface for the user with only a few parameters to tune.

CONCLUSIONS

We demonstrate here the usefulness of the method for voltage sensitive dye imaging of the mouse barrel cortex. The method could also benefit other experimental approaches and model species.

A fast algorithm to estimate inverse consistent image transformation based on corresponding landmarks

Inverse consistency is an important feature for non-rigid image transformation in medical imaging analysis. In this paper, a simple and efficient inverse consistent image transformation estimation algorithm is proposed to preserve correspondence of landmarks and accelerate convergence. The proposed algorithm estimates both the forward and backward transformations simultaneously in the way that they are inverse to each other based on the correspondence of landmarks. Instead of computing the inverse functions and the inverse consistent transformations, respectively, we combine them together, which can improve computation efficiency significantly. Moreover, radial basis functions (RBFs) based transformation is adopted in our algorithm, which can handle deformation with local or global support. Our algorithm maps one landmark to its corresponding position exactly using the forward and backward transformations. Moreover, our algorithm is employed to estimate the forward and backward transformations in robust point matching, as well to demonstrate the application of our algorithm in image registration. The experiment results of uniform grids and test images indicate the improvement of the proposed algorithm in the aspect of inverse consistency of transformations and the reduction of the computation time of the forward and the backward transformations. The performance of our algorithm applying to robust point matching is evaluated using both brain slices and lung slices. Our experiments show that by combing robust point matching with our algorithm, the registration accuracy can be improved and the smoothness of transformations can be preserved.

A Feasibility Study of Tumor Motion Estimate With Regional Deformable Registration Method for 4-Dimensional Radiation Therapy of Lung Cancer

This study aims to employ 4-dimensional computed tomography to quantify intrafractional tumor motion for patients with lung cancer to improve target localization in radiation therapy. A multistage regional deformable registration was implemented to calculate the excursion of gross tumor volume (GTV) during a breathing cycle. GTV was initially delineated on 0% phase of 4-dimensional computed tomography manually, and a subregion with 20 mm margin supplemented to GTV was generated with Eclipse treatment planning system (Varian Medical Systems, Palo Alto, California). The structures, together with the 4-dimensional computed tomography set, were exported into an in-house software, with which a 3-stage B-spline deformable registration was carried out to map the subregion and warp GTV contour to other breathing phases. The center of mass of the GTV was computed using the contours, and the tumor motion was appraised as the excursion of the center of mass between 0% phase and other phases. Application of the algorithm to the 10 patients showed that clinically satisfactory outcomes were achievable with a spatial accuracy around 2 mm for GTV contour propagation between adjacent phases and 3 mm between opposite phases. The tumor excursion was determined in the vast range of 1 mm through 1.6 cm, depending on the tumor location and tumor size. Compared to the traditional whole image-based registration, the regional method was found computationally a factor of 5 more efficient. The proposed technique has demonstrated its capability in extracting thoracic tumor motion and should find its application in 4-dimensional radiation therapy in the future to maximally utilize the available spatial-temporal information.

Fast Rotation-Free Feature-Based Image Registration Using Improved N-SIFT and GMM-Based Parallel Optimization

Image registration is a key problem in a variety of applications, such as computer vision, medical image processing, pattern recognition, etc., while the application of registration is limited by time consumption and the accuracy in the case of large pose differences. Aimed at these two kinds of problems, we propose a fast rotation-free feature-based rigid registration method based on our proposed accelerated-NSIFT and GMM registration-based parallel optimization (PO-GMMREG). Our method is accelerated by using the GPU/CUDA programming and preserving only the location information without constructing the descriptor of each interest point, while its robustness to missing correspondences and outliers is improved by converting the interest point matching to Gaussian mixture model alignment. The accuracy in the case of large pose differences is settled by our proposed PO-GMMREG algorithm by constructing a set of initial transformations. Experimental results demonstrate that our proposed algorithm can fast rigidly register 3-D medical images and is reliable for aligning 3-D scans even when they exhibit a poor initialization.

Multi-atlas segmentation of biomedical images: A survey

Multi-atlas segmentation (MAS), first introduced and popularized by the pioneering work of Rohlfing, et al. (2004), Klein, et al. (2005), and Heckemann, et al. (2006), is becoming one of the most widely-used and successful image segmentation techniques in biomedical applications. By manipulating and utilizing the entire dataset of "atlases" (training images that have been previously labeled, e.g., manually by an expert), rather than some model-based average representation, MAS has the flexibility to better capture anatomical variation, thus offering superior segmentation accuracy. This benefit, however, typically comes at a high computational cost. Recent advancements in computer hardware and image processing software have been instrumental in addressing this challenge and facilitated the wide adoption of MAS. Today, MAS has come a long way and the approach includes a wide array of sophisticated algorithms that employ ideas from machine learning, probabilistic modeling, optimization, and computer vision, among other fields. This paper presents a survey of published MAS algorithms and studies that have applied these methods to various biomedical problems. In writing this survey, we have three distinct aims. Our primary goal is to document how MAS was originally conceived, later evolved, and now relates to alternative methods. Second, this paper is intended to be a detailed reference of past research activity in MAS, which now spans over a decade (2003-2014) and entails novel methodological developments and application-specific solutions. Finally, our goal is to also present a perspective on the future of MAS, which, we believe, will be one of the dominant approaches in biomedical image segmentation.

Spinal Navigation and Imaging: History, Trends, and Future

The clinical practice of spine navigation has rapidly grown with the development of image-based guidance. In this paper, a brief history of spinal navigation is presented and a review of clinical outcomes for 12,622 pedicle screws placed using the latest technology in the sacral, lumbar and thoracic regions. The clinical evidence demonstrate that intraoperative 3D image guided surgery has a 96.8% success rate. A concluding section detailing existing barriers that limit more widespread adoption and future development efforts is presented.

A new region descriptor for multi-modal medical image registration and region detection

Establishing accurate anatomical correspondences plays a critical role in multi-modal medical image registration and region detection. Although many features based registration methods have been proposed to detect these correspondences, they are mostly based on the point descriptor which leads to high memory cost and could not represent local region information. In this paper, we propose a new region descriptor which depicts the features in each region, instead of in each point, as a vector. First, feature attributes of each point are extracted by a Gabor filter bank combined with a gradient filter. Then, the region descriptor is defined as the covariance of feature attributes of each point inside the region, based on which a cost function is constructed for multi-modal image registration. Finally, our proposed region descriptor is applied to both multi-modal region detection and similarity metric measurement in multi-modal image registration. Experiments demonstrate the feasibility and effectiveness of our proposed region descriptor.

Improving anatomical mapping of complexly deformed anatomy for external beam radiotherapy and brachytherapy dose accumulation in cervical cancer

PURPOSE

In the treatment of cervical cancer, large anatomical deformations, caused by, e.g., tumor shrinkage, bladder and rectum filling changes, organ sliding, and the presence of the brachytherapy (BT) applicator, prohibit the accumulation of external beam radiotherapy (EBRT) and BT dose distributions. This work proposes a structure-wise registration with vector field integration (SW+VF) to map the largely deformed anatomies between EBRT and BT, paving the way for 3D dose accumulation between EBRT and BT.

METHODS

T2w-MRIs acquired before EBRT and as a part of the MRI-guided BT procedure for 12 cervical cancer patients, along with the manual delineations of the bladder, cervix-uterus, and rectum-sigmoid, were used for this study. A rigid transformation was used to align the bony anatomy in the MRIs. The proposed SW+VF method starts by automatically segmenting features in the area surrounding the delineated organs. Then, each organ and feature pair is registered independently using a feature-based nonrigid registration algorithm developed in-house. Additionally, a background transformation is calculated to account for areas far from all organs and features. In order to obtain one transformation that can be used for dose accumulation, the organ-based, feature-based, and the background transformations are combined into one vector field using a weighted sum, where the contribution of each transformation can be directly controlled by its extent of influence (scope size). The optimal scope sizes for organ-based and feature-based transformations were found by an exhaustive analysis. The anatomical correctness of the mapping was independently validated by measuring the residual distances after transformation for delineated structures inside the cervix-uterus (inner anatomical correctness), and for anatomical landmarks outside the organs in the surrounding region (outer anatomical correctness). The results of the proposed method were compared with the results of the rigid transformation and nonrigid registration of all structures together (AST).

RESULTS

The rigid transformation achieved a good global alignment (mean outer anatomical correctness of 4.3 mm) but failed to align the deformed organs (mean inner anatomical correctness of 22.4 mm). Conversely, the AST registration produced a reasonable alignment for the organs (6.3 mm) but not for the surrounding region (16.9 mm). SW+VF registration achieved the best results for both regions (3.5 and 3.4 mm for the inner and outer anatomical correctness, respectively). All differences were significant (p < 0.02, Wilcoxon rank sum test). Additionally, optimization of the scope sizes determined that the method was robust for a large range of scope size values.

CONCLUSIONS

The novel SW+VF method improved the mapping of large and complex deformations observed between EBRT and BT for cervical cancer patients. Future studies that quantify the mapping error in terms of dose errors are required to test the clinical applicability of dose accumulation by the SW+VF method.

A new method to validate thoracic CT-CT deformable image registration using auto-segmented 3D anatomical landmarks

BACKGROUND

Deformable image registrations are prone to errors in aligning reliable anatomically features. Consequently, identification of registration inaccuracies is important. Particularly thoracic three-dimensional (3D) computed tomography (CT)-CT image registration is challenging due to lack of contrast in lung tissue. This study aims for validation of thoracic CT-CT image registration using auto-segmented anatomically landmarks.

MATERIAL AND METHODS

Five lymphoma patients were CT scanned three times within a period of 18 months, with the initial CT defined as the reference scan. For each patient the two successive CT scans were registered to the reference CT using three different image registration algorithms (Demons, B-spline and Affine). The image registrations were evaluated using auto-segmented anatomical landmarks (bronchial branch points) and Dice Similarity Coefficients (DSC). Deviation of corresponding bronchial landmarks were used to quantify inaccuracies in respect of both misalignment and geometric location within lungs.

RESULTS

The median bronchial branch point deviations were 1.6, 1.1 and 4.2 (mm) for the three tested algorithms (Demons, B-spline and Affine). The maximum deviations (> 15 mm) were found within both Demons and B-spline image registrations. In the upper part of the lungs the median deviation of 1.7 (mm) was significantly different (p < 0.02) relative to the median deviations of 2.0 (mm), found in the middle and lower parts of the lungs. The DSC revealed similar registration discrepancies among the three tested algorithms, with DSC values of 0.96, 0.97 and 0.91, for respectively Demons, B-spline and the Affine algorithms.

CONCLUSION

Bronchial branch points were found useful to validate thoracic CT-CT image registration. Bronchial branch points identified local registration errors > 15 mm in both Demons and B-spline deformable algorithms.

Low-Dimensional Non-Rigid Image Registration Using Statistical Deformation Models From Semi-Supervised Training Data

Accurate and robust image registration is a fundamental task in medical image analysis applications, and requires non-rigid transformations with a large number of degrees of freedom. Statistical deformation models (SDMs) attempt to learn the distribution of non-rigid deformations, and can be used both to reduce the transformation dimensionality and to constrain the registration process. However, high-dimensional SDMs are difficult to train given orders of magnitude fewer training samples. In this paper, we utilize both a small set of annotated imaging data and a large set of unlabeled data to effectively learn an SDM of non-rigid transformations in a semi-supervised training (SST) framework. We demonstrate results applying this framework towards inter-subject registration of skull-stripped, magnetic resonance (MR) brain images. Our approach makes use of 39 labeled MR datasets to create a set of supervised registrations, which we augment with a set of over 1200 unsupervised registrations using unlabeled MRIs. Through leave-one-out cross validation, we show that SST of a non-rigid SDM results in a robust registration algorithm with significantly improved accuracy compared to standard, intensity-based registration, and does so with a 99% reduction in transformation dimensionality.

Multimodal image registration with joint structure tensor and local entropy

PURPOSE

Nonrigid registration of multimodal medical images remains a challenge in image-guided interventions. A common approach is to use mutual information (MI), which is robust to the intensity variations across modalities. However, primarily based on intensity distribution, MI does not take into account of underlying spatial and structural information of the images, which might lead to local optimization. To address such a challenge, this paper proposes a two-stage multimodal nonrigid registration scheme with joint structural information and local entropy.

METHODS

In our two-stage multimodal nonrigid registration scheme, both the reference image and floating image are firstly converted to a common space. A unified representation in the common space for the images is constructed by fusing the structure tensor (ST) trace with the local entropy (LE). Through the representation that reflects its geometry uniformly across modalities, the complicated deformation field is estimated using L(1) or L(2) distance.

RESULTS

We compared our approach to four other methods: (1) the method using LE, (2) the method using ST, (3) the method using spatially weighted LE and (4) the conventional MI-based method. Quantitative evaluations on 80 multimodal image pairs of different organs including 50 pairs of MR images with artificial deformations, 20 pairs of medical brain MR images and 10 pairs of breast images showed that our proposed method outperformed the comparison methods. Student's t test demonstrated that our method achieved statistically significant improvement on registration accuracy.

CONCLUSION

The two-stage registration with joint ST and LE outperformed the conventional MI-based method for multimodal images. Both the ST and the LE contributed to the improved registration accuracy.

Robust and efficient linear registration of white-matter fascicles in the space of streamlines

The neuroscientific community today is very much interested in analyzing specific white matter bundles like the arcuate fasciculus, the corticospinal tract, or the recently discovered Aslant tract to study sex differences, lateralization and many other connectivity applications. For this reason, experts spend time manually segmenting these fascicles and bundles using streamlines obtained from diffusion MRI tractography. However, to date, there are very few computational tools available to register these fascicles directly so that they can be analyzed and their differences quantified across populations. In this paper, we introduce a novel, robust and efficient framework to align bundles of streamlines directly in the space of streamlines. We call this framework Streamline-based Linear Registration. We first show that this method can be used successfully to align individual bundles as well as whole brain streamlines. Additionally, if used as a piecewise linear registration across many bundles, we show that our novel method systematically provides higher overlap (Jaccard indices) than state-of-the-art nonlinear image-based registration in the white matter. We also show how our novel method can be used to create bundle-specific atlases in a straightforward manner and we give an example of a probabilistic atlas construction of the optic radiation. In summary, Streamline-based Linear Registration provides a solid registration framework for creating new methods to study the white matter and perform group-level tractometry analysis.

An Inexact Newton-Krylov Algorithm for Constrained Diffeomorphic Image Registration

We propose numerical algorithms for solving large deformation diffeomorphic image registration problems. We formulate the nonrigid image registration problem as a problem of optimal control. This leads to an infinite-dimensional partial differential equation (PDE) constrained optimization problem. The PDE constraint consists, in its simplest form, of a hyperbolic transport equation for the evolution of the image intensity. The control variable is the velocity field. Tikhonov regularization on the control ensures well-posedness. We consider standard smoothness regularization based on H¹- or H²-seminorms. We augment this regularization scheme with a constraint on the divergence of the velocity field (control variable) rendering the deformation incompressible (Stokes regularization scheme) and thus ensuring that the determinant of the deformation gradient is equal to one, up to the numerical error. We use a Fourier pseudospectral discretization in space and a Chebyshev pseudospectral discretization in time. The latter allows us to reduce the number of unknowns and enables the time-adaptive inversion for nonstationary velocity fields. We use a preconditioned, globalized, matrix-free, inexact Newton-Krylov method for numerical optimization. A parameter continuation is designed to estimate an optimal regularization parameter. Regularity is ensured by controlling the geometric properties of the deformation field. Overall, we arrive at a black-box solver that exploits computational tools that are precisely tailored for solving the optimality system. We study spectral properties of the Hessian, grid convergence, numerical accuracy, computational efficiency, and deformation regularity of our scheme. We compare the designed Newton-Krylov methods with a globalized Picard method (preconditioned gradient descent). We study the influence of a varying number of unknowns in time. The reported results demonstrate excellent numerical accuracy, guaranteed local deformation regularity, and computational efficiency with an optional control on local mass conservation. The Newton-Krylov methods clearly outperform the Picard method if high accuracy of the inversion is required. Our method provides equally good results for stationary and nonstationary velocity fields for two-image registration problems.

Patient-specific biomechanical model as whole-body CT image registration tool

Whole-body computed tomography (CT) image registration is important for cancer diagnosis, therapy planning and treatment. Such registration requires accounting for large differences between source and target images caused by deformations of soft organs/tissues and articulated motion of skeletal structures. The registration algorithms relying solely on image processing methods exhibit deficiencies in accounting for such deformations and motion. We propose to predict the deformations and movements of body organs/tissues and skeletal structures for whole-body CT image registration using patient-specific non-linear biomechanical modelling. Unlike the conventional biomechanical modelling, our approach for building the biomechanical models does not require time-consuming segmentation of CT scans to divide the whole body into non-overlapping constituents with different material properties. Instead, a Fuzzy C-Means (FCM) algorithm is used for tissue classification to assign the constitutive properties automatically at integration points of the computation grid. We use only very simple segmentation of the spine when determining vertebrae displacements to define loading for biomechanical models. We demonstrate the feasibility and accuracy of our approach on CT images of seven patients suffering from cancer and aortic disease. The results confirm that accurate whole-body CT image registration can be achieved using a patient-specific non-linear biomechanical model constructed without time-consuming segmentation of the whole-body images.

A hybrid biomechanical intensity based deformable image registration of lung 4DCT

Deformable image registration (DIR) has been extensively studied over the past two decades due to its essential role in many image-guided interventions (IGI). IGI demands a highly accurate registration that maintains its accuracy across the entire region of interest. This work evaluates the improvement in accuracy and consistency by refining the results of Morfeus, a biomechanical model-based DIR algorithm. A hybrid DIR algorithm is proposed based on, a biomechanical model-based DIR algorithm and a refinement step based on a B-spline intensity-based algorithm. Inhale and exhale reconstructions of four-dimensional computed tomography (4DCT) lung images from 31 patients were initially registered using the biomechanical DIR by modeling contact surface between the lungs and the chest cavity. The resulting deformations were then refined using the intensity-based algorithm to reduce any residual uncertainties. Important parameters in the intensity-based algorithm, including grid spacing, number of pyramids, and regularization coefficient, were optimized on 10 randomly-chosen patients (out of 31). Target registration error (TRE) was calculated by measuring the Euclidean distance of common anatomical points on both images after registration. For each patient a minimum of 30 points/lung were used. Grid spacing of 8 mm, 5 levels of grid pyramids, and regularization coefficient of 3.0 were found to provide optimal results on 10 randomly chosen patients. Overall the entire patient population (n = 31), the hybrid method resulted in mean ± SD (90th%) TRE of 1.5 ± 1.4 (2.9) mm compared to 3.1 ± 1.9 (5.6) using biomechanical DIR and 2.6 ± 2.5 (6.1) using intensity-based DIR alone. The proposed hybrid biomechanical modeling intensity based algorithm is a promising DIR technique which could be used in various IGI procedures. The current investigation shows the efficacy of this approach for the registration of 4DCT images of the lungs with average accuracy of 1.5 mm.

Using image synthesis for multi-channel registration of different image modalities

This paper presents a multi-channel approach for performing registration between magnetic resonance (MR) images with different modalities. In general, a multi-channel registration cannot be used when the moving and target images do not have analogous modalities. In this work, we address this limitation by using a random forest regression technique to synthesize the missing modalities from the available ones. This allows a single channel registration between two different modalities to be converted into a multi-channel registration with two mono-modal channels. To validate our approach, two openly available registration algorithms and five cost functions were used to compare the label transfer accuracy of the registration with (and without) our multi-channel synthesis approach. Our results show that the proposed method produced statistically significant improvements in registration accuracy (at an α level of 0.001) for both algorithms and all cost functions when compared to a standard multi-modal registration using the same algorithms with mutual information.

Ultrasonic characterization of crack-like defects using scattering matrix similarity metrics

Crack-like defects form an important type of target defect in nondestructive evaluation, and accurately characterizing them remains a challenge, particularly for small cracks and inclined cracks. In this paper, scattering matrices are used for defect characterization through use of the correlation coefficient and the structural similarity (SSIM) index as similarity metrics. A set of reference cracks that have different lengths and orientation angles are compared with the test defect and the best match is determined in terms of the maximum similarity score between the scattering matrices of the test defect and reference cracks. Defect characterization using similarity metrics is invariant to scale and shift, so calibration of experimental data is not needed. Principal component analysis (PCA) is adopted to reduce the effect of measurement noise and recover the original shape of scattering matrices from noisy data. The performance of the proposed algorithm is studied in both simulation and experiments. The length and orientation angle of four different test cracks are measured at two different noise levels in the simulation case, and excellent agreement is achieved between the measurement results and the actual values. Experimentally, the lengths of five subwavelength cracks are measured to within 0.10 mm, and their orientation angles are measured to within 5°.

Computerized PET/CT image analysis in the evaluation of tumour response to therapy

Current cancer therapy strategy is mostly population based, however, there are large differences in tumour response among patients. It is therefore important for treating physicians to know individual tumour response. In recent years, many studies proposed the use of computerized positron emission tomography/CT image analysis in the evaluation of tumour response. Results showed that computerized analysis overcame some major limitations of current qualitative and semiquantitative analysis and led to improved accuracy. In this review, we summarize these studies in four steps of the analysis: image registration, tumour segmentation, image feature extraction and response evaluation. Future works are proposed and challenges described.

View-centralized multi-atlas classification for Alzheimer's disease diagnosis

Multi-atlas based methods have been recently used for classification of Alzheimer's disease (AD) and its prodromal stage, that is, mild cognitive impairment (MCI). Compared with traditional single-atlas based methods, multiatlas based methods adopt multiple predefined atlases and thus are less biased by a certain atlas. However, most existing multiatlas based methods simply average or concatenate the features from multiple atlases, which may ignore the potentially important diagnosis information related to the anatomical differences among different atlases. In this paper, we propose a novel view (i.e., atlas) centralized multi-atlas classification method, which can better exploit useful information in multiple feature representations from different atlases. Specifically, all brain images are registered onto multiple atlases individually, to extract feature representations in each atlas space. Then, the proposed view-centralized multi-atlas feature selection method is used to select the most discriminative features from each atlas with extra guidance from other atlases. Next, we design a support vector machine (SVM) classifier using the selected features in each atlas space. Finally, we combine multiple SVM classifiers for multiple atlases through a classifier ensemble strategy for making a final decision. We have evaluated our method on 459 subjects [including 97 AD, 117 progressive MCI (p-MCI), 117 stable MCI (s-MCI), and 128 normal controls (NC)] from the Alzheimer's Disease Neuroimaging Initiative database, and achieved an accuracy of 92.51% for AD versus NC classification and an accuracy of 78.88% for p-MCI versus s-MCI classification. These results demonstrate that the proposed method can significantly outperform the previous multi-atlas based classification methods.

Inverse consistent non-rigid image registration based on robust point set matching

Background

Robust point matching (RPM) has been extensively used in non-rigid registration of images to robustly register two sets of image points. However, except for the location at control points, RPM cannot estimate the consistent correspondence between two images because RPM is a unidirectional image matching approach. Therefore, it is an important issue to make an improvement in image registration based on RPM.

Methods

In our work, a consistent image registration approach based on the point sets matching is proposed to incorporate the property of inverse consistency and improve registration accuracy. Instead of only estimating the forward transformation between the source point sets and the target point sets in state-of-the-art RPM algorithms, the forward and backward transformations between two point sets are estimated concurrently in our algorithm. The inverse consistency constraints are introduced to the cost function of RPM and the fuzzy correspondences between two point sets are estimated based on both the forward and backward transformations simultaneously. A modified consistent landmark thin-plate spline registration is discussed in detail to find the forward and backward transformations during the optimization of RPM. The similarity of image content is also incorporated into point matching in order to improve image matching.

Results

Synthetic data sets, medical images are employed to demonstrate and validate the performance of our approach. The inverse consistent errors of our algorithm are smaller than RPM. Especially, the topology of transformations is preserved well for our algorithm for the large deformation between point sets. Moreover, the distance errors of our algorithm are similar to that of RPM, and they maintain a downward trend as whole, which demonstrates the convergence of our algorithm. The registration errors for image registrations are evaluated also. Again, our algorithm achieves the lower registration errors in same iteration number. The determinant of the Jacobian matrix of the deformation field is used to analyse the smoothness of the forward and backward transformations. The forward and backward transformations estimated by our algorithm are smooth for small deformation. For registration of lung slices and individual brain slices, large or small determinant of the Jacobian matrix of the deformation fields are observed.

Conclusions

Results indicate the improvement of the proposed algorithm in bi-directional image registration and the decrease of the inverse consistent errors of the forward and the reverse transformations between two images.

A combined method for segmentation and registration for an advanced and progressive evaluation of thermal images

In this paper, a method that combines image analysis techniques, such as segmentation and registration, is proposed for an advanced and progressive evaluation of thermograms. The method is applied for the prevention of muscle injury in high-performance athletes, in collaboration with a Brazilian professional soccer club. The goal is to produce information on spatio-temporal variations of thermograms favoring the investigation of the athletes' conditions along the competition. The proposed method improves on current practice by providing a means for automatically detecting adaptive body-shaped regions of interest, instead of the manual selection of simple shapes. Specifically, our approach combines the optimization features in Otsu's method with a correction factor and post-processing techniques, enhancing thermal-image segmentation when compared to other methods. Additional contributions resulting from the combination of the segmentation and registration steps of our approach are the progressive analyses of thermograms in a unique spatial coordinate system and the accurate extraction of measurements and isotherms.

Predict brain MR image registration via sparse learning of appearance and transformation

We propose a new approach to register the subject image with the template by leveraging a set of intermediate images that are pre-aligned to the template. We argue that, if points in the subject and the intermediate images share similar local appearances, they may have common correspondence in the template. In this way, we learn the sparse representation of a certain subject point to reveal several similar candidate points in the intermediate images. Each selected intermediate candidate can bridge the correspondence from the subject point to the template space, thus predicting the transformation associated with the subject point at the confidence level that relates to the learned sparse coefficient. Following this strategy, we first predict transformations at selected key points, and retain multiple predictions on each key point, instead of allowing only a single correspondence. Then, by utilizing all key points and their predictions with varying confidences, we adaptively reconstruct the dense transformation field that warps the subject to the template. We further embed the prediction-reconstruction protocol above into a multi-resolution hierarchy. In the final, we refine our estimated transformation field via existing registration method in effective manners. We apply our method to registering brain MR images, and conclude that the proposed framework is competent to improve registration performances substantially.

Assessing tumor response after loco-regional liver cancer therapies: the role of 3D MRI

Assessing the tumor response of liver cancer lesions after intraarterial therapies is of major clinical interest. Over the last two decades, tumor response criteria have come a long way from purely size-based, anatomic methods such as the Response Evaluation Criteria in Solid Tumors towards more functional, enhancement- and diffusion-based parameters with a strong emphasis on MRI as the ultimate imaging modality. However, the relatively low reproducibility of those one- and 2D techniques (modified Response Evaluation Criteria in Solid Tumors and the European Association for the Study of the Liver criteria) provided the rationale for the development of new, 3D quantitative assessment techniques. This review will summarize and compare the existing methodologies used for 3D quantitative tumor analysis and provide an overview of the published clinical evidence for the benefits of 3D quantitative tumor response assessment techniques.

Evaluation of accuracy of B-spline transformation-based deformable image registration with different parameter settings for thoracic images

Deformable image registration (DIR) is fundamental technique for adaptive radiotherapy and image-guided radiotherapy. However, further improvement of DIR is still needed. We evaluated the accuracy of B-spline transformation-based DIR implemented in elastix. This registration package is largely based on the Insight Segmentation and Registration Toolkit (ITK), and several new functions were implemented to achieve high DIR accuracy. The purpose of this study was to clarify whether new functions implemented in elastix are useful for improving DIR accuracy. Thoracic 4D computed tomography images of ten patients with esophageal or lung cancer were studied. Datasets for these patients were provided by DIR-lab (dir-lab.com) and included a coordinate list of anatomical landmarks that had been manually identified. DIR between peak-inhale and peak-exhale images was performed with four types of parameter settings. The first one represents original ITK (Parameter 1). The second employs the new function of elastix (Parameter 2), and the third was created to verify whether new functions improve DIR accuracy while keeping computational time (Parameter 3). The last one partially employs a new function (Parameter 4). Registration errors for these parameter settings were calculated using the manually determined landmark pairs. 3D registration errors with standard deviation over all cases were 1.78 (1.57), 1.28 (1.10), 1.44 (1.09) and 1.36 (1.35) mm for Parameter 1, 2, 3 and 4, respectively, indicating that the new functions are useful for improving DIR accuracy, even while maintaining the computational time, and this B-spline-based DIR could be used clinically to achieve high-accuracy adaptive radiotherapy.

Avoiding symmetry-breaking spatial non-uniformity in deformable image registration via a quasi-volume-preserving constraint

The choice of a reference image typically influences the results of deformable image registration, thereby making it asymmetric. This is a consequence of a spatially non-uniform weighting in the cost function integral that leads to general registration inaccuracy. The inhomogeneous integral measure--which is the local volume change in the transformation, thus varying through the course of the registration--causes image regions to contribute differently to the objective function. More importantly, the optimization algorithm is allowed to minimize the cost function by manipulating the volume change, instead of aligning the images. The approaches that restore symmetry to deformable registration successfully achieve inverse-consistency, but do not eliminate the regional bias that is the source of the error. In this work, we address the root of the problem: the non-uniformity of the cost function integral. We introduce a new quasi-volume-preserving constraint that allows for volume change only in areas with well-matching image intensities, and show that such a constraint puts a bound on the error arising from spatial non-uniformity. We demonstrate the advantages of adding the proposed constraint to standard (asymmetric and symmetrized) demons and diffeomorphic demons algorithms through experiments on synthetic images, and real X-ray and 2D/3D brain MRI data. Specifically, the results show that our approach leads to image alignment with more accurate matching of manually defined neuroanatomical structures, better tradeoff between image intensity matching and registration-induced distortion, improved native symmetry, and lower susceptibility to local optima. In summary, the inclusion of this space- and time-varying constraint leads to better image registration along every dimension that we have measured it.

Comparative evaluation of registration algorithms in different brain databases with varying difficulty: results and insights

Evaluating various algorithms for the inter-subject registration of brain magnetic resonance images (MRI) is a necessary topic receiving growing attention. Existing studies evaluated image registration algorithms in specific tasks or using specific databases (e.g., only for skull-stripped images, only for single-site images, etc.). Consequently, the choice of registration algorithms seems task- and usage/parameter-dependent. Nevertheless, recent large-scale, often multi-institutional imaging-related studies create the need and raise the question whether some registration algorithms can 1) generally apply to various tasks/databases posing various challenges; 2) perform consistently well, and while doing so, 3) require minimal or ideally no parameter tuning. In seeking answers to this question, we evaluated 12 general-purpose registration algorithms, for their generality, accuracy and robustness. We fixed their parameters at values suggested by algorithm developers as reported in the literature. We tested them in 7 databases/tasks, which present one or more of 4 commonly-encountered challenges: 1) inter-subject anatomical variability in skull-stripped images; 2) intensity homogeneity, noise and large structural differences in raw images; 3) imaging protocol and field-of-view (FOV) differences in multi-site data; and 4) missing correspondences in pathology-bearing images. Totally 7,562 registrations were performed. Registration accuracies were measured by (multi-)expert-annotated landmarks or regions of interest (ROIs). To ensure reproducibility, we used public software tools, public databases (whenever possible), and we fully disclose the parameter settings. We show evaluation results, and discuss the performances in light of algorithms' similarity metrics, transformation models and optimization strategies. We also discuss future directions for the algorithm development and evaluations.

Strain analysis from 4-D cardiac CT image data

Strain is a discriminative parameter of regional myocardial dysfunction. Despite the large body of research on myocardial strain analysis in echocardiography and MR images, such techniques have not often been applied to cardiac CT data. Reasons for this include the challenges of sparse image deformation clues and the low temporal resolution. In the current study, we propose an algorithm that uses cardiac CT data to evaluate the mechanical function of the left ventricle. The algorithm is based on a deformable LV model that contains both the myocardium and the blood pool regions and that accounts for the elasticity and incompressibility of the myocardium with the rapid contraction of the blood pool. Our algorithm uses the image intensities of the trabeculle and papillary muscles as well as the border edges in an optical flow manner to extract the 3-D velocities. The resulting strains and rotational values derived from a set of normal patients correlate highly with values from the research literature. We validated our algorithm against 2-D speckle tracking analysis and against visual scores obtained by an expert. Our study shows that strain analysis using CT data can be used in clinical practice.

LittleQuickWarp: an ultrafast image warping tool

Warping images into a standard coordinate space is critical for many image computing related tasks. However, for multi-dimensional and high-resolution images, an accurate warping operation itself is often very expensive in terms of computer memory and computational time. For high-throughput image analysis studies such as brain mapping projects, it is desirable to have high performance image warping tools that are compatible with common image analysis pipelines. In this article, we present LittleQuickWarp, a swift and memory efficient tool that boosts 3D image warping performance dramatically and at the same time has high warping quality similar to the widely used thin plate spline (TPS) warping. Compared to the TPS, LittleQuickWarp can improve the warping speed 2-5 times and reduce the memory consumption 6-20 times. We have implemented LittleQuickWarp as an Open Source plug-in program on top of the Vaa3D system (http://vaa3d.org). The source code and a brief tutorial can be found in the Vaa3D plugin source code repository.

Gastroscopic image graph: application to noninvasive multitarget tracking under gastroscopy

Gastroscopic examination is one of the most common methods for gastric disease diagnosis. In this paper, a multitarget tracking approach is proposed to assist endoscopists in identifying lesions under gastroscopy. This approach analyzes numerous preobserved gastroscopic images and constructs a gastroscopic image graph. In this way, the deformation registration between gastroscopic images is regarded as a graph search problem. During the procedure, the endoscopist marks suspicious lesions on the screen and the graph is utilized to locate and display the lesions in the appropriate frames based on the calculated registration model. Compared to traditional gastroscopic lesion surveillance methods (e.g., tattooing or probe-based optical biopsy), this approach is noninvasive and does not require additional instruments. In order to assess and quantify the performance, this approach was applied to stomach phantom data and in vivo data. The clinical experimental results demonstrated that the accuracy at angularis, antral, and stomach body was 6.3 ± 2.4 mm, 7.6 ± 3.1 mm, and 7.9 ± 1.6 mm, respectively. The mean accuracy was 7.31 mm, average targeting time was 56 ms, and the P value was 0.032, which makes it an attractive candidate for clinical practice. Furthermore, this approach provides a significant reference for endoscopic target tracking of other soft tissue organs.