Image registration using hierarchical B-splines

SFM-Net: Semantic Feature-Based Multi-Stage Network for Unsupervised Image Registration

It is difficult for general registration methods to establish the fine correspondence between images with complex anatomical structures. To overcome the above problem, this work presents SFM-Net, an unsupervised multi-stage semantic feature-based network. In addition to using the pixel-based similarity metrics, we propose a feature operator and emphasize a feature registration to improve the alignment of semantic related areas. Specifically, we design a two-stage training strategy, the intensity image registration stage and the semantic feature registration stage. The former is for valid semantic features learning and intensity-based coarse registration, while the latter is for semantic areas alignment, achieving fine transformation of anatomical structure. The same structure of both stages is composed of a dual-stream feature extraction module (DFEM) and a refined deformation field generation module (RDGM). Unlike the deep learning-based approaches that utilizing down-sampled encoder to extract features, DFEM constructed by dual-stream U-Net structure can capture semantic information in decoder feature for structural alignment. Different with approaches applying cascaded networks to learn deformation field, our proposed RDGM generates multi-scale deformation fields by performing a coarse-to-fine registration within a single network. Experiments on 3D brain MRI and liver CT datasets confirm that the proposed SFM-Net achieves accurate and diffeomorphic registration results, outperforming other state-of-the-art methods.

A Segmentation Framework With Unsupervised Learning-Based Label Mapper for the Ventricular Target of Intracranial Germ Cell Tumor

Intracranial germ cell tumors are rare tumors that mainly affect children and adolescents. Radiotherapy is the cornerstone of interdisciplinary treatment methods. Radiation of the whole ventricle system and the local tumor can reduce the complications in the late stage of radiotherapy while ensuring the curative effect. However, manually delineating the ventricular system is labor-intensive and time-consuming for physicians. The diverse ventricle shape and the hydrocephalus-induced ventricle dilation increase the difficulty of automatic segmentation algorithms. Therefore, this study proposed a fully automatic segmentation framework. Firstly, we designed a novel unsupervised learning-based label mapper, which is used to handle the ventricle shape variations and obtain the preliminary segmentation result. Then, to boost the segmentation performance of the framework, we improved the region growth algorithm and combined the fully connected conditional random field to optimize the preliminary results from both regional and voxel scales. In the case of only one set of annotated data is required, the average time cost is 153.01 s, and the average target segmentation accuracy can reach 84.69%. Furthermore, we verified the algorithm in practical clinical applications. The results demonstrate that our proposed method is beneficial for physicians to delineate radiotherapy targets, which is feasible and clinically practical, and may fill the gap of automatic delineation methods for the ventricular target of intracranial germ celltumors.

Quantitative Validation of the Correlation Between Optimized Pyramidal Tract Delineation After Brain Shift Compensation and Direct Electrical Subcortical Stimulation During Brain Tumor Surgery

It remains unclear whether tractography of pyramidal tracts is correlated with the intraoperative direct electrical subcortical stimulation (DESS), and brain shift further complicates the issue. The objective of this research is to quantitatively verify the correlation between optimized tractography (OT) of pyramidal tracts after brain shift compensation and DESS during brain tumor surgery. OT was performed for 20 patients with lesions in proximity to the pyramidal tracts based on preoperative diffusion-weighted magnetic resonance imaging. During surgery, tumor resection was guided by DESS. A total of 168 positive stimulation points and their corresponding stimulation intensity thresholds were recorded. Using the brain shift compensation algorithm based on hierarchical B-spline grids combined with a Gaussian resolution pyramid, we warped the preoperative pyramidal tract models and used receiver operating characteristic (ROC) curves to investigate the reliability of our brain shift compensation method based on anatomic landmarks. Additionally, the minimum distance between the DESS points and warped OT (wOT) model was measured and correlated with DESS intensity threshold. Brain shift compensation was achieved in all cases, and the area under the ROC curve was 0.96 in the registration accuracy analysis. The minimum distance between the DESS points and the wOT model was found to have a significantly high correlation with the DESS stimulation intensity threshold (r = 0.87, P < 0.001), with a linear regression coefficient of 0.96. Our OT method can provide comprehensive and accurate visualization of the pyramidal tracts for neurosurgical navigation and was quantitatively verified by intraoperative DESS after brain shift compensation.

A novel method for spine ultrasound and X-ray radiograph registration

Ultrasound is a promising imaging method for scoliosis evaluation because it is radiation free and provide real-time images. However, it cannot provide bony details because ultrasound cannot penetrate the bony structure. Therefore, registration of real-time ultrasound images with the previous X-ray radiograph can help physicians understand the spinal deformity of patients. In this study, an improved free-from deformation registration method based on mutual registration and hierarchical adaptive grid (MRHA-FFD) was developed. The method first performed registration grid preprocessing and then optimized control points and conducted mutual registration. Finally, a Blur-aware Attention Network was adopted for image deblurring. The performance of each step was verified by ablation experiments. Comparison experiment between the proposed method and traditional registration methods was also conducted. The qualitative and quantitative results suggested that MRHA-FFD is a promising approach for registering spine ultrasound image and X-ray radiograph.

Robust Measures of Image-Registration-Derived Lung Biomechanics in SPIROMICS

Chronic obstructive pulmonary disease (COPD) is an umbrella term used to define a collection of inflammatory lung diseases that cause airflow obstruction and severe damage to the lung parenchyma. This study investigated the robustness of image-registration-based local biomechanical properties of the lung in individuals with COPD as a function of Global Initiative for Chronic Obstructive Lung Disease (GOLD) stage. Image registration was used to estimate the pointwise correspondences between the inspiration (total lung capacity) and expiration (residual volume) computed tomography (CT) images of the lung for each subject. In total, three biomechanical measures were computed from the correspondence map: the Jacobian determinant; the anisotropic deformation index (ADI); and the slab-rod index (SRI). CT scans from 245 subjects with varying GOLD stages were analyzed from the SubPopulations and InteRmediate Outcome Measures In COPD Study (SPIROMICS). Results show monotonic increasing or decreasing trends in the three biomechanical measures as a function of GOLD stage for the entire lung and on a lobe-by-lobe basis. Furthermore, these trends held across all five image registration algorithms. The consistency of the five image registration algorithms on a per individual basis is shown using Bland-Altman plots.

Space-variant Shack-Hartmann wavefront sensing based on affine transformation estimation

The space-variant wavefront reconstruction problem inherently exists in deep tissue imaging. In this paper, we propose a framework of Shack-Hartmann wavefront space-variant sensing with extended source illumination. The space-variant wavefront is modeled as a four-dimensional function where two dimensions are in the spatial domain and two are in the Fourier domain with priors that both gently vary. Here, the affine transformation is used to characterize the wavefront space-variant function. Correspondingly, the zonal and modal methods are both escalated to adapt to four-dimensional representation and reconstruction. Experiments and simulations show double to quadruple improvements in space-variant wavefront reconstruction accuracy compared to the conventional space-invariant correlation method.

PDE-constrained shape registration to characterize biological growth and morphogenesis from imaging data

We propose a PDE-constrained shape registration algorithm that captures the deformation and growth of biological tissue from imaging data. Shape registration is the process of evaluating optimum alignment between pairs of geometries through a spatial transformation function. We start from our previously reported work, which uses 3D tensor product B-spline basis functions to interpolate 3D space. Here, the movement of the B-spline control points, composed with an implicit function describing the shape of the tissue, yields the total deformation gradient field. The deformation gradient is then split into growth and elastic contributions. The growth tensor captures addition of mass, i.e. growth, and evolves according to a constitutive equation which is usually a function of the elastic deformation. Stress is generated in the material due to the elastic component of the deformation alone. The result of the registration is obtained by minimizing a total energy functional which includes: a distance measure reflecting similarity between the shapes, and the total elastic energy accounting for the growth of the tissue. We apply the proposed shape registration framework to study zebrafish embryo epiboly process and tissue expansion during skin reconstruction surgery. We anticipate that our PDE-constrained shape registration method will improve our understanding of biological and medical problems in which tissues undergo extreme deformations over time.

An Unsupervised Convolution Neural Network for Deformable Registration of Mono/Multi-Modality Medical Images

Image registration is a fundamental and crucial step in medical image analysis. However, due to the differences between mono-mode and multi-mode registration tasks and the complexity of the corresponding relationship between multimode image intensity, the existing unsupervised methods based on deep learning can hardly achieve the two registration tasks simultaneously. In this paper, we proposed a novel approach to register both mono- and multi-mode images $\color{blue}{\text{in a differentiable }}$. By approximately calculating the mutual information in a $\color{blue}{\text{differentiable}}$ form and combining it with CNN, the deformation field can be predicted quickly and accurately without any prior information about the image intensity relationship. The registration process is implemented in an unsupervised manner, avoiding the need for the ground truth of the deformation field. We utilize two public datasets to evaluate the performance of the algorithm for mono-mode and multi-mode image registration, which confirms the effectiveness and feasibility of our method. In addition, the experiments on patient data also demonstrate the practicability and robustness of the proposed method.

Regional Localization of Mouse Brain Slices Based on Unified Modal Transformation

Brain science research often requires accurate localization and quantitative analysis of neuronal activity in different brain regions. The premise of related analysis is to determine the brain region of each site on the brain slice by referring to the Allen Reference Atlas (ARA), namely the regional localization of the brain slice. The image registration methodology can be used to solve the problem of regional localization. However, the conventional multi-modal image registration method is not satisfactory because of the complexity of modality between the brain slice and the ARA. Inspired by the idea that people can automatically ignore noise and establish correspondence based on key regions, we proposed a novel method known as the Joint Enhancement of Multimodal Information (JEMI) network, which is based on a symmetric encoder–decoder. In this way, the brain slice and the ARA are converted into a segmentation map with unified modality, which greatly reduces the difficulty of registration. Furthermore, combined with the diffeomorphic registration algorithm, the existing topological structure was preserved. The results indicate that, compared with the existing methods, the method proposed in this study can effectively overcome the influence of non-unified modal images and achieve accurate and rapid localization of the brain slice.

Prediction of the pre-morbid 3D anatomy of the proximal humerus based on statistical shape modelling

Aims

Restoring the pre-morbid anatomy of the proximal humerus is a goal of anatomical shoulder arthroplasty, but reliance is placed on the surgeon’s experience and on anatomical estimations. The purpose of this study was to present a novel method, ‘Statistical Shape Modelling’, which accurately predicts the pre-morbid proximal humeral anatomy and calculates the 3D geometric parameters needed to restore normal anatomy in patients with severe degenerative osteoarthritis or a fracture of the proximal humerus.

Materials and Methods

From a database of 57 humeral CT scans 3D humeral reconstructions were manually created. The reconstructions were used to construct a statistical shape model (SSM), which was then tested on a second set of 52 scans. For each humerus in the second set, 3D reconstructions of four diaphyseal segments of varying lengths were created. These reconstructions were chosen to mimic severe osteoarthritis, a fracture of the surgical neck of the humerus and a proximal humeral fracture with diaphyseal extension. The SSM was then applied to the diaphyseal segments to see how well it predicted proximal morphology, using the actual proximal humeral morphology for comparison.

Results

With the metaphysis included, mimicking osteoarthritis, the errors of prediction for retroversion, inclination, height, radius of curvature and posterior and medial offset of the head of the humerus were 2.9° (± 2.3°), 4.0° (± 3.3°), 1.0 mm (± 0.8 mm), 0.8 mm (± 0.6 mm), 0.7 mm (± 0.5 mm) and 1.0 mm (± 0.7 mm), respectively. With the metaphysis excluded, mimicking a fracture of the surgical neck, the errors of prediction for retroversion, inclination, height, radius of curvature and posterior and medial offset of the head of the humerus were 3.8° (± 2.9°), 3.9° (± 3.4°), 2.4 mm (± 1.9 mm), 1.3 mm (± 0.9 mm), 0.8 mm (± 0.5 mm) and 0.9 mm (± 0.6 mm), respectively.

Conclusion

This study reports a novel, computerised method that accurately predicts the pre-morbid proximal humeral anatomy even in challenging situations. This information can be used in the surgical planning and operative reconstruction of patients with severe degenerative osteoarthritis or with a fracture of the proximal humerus. Cite this article: Bone Joint J 2017;99-B:927–33.

A Neonatal Bimodal MR-CT Head Template

Neonatal MR templates are appropriate for brain structural analysis and spatial normalization. However, they do not provide the essential accurate details of cranial bones and fontanels-sutures. Distinctly, CT images provide the best contrast for bone definition and fontanels-sutures. In this paper, we present, for the first time, an approach to create a fully registered bimodal MR-CT head template for neonates with a gestational age of 39 to 42 weeks. Such a template is essential for structural and functional brain studies, which require precise geometry of the head including cranial bones and fontanels-sutures. Due to the special characteristics of the problem (which requires inter-subject inter-modality registration), a two-step intensity-based registration method is proposed to globally and locally align CT images with an available MR template. By applying groupwise registration, the new neonatal CT template is then created in full alignment with the MR template to build a bimodal MR-CT template. The mutual information value between the CT and the MR template is 1.17 which shows their perfect correspondence in the bimodal template. Moreover, the average mutual information value between normalized images and the CT template proposed in this study is 1.24±0.07. Comparing this value with the one reported in a previously published approach (0.63±0.07) demonstrates the better generalization properties of the new created template and the superiority of the proposed method for the creation of CT template in the standard space provided by MR neonatal head template. The neonatal bimodal MR-CT head template is freely downloadable from https://www.u-picardie.fr/labo/GRAMFC.

Deformable registration of CT and cone-beam CT with local intensity matching

Cone-beam CT (CBCT) is a widely used intra-operative imaging modality in image-guided radiotherapy and surgery. A short scan followed by a filtered-backprojection is typically used for CBCT reconstruction. While data on the mid-plane (plane of source-detector rotation) is complete, off-mid-planes undergo different information deficiency and the computed reconstructions are approximate. This causes different reconstruction artifacts at off-mid-planes depending on slice locations, and therefore impedes accurate registration between CT and CBCT. In this paper, we propose a method to accurately register CT and CBCT by iteratively matching local CT and CBCT intensities. We correct CBCT intensities by matching local intensity histograms slice by slice in conjunction with intensity-based deformable registration. The correction-registration steps are repeated in an alternating way until the result image converges. We integrate the intensity matching into three different deformable registration methods, B-spline, demons, and optical flow that are widely used for CT-CBCT registration. All three registration methods were implemented on a graphics processing unit for efficient parallel computation. We tested the proposed methods on twenty five head and neck cancer cases and compared the performance with state-of-the-art registration methods. Normalized cross correlation (NCC), structural similarity index (SSIM), and target registration error (TRE) were computed to evaluate the registration performance. Our method produced overall NCC of 0.96, SSIM of 0.94, and TRE of 2.26 → 2.27 mm, outperforming existing methods by 9%, 12%, and 27%, respectively. Experimental results also show that our method performs consistently and is more accurate than existing algorithms, and also computationally efficient.

A Parallel Nonrigid Registration Algorithm Based on B-Spline for Medical Images

The nonrigid registration algorithm based on B-spline Free-Form Deformation (FFD) plays a key role and is widely applied in medical image processing due to the good flexibility and robustness. However, it requires a tremendous amount of computing time to obtain more accurate registration results especially for a large amount of medical image data. To address the issue, a parallel nonrigid registration algorithm based on B-spline is proposed in this paper. First, the Logarithm Squared Difference (LSD) is considered as the similarity metric in the B-spline registration algorithm to improve registration precision. After that, we create a parallel computing strategy and lookup tables (LUTs) to reduce the complexity of the B-spline registration algorithm. As a result, the computing time of three time-consuming steps including B-splines interpolation, LSD computation, and the analytic gradient computation of LSD, is efficiently reduced, for the B-spline registration algorithm employs the Nonlinear Conjugate Gradient (NCG) optimization method. Experimental results of registration quality and execution efficiency on the large amount of medical images show that our algorithm achieves a better registration accuracy in terms of the differences between the best deformation fields and ground truth and a speedup of 17 times over the single-threaded CPU implementation due to the powerful parallel computing ability of Graphics Processing Unit (GPU).

Technical Note: scuda: A software platform for cumulative dose assessment

PURPOSE

Accurate tracking of anatomical changes and computation of actually delivered dose to the patient are critical for successful adaptive radiation therapy (ART). Additionally, efficient data management and fast processing are practically important for the adoption in clinic as ART involves a large amount of image and treatment data. The purpose of this study was to develop an accurate and efficient Software platform for CUmulative Dose Assessment (scuda) that can be seamlessly integrated into the clinical workflow.

METHODS

scuda consists of deformable image registration (DIR), segmentation, dose computation modules, and a graphical user interface. It is connected to our image PACS and radiotherapy informatics databases from which it automatically queries/retrieves patient images, radiotherapy plan, beam data, and daily treatment information, thus providing an efficient and unified workflow. For accurate registration of the planning CT and daily CBCTs, the authors iteratively correct CBCT intensities by matching local intensity histograms during the DIR process. Contours of the target tumor and critical structures are then propagated from the planning CT to daily CBCTs using the computed deformations. The actual delivered daily dose is computed using the registered CT and patient setup information by a superposition/convolution algorithm, and accumulated using the computed deformation fields. Both DIR and dose computation modules are accelerated by a graphics processing unit.

RESULTS

The cumulative dose computation process has been validated on 30 head and neck (HN) cancer cases, showing 3.5 ± 5.0 Gy (mean±STD) absolute mean dose differences between the planned and the actually delivered doses in the parotid glands. On average, DIR, dose computation, and segmentation take 20 s/fraction and 17 min for a 35-fraction treatment including additional computation for dose accumulation.

CONCLUSIONS

The authors developed a unified software platform that provides accurate and efficient monitoring of anatomical changes and computation of actually delivered dose to the patient, thus realizing an efficient cumulative dose computation workflow. Evaluation on HN cases demonstrated the utility of our platform for monitoring the treatment quality and detecting significant dosimetric variations that are keys to successful ART.

Automatic deformable surface registration for medical applications by radial basis function-based robust point-matching

Deformable surface mesh registration is a useful technique for various medical applications, such as intra-operative treatment guidance and intra- or inter-patient study. In this paper, we propose an automatic deformable mesh registration technique. The proposed method iteratively deforms a source mesh to a target mesh without manual feature extraction. Each iteration of the registration consists of two steps, automatic correspondence finding using robust point-matching (RPM) and local deformation using a radial basis function (RBF). The proposed RBF-based RPM algorithm solves the interlocking problems of correspondence and deformation using a deterministic annealing framework with fuzzy correspondence and RBF interpolation. Simulation tests showed promising results, with the average deviations decreasing by factors of 21.2 and 11.9, respectively. In the human model test, the average deviation decreased from 1.72±1.88mm to 0.57±0.66mm. We demonstrate the effectiveness of the proposed method by presenting some medical applications.

MRI-based strain and strain rate analysis of left ventricle: a modified hierarchical transformation model

BACKGROUND

Different from other indicators of cardiac function, such as ejection fraction and transmitral early diastolic velocity, myocardial strain is promising to capture subtle alterations that result from early diseases of the myocardium. In order to extract the left ventricle (LV) myocardial strain and strain rate from cardiac cine-MRI, a modified hierarchical transformation model was proposed.

METHODS

A hierarchical transformation model including the global and local LV deformations was employed to analyze the strain and strain rate of the left ventricle by cine-MRI image registration. The endocardial and epicardial contour information was introduced to enhance the registration accuracy by combining the original hierarchical algorithm with an Iterative Closest Points using Invariant Features algorithm. The hierarchical model was validated by a normal volunteer first and then applied to two clinical cases (i.e., the normal volunteer and a diabetic patient) to evaluate their respective function.

RESULTS

Based on the two clinical cases, by comparing the displacement fields of two selected landmarks in the normal volunteer, the proposed method showed a better performance than the original or unmodified model. Meanwhile, the comparison of the radial strain between the volunteer and patient demonstrated their apparent functional difference.

CONCLUSIONS

The present method could be used to estimate the LV myocardial strain and strain rate during a cardiac cycle and thus to quantify the analysis of the LV motion function.

Computing global minimizers to a constrained B-spline image registration problem from optimal l1 perturbations to block match data

PURPOSE

Block matching is a well-known strategy for estimating corresponding voxel locations between a pair of images according to an image similarity metric. Though robust to issues such as image noise and large magnitude voxel displacements, the estimated point matches are not guaranteed to be spatially accurate. However, the underlying optimization problem solved by the block matching procedure is similar in structure to the class of optimization problem associated with B-spline based registration methods. By exploiting this relationship, the authors derive a numerical method for computing a global minimizer to a constrained B-spline registration problem that incorporates the robustness of block matching with the global smoothness properties inherent to B-spline parameterization.

METHODS

The method reformulates the traditional B-spline registration problem as a basis pursuit problem describing the minimall1-perturbation to block match pairs required to produce a B-spline fitting error within a given tolerance. The sparsity pattern of the optimal perturbation then defines a voxel point cloud subset on which the B-spline fit is a global minimizer to a constrained variant of the B-spline registration problem. As opposed to traditional B-spline algorithms, the optimization step involving the actual image data is addressed by block matching.

RESULTS

The performance of the method is measured in terms of spatial accuracy using ten inhale/exhale thoracic CT image pairs (available for download atwww.dir-lab.com) obtained from the COPDgene dataset and corresponding sets of expert-determined landmark point pairs. The results of the validation procedure demonstrate that the method can achieve a high spatial accuracy on a significantly complex image set.

CONCLUSIONS

The proposed methodology is demonstrated to achieve a high spatial accuracy and is generalizable in that in can employ any displacement field parameterization described as a least squares fit to block match generated estimates. Thus, the framework allows for a wide range of image similarity block match metric and physical modeling combinations.

A Learning Based Fiducial-driven Registration Scheme for Evaluating Laser Ablation Changes in Neurological Disorders

In this work, we present a novel learning based fiducial driven registration (LeFiR) scheme which utilizes a point matching technique to identify the optimal configuration of landmarks to better recover deformation between a target and a moving image. Moreover, we employ the LeFiR scheme to model the localized nature of deformation introduced by a new treatment modality - laser induced interstitial thermal therapy (LITT) for treating neurological disorders. Magnetic resonance (MR) guided LITT has recently emerged as a minimally invasive alternative to craniotomy for local treatment of brain diseases (such as glioblastoma multiforme (GBM), epilepsy). However, LITT is currently only practised as an investigational procedure world-wide due to lack of data on longer term patient outcome following LITT. There is thus a need to quantitatively evaluate treatment related changes between post- and pre-LITT in terms of MR imaging markers. In order to validate LeFiR, we tested the scheme on a synthetic brain dataset (SBD) and in two real clinical scenarios for treating GBM and epilepsy with LITT. Four experiments under different deformation profiles simulating localized ablation effects of LITT on MRI were conducted on 286 pairs of SBD images. The training landmark configurations were obtained through 2000 iterations of registration where the points with consistently best registration performance were selected. The estimated landmarks greatly improved the quality metrics compared to a uniform grid (UniG) placement scheme, a speeded-up robust features (SURF) based method, and a scale-invariant feature transform (SIFT) based method as well as a generic free-form deformation (FFD) approach. The LeFiR method achieved average 90% improvement in recovering the local deformation compared to 82% for the uniform grid placement, 62% for the SURF based approach, and 16% for the generic FFD approach. On the real GBM and epilepsy data, the quantitative results showed that LeFiR outperformed UniG by 28% improvement in average.

Prostate CT segmentation method based on nonrigid registration in ultrasound-guided CT-based HDR prostate brachytherapy

PURPOSE

The technological advances in real-time ultrasound image guidance for high-dose-rate (HDR) prostate brachytherapy have placed this treatment modality at the forefront of innovation in cancer radiotherapy. Prostate HDR treatment often involves placing the HDR catheters (needles) into the prostate gland under the transrectal ultrasound (TRUS) guidance, then generating a radiation treatment plan based on CT prostate images, and subsequently delivering high dose of radiation through these catheters. The main challenge for this HDR procedure is to accurately segment the prostate volume in the CT images for the radiation treatment planning. In this study, the authors propose a novel approach that integrates the prostate volume from 3D TRUS images into the treatment planning CT images to provide an accurate prostate delineation for prostate HDR treatment.

METHODS

The authors' approach requires acquisition of 3D TRUS prostate images in the operating room right after the HDR catheters are inserted, which takes 1-3 min. These TRUS images are used to create prostate contours. The HDR catheters are reconstructed from the intraoperative TRUS and postoperative CT images, and subsequently used as landmarks for the TRUS-CT image fusion. After TRUS-CT fusion, the TRUS-based prostate volume is deformed to the CT images for treatment planning. This method was first validated with a prostate-phantom study. In addition, a pilot study of ten patients undergoing HDR prostate brachytherapy was conducted to test its clinical feasibility. The accuracy of their approach was assessed through the locations of three implanted fiducial (gold) markers, as well as T2-weighted MR prostate images of patients.

RESULTS

For the phantom study, the target registration error (TRE) of gold-markers was 0.41 ± 0.11 mm. For the ten patients, the TRE of gold markers was 1.18 ± 0.26 mm; the prostate volume difference between the authors' approach and the MRI-based volume was 7.28% ± 0.86%, and the prostate volume Dice overlap coefficient was 91.89% ± 1.19%.

CONCLUSIONS

The authors have developed a novel approach to improve prostate contour utilizing intraoperative TRUS-based prostate volume in the CT-based prostate HDR treatment planning, demonstrated its clinical feasibility, and validated its accuracy with MRIs. The proposed segmentation method would improve prostate delineations, enable accurate dose planning and treatment delivery, and potentially enhance the treatment outcome of prostate HDR brachytherapy.

Comparison of total variation with a motion estimation based compressed sensing approach for self-gated cardiac cine MRI in small animal studies

Purpose

Compressed sensing (CS) has been widely applied to prospective cardiac cine MRI. The aim of this work is to study the benefits obtained by including motion estimation in the CS framework for small-animal retrospective cardiac cine.

Methods

We propose a novel B-spline-based compressed sensing method (SPLICS) that includes motion estimation and generalizes previous spatiotemporal total variation (ST-TV) methods by taking into account motion between frames. In addition, we assess the effect of an optimum weighting between spatial and temporal sparsity to further improve results. Both methods were implemented using the efficient Split Bregman methodology and were evaluated on rat data comparing animals with myocardial infarction with controls for several acceleration factors.

Results

ST-TV with optimum selection of the weighting sparsity parameter led to results similar to those of SPLICS; ST-TV with large relative temporal sparsity led to temporal blurring effects. However, SPLICS always properly corrected temporal blurring, independently of the weighting parameter. At acceleration factors of 15, SPLICS did not distort temporal intensity information but led to some artefacts and slight over-smoothing. At an acceleration factor of 7, images were reconstructed without significant loss of quality.

Conclusion

We have validated SPLICS for retrospective cardiac cine in small animal, achieving high acceleration factors. In addition, we have shown that motion modelling may not be essential for retrospective cine and that similar results can be obtained by using ST-TV provided that an optimum selection of the spatiotemporal sparsity weighting parameter is performed.

Registration of in vivo prostate MRI and pseudo-whole mount histology using Local Affine Transformations guided by Internal Structures (LATIS)

PURPOSE

To present a novel registration approach called LATIS (Local Affine Transformation guided by Internal Structures) for coregistering post prostatectomy pseudo-whole mount (PWM) pathological sections with in vivo MRI (magnetic resonance imaging) images.

MATERIALS AND METHODS

Thirty-five patients with biopsy-proven prostate cancer were imaged at 3T with an endorectal coil. Excised prostate specimens underwent quarter mount step-section pathologic processing, digitization, annotation, and assembly into a PWM. Manually annotated macro-structures on both pathology and MRI were used to assist registration using a relaxed local affine transformation approximation. Registration accuracy was assessed by calculation of the Dice similarity coefficient (DSC) between transformed and target capsule masks and least-square distance between transformed and target landmark positions.

RESULTS

LATIS registration resulted in a DSC value of 0.991 ± 0.004 and registration accuracy of 1.54 ± 0.64 mm based on identified landmarks common to both datasets. Image registration performed without the use of internal structures led to an 87% increase in landmark-based registration error. Derived transformation matrices were used to map regions of pathologically defined disease to MRI.

CONCLUSION

LATIS was used to successfully coregister digital pathology with in vivo MRI to facilitate improved correlative studies between pathologically identified features of prostate cancer and multiparametric MRI.

Nonrigid point registration for 2D curves and 3D surfaces and its various applications

A nonrigid B-spline-based point-matching (BPM) method is proposed to match dense surface points. The method solves both the point correspondence and nonrigid transformation without features extraction. The registration method integrates a motion model, which combines a global transformation and a B-spline-based local deformation, into a robust point-matching framework. The point correspondence and deformable transformation are estimated simultaneously by fuzzy correspondence and by a deterministic annealing technique. Prior information about global translation, rotation and scaling is incorporated into the optimization. A local B-spline motion model decreases the degrees of freedom for optimization and thus enables the registration of a larger number of feature points. The performance of the BPM method has been demonstrated and validated using synthesized 2D and 3D data, mouse MRI and micro-CT images. The proposed BPM method can be used to register feature point sets, 2D curves, 3D surfaces and various image data.

Temporal sparse free-form deformations

FFD represent a widely used model for the non-rigid registration of medical images. The balance between robustness to noise and accuracy in modelling localised motion is typically controlled by the control point grid spacing and the amount of regularisation. More recently, TFFD have been proposed which extend the FFD approach in order to recover smooth motion from temporal image sequences. In this paper, we revisit the classic FFD approach and propose a sparse representation using the principles of compressed sensing. The sparse representation can model both global and local motion accurately and robustly. We view the registration as a deformation reconstruction problem. The deformation is reconstructed from a pair of images (or image sequences) with a sparsity constraint applied to the parametric space. Specifically, we introduce sparsity into the deformation via L1 regularisation, and apply a bending energy regularisation between neighbouring control points within each level to encourage a grouped sparse solution. We further extend the sparsity constraint to the temporal domain and propose a TSFFD which can capture fine local details such as motion discontinuities in both space and time without sacrificing robustness. We demonstrate the capabilities of the proposed framework to accurately estimate deformations in dynamic 2D and 3D image sequences. Compared to the classic FFD and TFFD approach, a significant increase in registration accuracy can be observed in natural images as well as in cardiac images.

Shape Representation and Registration in Vector Implicit Spaces: Adopting a Closed-Form Solution in the Optimization Process

In this paper, a novel method to solve the shape registration problem covering both global and local deformations is proposed. The vector distance function (VDF) is used to represent source and target shapes. The problem is formulated as an energy optimization process by matching the VDFs of the source and target shapes. The minimization process results in estimating the transformation parameters for the global and local deformation cases. Gradient descent optimization handles the computation of scaling, rotation, and translation matrices used to minimize the global differences between source and target shapes. Nonrigid deformations require a large number of parameters which make the use of the gradient descent minimization a very time-consuming process. We propose to compute the local deformation parameters using a closed-form solution as a linear system of equations derived from approximating an objective function. Extensive experimental validations and comparisons performed on generalized 2D shape data demonstrate the robustness and effectiveness of the method.

Serial section registration of axonal confocal microscopy datasets for long-range neural circuit reconstruction

In the context of long-range digital neural circuit reconstruction, this paper investigates an approach for registering axons across histological serial sections. Tracing distinctly labeled axons over large distances allows neuroscientists to study very explicit relationships between the brain's complex interconnects and, for example, diseases or aberrant development. Large scale histological analysis requires, however, that the tissue be cut into sections. In immunohistochemical studies thin sections are easily distorted due to the cutting, preparation, and slide mounting processes. In this work we target the registration of thin serial sections containing axons. Sections are first traced to extract axon centerlines, and these traces are used to define registration landmarks where they intersect section boundaries. The trace data also provides distinguishing information regarding an axon's size and orientation within a section. We propose the use of these features when pairing axons across sections in addition to utilizing the spatial relationships among the landmarks. The global rotation and translation of an unregistered section are accounted for using a random sample consensus (RANSAC) based technique. An iterative nonrigid refinement process using B-spline warping is then used to reconnect axons and produce the sought after connectivity information.

Automated approaches for analysis of multimodal MRI acquisitions in a study of cognitive aging

In this work we describe an integrated and automated workflow for a comprehensive and robust analysis of multimodal MR images from a cohort of more than hundred subjects. Image examinations are done three years apart and consist of 3D high-resolution anatomical images, low resolution tensor-valued DTI recordings and 4D resting state fMRI time series. The integrated analysis of the data requires robust tools for segmentation, registration and fiber tracking, which we combine in an automated manner. Our automated workflow is strongly desired due to the large number of subjects. Especially, we introduce the use of histogram segmentation to processed fMRI data to obtain functionally important seed and target regions for fiber tracking between them. This enables analysis of individually important resting state networks. We also discuss various approaches for the assessment of white matter integrity parameters along tracts, and in particular we introduce the use of functional data analysis (FDA) for this task.

Medical image registration: a review

This paper presents a review of automated image registration methodologies that have been used in the medical field. The aim of this paper is to be an introduction to the field, provide knowledge on the work that has been developed and to be a suitable reference for those who are looking for registration methods for a specific application. The registration methodologies under review are classified into intensity or feature based. The main steps of these methodologies, the common geometric transformations, the similarity measures and accuracy assessment techniques are introduced and described.

Characterizing the regional structural difference of the brain between tau transgenic (rTg4510) and wild-type mice using MRI

rTg4510 transgenic mouse model demonstrates features resembling Alzheimer's disease including neurofibrillary degeneration and progressive neuronal loss. We investigated the volumetric differences of brain structures between transgenic and wild-type mice using MR images of fourteen 5.5 month old female mice. Tensor-based morphometry and atlas-based segmentation were applied to MRI images. Severe atrophy of hippocampus and neocortex as well as ventricular dilatation were observed in the transgenic mice. These findings were confirmed by histopathologic evaluation of the same mice. The results suggest that MRI should be useful for evaluating disease-modifying therapies for Alzheimer's disease in the rTg4510 model and comparing treatment responses in mice and humans.

In vivo MRI assessment of knee cartilage in the medial meniscal tear model of osteoarthritis in rats

We present a new approach for quantifying the degradation of knee cartilage in the medial meniscal tear (MMT) model of osteoarthritis in the rat. A statistical strategy was used to guide the selection of a region of interest (ROI) from the images obtained from a pilot study. We hypothesize that this strategy can be used to localize a region of cartilage most vulnerable to MMT-induced damage. In order to test this hypothesis, a longitudinal study was conducted in which knee cartilage thickness in a pre-selected ROI was monitored for three weeks and comparisons were made between MMT and control rats. We observed a significant decrease in cartilage thickness in MMT rats and a significant increase in cartilage thickness in sham-operated rats as early as one week post surgery when compared to pre-surgery measurements.

Technical note: deformable image registration on partially matched images for radiotherapy applications

In radiation therapy applications, deformable image registrations (DIRs) are often carried out between two images that only partially match. Image mismatching could present as superior-inferior coverage differences, field-of-view (FOV) cutoffs, or motion crossing the image boundaries. In this study, the authors propose a method to improve the existing DIR algorithms so that DIR can be carried out in such situations. The basic idea is to extend the image volumes and define the extension voxels (outside the FOV or outside the original image volume) as NaN (not-a-number) values that are transparent to all floating-point computations in the DIR algorithms. Registrations are then carried out with one additional rule that NaN voxels can match any voxels. In this way, the matched sections of the images are registered properly, and the mismatched sections of the images are registered to NaN voxels. This method makes it possible to perform DIR on partially matched images that otherwise are difficult to register. It may also improve DIR accuracy, especially near or in the mismatched image regions.

Directly manipulated free-form deformation image registration

Previous contributions to both the research and open source software communities detailed a generalization of a fast scalar field fitting technique for cubic B-splines based on the work originally proposed by Lee . One advantage of our proposed generalized B-spline fitting approach is its immediate application to a class of nonrigid registration techniques frequently employed in medical image analysis. Specifically, these registration techniques fall under the rubric of free-form deformation (FFD) approaches in which the object to be registered is embedded within a B-spline object. The deformation of the B-spline object describes the transformation of the image registration solution. Representative of this class of techniques, and often cited within the relevant community, is the formulation of Rueckert who employed cubic splines with normalized mutual information to study breast deformation. Similar techniques from various groups provided incremental novelty in the form of disparate explicit regularization terms, as well as the employment of various image metrics and tailored optimization methods. For several algorithms, the underlying gradient-based optimization retained the essential characteristics of Rueckert's original contribution. The contribution which we provide in this paper is two-fold: 1) the observation that the generic FFD framework is intrinsically susceptible to problematic energy topographies and 2) that the standard gradient used in FFD image registration can be modified to a well-understood preconditioned form which substantially improves performance. This is demonstrated with theoretical discussion and comparative evaluation experimentation.

Registering histologic and MR images of prostate for image-based cancer detection

RATIONALE AND OBJECTIVES

Needle biopsy is currently the only way to confirm prostate cancer. To increase prostate cancer diagnostic rate, needles are expected to be deployed at suspicious cancer locations. High-contrast magnetic resonance (MR) imaging provides a powerful tool for detecting suspicious cancerous tissues. To do this, MR appearances of cancerous tissue should be characterized and learned from a sufficient number of prostate MR images with known cancer information. However, ground-truth cancer information is only available in histologic images. Therefore it is necessary to warp ground-truth cancerous regions in histological images to MR images by a registration procedure. The objective of this article is to develop a registration technique for aligning histological and MR images of the same prostate.

MATERIAL AND METHODS

Five pairs of histological and T2-weighted MR images of radical prostatectomy specimens are collected. For each pair, registration is guided by two sets of correspondences that can be reliably established on prostate boundaries and internal salient bloblike structures of histologic and MR images.

RESULTS

Our developed registration method can accurately register histologic and MR images. It yields results comparable to manual registration, in terms of landmark distance and volume overlap. It also outperforms both affine registration and boundary-guided registration methods.

CONCLUSIONS

We have developed a novel method for deformable registration of histologic and MR images of the same prostate. Besides the collection of ground-truth cancer information in MR images, the method has other potential applications. An automatic, accurate registration of histologic and MR images actually builds a bridge between in vivo anatomical information and ex vivo pathologic information, which is valuable for various clinical studies.

Brain functional localization: a survey of image registration techniques

Functional localization is a concept which involves the application of a sequence of geometrical and statistical image processing operations in order to define the location of brain activity or to produce functional/parametric maps with respect to the brain structure or anatomy. Considering that functional brain images do not normally convey detailed structural information and, thus, do not present an anatomically specific localization of functional activity, various image registration techniques are introduced in the literature for the purpose of mapping functional activity into an anatomical image or a brain atlas. The problems addressed by these techniques differ depending on the application and the type of analysis, i.e., single-subject versus group analysis. Functional to anatomical brain image registration is the core part of functional localization in most applications and is accompanied by intersubject and subject-to-atlas registration for group analysis studies. Cortical surface registration and automatic brain labeling are some of the other tools towards establishing a fully automatic functional localization procedure. While several previous survey papers have reviewed and classified general-purpose medical image registration techniques, this paper provides an overview of brain functional localization along with a survey and classification of the image registration techniques related to this problem.

A comparative study of transformation functions for nonrigid image registration

Transformation functions play a major role in nonrigid image registration. In this paper, the characteristics of thin-plate spline (TPS), multiquadric (MQ), piecewise linear (PL), and weighted mean (WM) transformations are explored and their performances in nonrigid image registration are compared. TPS and MQ are found to be most suitable when the set of control-point correspondences is not large (fewer than a thousand) and variation in spacing between the control points is not large. When spacing between the control points varies greatly, PL is found to produce a more accurate registration than TPS and MQ. When a very large set of control points is given and the control points contain positional inaccuracies, WM is preferred over TPS, MQ, and PL because it uses an averaging process that smoothes the noise and does not require the solution of a very large system of equations. Use of transformation functions in the detection of incorrect correspondences is also discussed.

Regional structural characterization of the brain of schizophrenia patients

RATIONALE AND OBJECTIVES

We study morphologic characteristics and age-related changes in patients with schizophrenia to investigate whether abnormal neurodevelopment and brain structure have a role in the pathophysiological course of this disease.

MATERIALS AND METHODS

Our data consist of a set of cranial magnetic resonance images of 46 patients with schizophrenia and age- and sex-matched healthy controls. We deformed a template brain image to our set of subject images. Jacobian fields of these deformations were reduced to sets of 52 normalized region volumes for each subject by using a neuroanatomic atlas. Normalized regional volumes of the control and patient groups were compared by using Student t-test, and age correlation of each region volume was calculated for the two groups. All results were corrected for multiple comparisons by using permutation testing. We used a classifier based on support vector machines and a feature selection method to determine our ability to discriminate brains of controls from those of patients.

RESULTS

Analysis of normalized region volumes shows enlargement of the third ventricle in patients. The age-correlation study showed a significant positive correlation in the third ventricle and right thalamus of controls, but not patients. Using an average of 6.5 features, our classifier was able to correctly identify 72% of patients and 70% of controls.

CONCLUSION

In addition to enlargement of the third ventricle, brains of patients with schizophrenia show a different pattern of age-related changes.

Capturing anatomical shape variability using B-spline registration

Registration based on B-spline transformations has attracted much attention in medical image processing recently. Non-rigid registration provides the basis for many important techniques, such as statistical shape modeling. Validating the results, however, remains difficult--especially in intersubject registration. This work explores the ability of B-spline registration methods to capture intersubject shape deformations. We study the effect of different established and new shape representations, similarity measures and optimization strategies on the matching quality. To this end we conduct experiments on synthetic shapes representing deformations which typically may arise in intersubject registration, as well as on real patient data of the liver and pelvic bone. The experiments clearly reveal the influence of each component on the registration performance. The results may serve as a guideline for assessing intensity based registration.