Shape-based interpolation of multidimensional grey-level images

Image Registration Based on Lifting Process: An Application to Digital Subtraction Radiography

In this paper, a digital subtraction radiology scheme is presented based on a new method for the automatic registration of dental radiographs acquired with or without rigorous a priori standardization. The scheme is comprised of an automatic registration method and a subtraction process. The proposed registration method can be considered as an object-based registration method without imposing the prerequisite of image segmentation in order to detect the boundary of the objects of interest or the automatic detection of matching landmarks. This is achieved by augmenting the dimensionality of the problem from two-dimensional gray-level matching to three-dimensional surface matching using the process of lifting in combination with a surface-matching technique. The pseudo three-dimensional affine transformation that matches the lifted images incorporates advantageous characteristics including spatial alignment of the surfaces, anisotropic correction of brightness/contrast differences, and stable convergence of the similarity function to its optimal value. The performance of the proposed automatic registration method is assessed against a manual method based on the projective transformation. The qualitative and quantitative assessments of the experiments have shown advantageous performance of the proposed automatic registration method against the manual one. Finally, the proposed registration method has been further improved in terms of execution time by the implementation of a surface decimation process.

Shape-based averaging for combination of multiple segmentations

Combination of multiple segmentations has recently been introduced as an effective method to obtain segmentations that are more accurate than any of the individual input segmentations. This paper introduces a new way to combine multiple segmentations using a novel shape-based averaging method. Individual segmentations are combined based on the signed Euclidean distance maps of the labels in each input segmentation. Compared to label voting, the new combination method produces smoother, more regular output segmentations and avoids fragmentation of contiguous structures. Using publicly available segmented human brain MR images (IBSR database), we perform a quantitative comparison between shape-based averaging and label voting by combining random segmentations with controlled error magnitudes and known ground truth. Shape-based averaging generated combined segmentations that were closer to the ground truth than combinations from label voting for all numbers of input segmentations (up to ten). The relative advantage of shape-based averaging over voting was larger for fewer input segmentations, and larger for greater deviations of the input segmentations from the ground truth. We conclude that shape-based averaging improves the accuracy of combined segmentations, in particular when only a few input segmentations are available and when the quality of the input segmentations is low.

SPASM: A 3D-ASM for segmentation of sparse and arbitrarily oriented cardiac MRI data

A new technique (SPASM) based on a 3D-ASM is presented for automatic segmentation of cardiac MRI image data sets consisting of multiple planes with arbitrary orientations, and with large undersampled regions. Model landmark positions are updated in a two-stage iterative process. First, landmark positions close to intersections with images are updated. Second, the update information is propagated to the regions without image information, such that new locations for the whole set of the model landmarks are obtained. Feature point detection is performed by a fuzzy inference system, based on fuzzy C-means clustering. Model parameters were optimized on a computer cluster and the computational load distributed by grid computing. SPASM was applied to image data sets with an increasing sparsity (from 2 to 11 slices) comprising images with different orientations and stemming from different MRI acquisition protocols. Segmentation outcomes and calculated volumes were compared to manual segmentation on a dense short-axis data configuration in a 3D manner. For all data configurations, (sub-)pixel accuracy was achieved. Performance differences between data configurations were significantly different (p<0.05) for SA data sets with less than 6 slices, but not clinically relevant (volume differences<4 ml). Comparison to results from other 3D model-based methods showed that SPASM performs comparable to or better than these other methods, but SPASM uses considerably less image data. Sensitivity to initial model placement proved to be limited within a range of position perturbations of approximately 20 mm in all directions.

Partial volume reduction by interpolation with reverse diffusion

Many medical images suffer from the partial volume effect where a boundary between two structures of interest falls in the midst of a voxel giving a signal value that is a mixture of the two. We propose a method to restore the ideal boundary by splitting a voxel into subvoxels and reapportioning the signal into the subvoxels. Each voxel is divided by nearest neighbor interpolation. The gray level of each subvoxel is considered as “material” able to move between subvoxels but not between voxels. A partial differential equation is written to allow the material to flow towards the highest gradient direction, creating a “reverse” diffusion process. Flow is subject to constraints that tend to create step edges. Material is conserved in the process thereby conserving signal. The method proceeds until the flow decreases to a low value. To test the method, synthetic images were downsampled to simulate the partial volume artifact and restored. Corrected images were remarkably closer both visually and quantitatively to the original images than those obtained from common interpolation methods: on simulated data standard deviation of the errors were 3.8%, 6.6%, and 7.1% of the dynamic range for the proposed method, bicubic, and bilinear interpolation, respectively. The method was relatively insensitive to noise. On gray level, scanned text, MRI physical phantom, and brain images, restored images processed with the new method were visually much closer to high-resolution counterparts than those obtained with common interpolation methods.

Morphometrical study of plant vacuolar dynamics in single cells using three-dimensional reconstruction from optical sections

In higher plants, vacuoles increase their volumes in accordance with cell enlargement and occupy most of the cell volume. However, quantitative analyses of vacuolar contributions during changes in cell morphology have been hampered by the inadequacies and frequent artifacts associated with current three-dimensional (3-D) reconstruction methods of images derived from light microscopy. To overcome the limitations of quantifying 3-D structures, we have introduced 3-D morphometrics into light microscopy, adopting a contour-based approach for which we have developed an interpolation method. Using this software, named REANT, the morphological and morphometrical changes in protoplasts and vacuoles during plasmolysis could be investigated. We employed the tobacco (Nicotiana tabacum) BY-2 cell line No.7, expressing a GFP-AtVam3p fusion protein, BY-GV7, using GFP as a marker of vacuolar membranes (VMs). By vital staining of the plasma membrane (PM) of cells, we simultaneously obtained optical sections of both the PM and VM. We, therefore, reconstructed the 3-D structures of protoplasts and vacuoles before and after plasmolysis. We were able to identify the appearance of elliptical structures of VMs in the vacuolar lumen, and to determine that they were derived from cytoplasmic strands. From the 3-D structures, the volumes and surface areas were measured at the single cell level. The shrinkage of vacuoles accounted for most of the decrease in protoplast volume, while the surface area of the vacuoles remained mostly unchanged. These morphometrical analyses suggest that the elliptical structures are reservoirs for excess VMs that result from the response to rapid decreases in vacuolar and protoplast volumes.

Correction of motion artifacts from cardiac cine magnetic resonance images

RATIONALE AND OBJECTIVES

An image registration method was developed to automatically correct motion artifacts, mostly from breathing, from cardiac cine magnetic resonance (MR) images.

MATERIALS AND METHODS

The location of each slice in an image stack was optimized by maximizing a similarity measure of the slice with another image slice stack. The optimization was performed iteratively and both image stacks were corrected simultaneously. Two procedures to optimize the similarity were tested: standard gradient optimization and stochastic optimization in which one slice is chosen randomly from the image stacks and its location is optimized. In this work, cine short- and long-axis images were used. In addition to visual inspection results from real data, the performance of the algorithm was evaluated quantitatively by simulating the movements in four real MR data sets. The mean error and standard deviation were defined for 50 simulated movements as each slice was randomly displaced. The error rate, defined as the percentage of non-satisfactory registration results, was evaluated. The paired t-test was used to evaluate the statistical difference between the tested optimization methods.

RESULTS

The algorithm developed was successfully applied to correct motion artifacts from real and simulated data. The results, where typical motion artifacts were simulated, indicated an error rate of about 3%. Subvoxel registration accuracy was also achieved. When different optimization methods were compared, the registration accuracy of the stochastic approach proved to be superior to the standard gradient technique (P < 10(-9)).

CONCLUSIONS

The novel method was capable of robustly and accurately correcting motion artifacts from cardiac cine MR images.

Three-dimensional velocity field reconstruction

The problem of inter-slice magnetic resonance (MR) image reconstruction is encountered often in medical imaging applications, in such scenarios, there is a need to approximate information not captured in contiguously acquired MR images due to hardware sampling limitations. In the context of velocity field reconstruction, these data are required for visualization and computational analyses of flow fields to be effective. To provide more complete velocity information, a method has been developed for the reconstruction of flow fields based on adaptive control grid interpolation (ACGI). In this study, data for reconstruction were acquired via MRJ from in vitro models of surgically corrected pediatric cardiac vasculatures. Reconstructed velocity fields showed strong qualitative agreement with those obtained via other acquisition techniques. Quantitatively reconstruction was shown to produce data of comparable quality to accepted velocity data acquisition methods. Results indicate that ACGI-based velocity field reconstruction is capable of producing information suitable for a variety of applications demanding three-dimensional in vivo velocity data.

3D long bone reconstruction based on level sets

In medical imaging a three-dimensional (3D) object must often be reconstructed from serial cross-sections to aid in the comprehension of the object's structure as well as to facilitate its automatic manipulation and analysis. The most popular interpolation scheme for a sequence of image slices is the shape-based method, where object information extracted from a given 3D volume image is used in guiding the interpolation process. The paper presents a level set reformulation of the well-known shape-based method as well as a new automatic level set method, which offers better performance. In particular, we focus on X-ray examinations of long bones, which also requires us to deal with the problem of an optimal slice positioning. To this aim, a 2D version of the proposed algorithm will be used to localize a subset of slices from the entire volume image. A number of experiments were performed on computed tomographic real images to evaluate the proposed approach. The experimental results show a substantial improvement of visual effects (qualitative evaluation) using the proposed method in comparison to both the conventional gray-level interpolation scheme and the shape-based method. Compared with the shape-based interpolation scheme the proposed method has much lower computational cost.

Registration-based interpolation

A method is presented to interpolate between neighboring slices in a grey-scale tomographic data set. Spatial correspondence between adjacent slices is established using a nonrigid registration algorithm based on B-splines which optimizes the normalized mutual information similarity measure. Linear interpolation of the image intensities is then carried out along the directions calculated by the registration algorithm. The registration-based method is compared to both standard linear interpolation and shape-based interpolation in 20 tomographic data sets. Results show that the proposed method statistically significantly outperforms both linear and shape-based interpolation.

A 3-D model-based registration approach for the PET, MR and MCG cardiac data fusion

In this paper, a new approach is presented for the assessment of a 3-D anatomical and functional model of the heart including structural information from magnetic resonance imaging (MRI) and functional information from positron emission tomography (PET) and magnetocardiography (MCG). The method uses model-based co-registration of MR and PET images and marker-based registration for MRI and MCG. Model-based segmentation of MR anatomical images results in an individualized 3-D biventricular model of the heart including functional parameters from PET and MCG in an easily interpretable 3-D form.

Application of an adaptive control grid interpolation technique to morphological vascular reconstruction

The problem of interslice magnetic resonance (MR) image reconstruction arises in a broad range of medical applications. In such cases, there is a need to approximate information present in the original subject that is not reflected in contiguously acquired MR images because of hardware sampling limitations. In the context of vascular morphology reconstruction, this information is required in order for subsequent visualization and computational analysis of blood vessels to be most effective. Toward that end we have developed a method of vascular morphology reconstruction based on adaptive control grid interpolation (ACGI) to function as a precursor to visualization and computational analysis. ACGI has previously been implemented in addressing various problems including video coding and tracking. This paper focuses on the novel application of the technique to medical image processing. ACGI combines features of optical flow-based and block-based motion estimation algorithms to enhance insufficiently dense MR data sets accurately with a minimal degree of computational complexity. The resulting enhanced data sets describe vascular geometries. These reconstructions can then be used as visualization tools and in conjunction with computational fluid dynamics (CFD) simulations to offer the pressure and velocity information necessary to quantify power loss. The proposed ACGI methodology is envisioned ultimately to play a role in surgical planning aimed at producing optimal vascular configurations for successful surgical outcomes.

Feature-guided shape-based image interpolation

A feature-guided image interpolation scheme is presented. It is an effective and improved, shape-based interpolation method used for interpolating image slices in medical applications. The proposed method integrates feature line-segments to guide the shape-based method for better shape interpolation. An automatic method for finding these line segments is given. The proposed feature-guided shape-based method can manage translation, rotation and scaling situations when the slices have similar shapes. It can also interpolate intermediate shapes when the successive slices do not have similar shapes. This method is experimentally evaluated using artificial and real two-dimensional and three-dimensional data. The proposed method generated satisfactory interpolated results in these experiments. We demonstrate the practicality, effectiveness and reproducibility of the proposed method for interpolating medical images.

Binary morphological shape-based interpolation applied to 3-D tooth reconstruction

In this paper, we propose an interpolation algorithm using a mathematical morphology morphing approach. The aim of this algorithm is to reconstruct the n-dimensional object from a group of (n - 1)-dimensional sets representing sections of that object. The morphing transformation modifies pairs of consecutive sets such that they approach in shape and size. The interpolated set is achieved when the two consecutive sets are made idempotent by the morphing transformation. We prove the convergence of the morphological morphing. The entire object is modeled by successively interpolating a certain number of intermediary sets between each two consecutive given sets. We apply the interpolation algorithm for three-dimensional tooth reconstruction.

Quantitative evaluation of convolution-based methods for medical image interpolation

Interpolation is required in a variety of medical image processing applications. Although many interpolation techniques are known from the literature, evaluations of these techniques for the specific task of applying geometrical transformations to medical images are still lacking. In this paper we present such an evaluation. We consider convolution-based interpolation methods and rigid transformations (rotations and translations). A large number of sinc-approximating kernels are evaluated, including piecewise polynomial kernels and a large number of windowed sinc kernels, with spatial supports ranging from two to ten grid intervals. In the evaluation we use images from a wide variety of medical image modalities. The results show that spline interpolation is to be preferred over all other methods, both for its accuracy and its relatively low computational cost.

Morphology-based three-dimensional interpolation

In many medical applications, the number of available two-dimensional (2-D) images is always insufficient. Therefore, the three-dimensional (3-D) reconstruction must be accomplished by appropriate interpolation methods to fill gaps between available image slices. In this paper, we propose a morphology-based algorithm to interpolate the missing data. The proposed algorithm consists of several steps. First, the object or hole contours are extracted using conventional image-processing techniques. Second, the object or hole matching issue is evaluated. Prior to interpolation, the centroids of the objects are aligned. Next, we employ a dilation operator to transform digital images into distance maps and we correct the distance maps if required. Finally, we utilize an erosion operator to accomplish the interpolation. Furthermore, if multiple objects or holes are interpolated, we blend them together to complete the algorithm. We experimentally evaluate the proposed method against various synthesized cases reported in the literature. Experimental results show that the proposed method is able to handle general object interpolation effectively.

Interpolation of 3-D binary images based on morphological skeletonization

In this paper, the morphological skeleton interpolation (MSI) algorithm is presented. It is an efficient, shape-based interpolation method used for interpolating slices in a three-dimensional (3-D) binary object. It is based on morphological skeletonization, which is used for two-dimensional (2-D) slice representation. The proposed morphological skeleton matching process provides translation, rotation, and scaling information at the same time. The interpolated slices preserve the shape of the original object slices, when the slices have similar shapes. It can also modify the shape of an object when the successive slices do not have similar shapes. Applications on artificial and real data are also presented.

Subunity coordinate translation with Fourier transform to achieve efficient and quality three-dimensional medical image interpolation

A new approach to the interpolation of three-dimensional (3D) medical images is presented. Instead of going through the conventional interpolation scheme where the continuous function is first reconstructed from the discrete data set and then resampled, the interpolation is achieved with a subunity coordinate translation technique. The original image is first transformed into the spatial-frequency domain. The phase of the transform is then modified with n-1 linear phase terms in the axial direction to achieve n-1 subunity coordinate translations with a distance 1/n, where n is an interpolation ratio, following the phase shift theorem of Fourier transformation. All the translated images after inverse Fourier transformation are then interspersed in turn into the original image. Since windowing plays an important role in the process, different window functions have been studied and a proper recommendation is provided. The interpolation quality produced with the present method is as good as that with the sampling (sinc) function, while the efficiency, thanks to the fast Fourier transformation, is very much improved. The approach has been validated with both computed tomography (CT) and magnetic resonance (MR) images. The interpolations of 3D CT and MR images are demonstrated.

Fast surface and volume estimation from non-parallel cross-sections, for freehand three-dimensional ultrasound

Volume measurements from ultrasound B-scans are useful in many clinical areas. It has been demonstrated previously that using three-dimensional (3-D) ultrasound can greatly increase the accuracy of these measurements. Freehand 3-D ultrasound allows freedom of movement in scanning, but the processing is complicated by having non-parallel scan planes. Two techniques are proposed for volume measurement from such data, which also improve surface and volume estimation from data acquired on parallel planes. Cubic planimetry is a more accurate extension of a volume measurement technique involving vector areas and centroids of cross-sections. Maximal-disc shape-based interpolation is an extension of shape-based interpolation which uses maximal disc representations to adjust the interpolation direction locally and hence improve the quality of the surface generated. Both methods are tested in simulation and in vivo. Volumes estimated using cubic planimetry are more accurate than step-section planimetry, and require fewer cross-sections, even for complex objects. Maximal-disc shape-based interpolation provides a reliable means of reconstructing surfaces from a handful of cross-sections, and can therefore be used to give confidence in the segmentation and hence also the cubic planimetry volume.

Shape-based grey-level image interpolation

The three-dimensional (3D) object data obtained from a CT scanner usually have unequal sampling frequencies in the x-, y- and z-directions. Generally, the 3D data are first interpolated between slices to obtain isotropic resolution, reconstructed, then operated on using object extraction and display algorithms. The traditional grey-level interpolation introduces a layer of intermediate substance and is not suitable for objects that are very different from the opposite background. The shape-based interpolation method transfers a pixel location to a parameter related to the object shape and the interpolation is performed on that parameter. This process is able to achieve a better interpolation but its application is limited to binary images only. In this paper, we present an improved shape-based interpolation method for grey-level images. The new method uses a polygon to approximate the object shape and performs the interpolation using polygon vertices as references. The binary images representing the shape of the object were first generated via image segmentation on the source images. The target object binary image was then created using regular shape-based interpolation. The polygon enclosing the object for each slice can be generated from the shape of that slice. We determined the relative location in the source slices of each pixel inside the target polygon using the vertices of a polygon as the reference. The target slice grey-level was interpolated from the corresponding source image pixels. The image quality of this interpolation method is better and the mean squared difference is smaller than with traditional grey-level interpolation.

Three-dimensional visualization and analysis methodologies: a current perspective

Three-dimensional (3D) imaging was developed to provide both qualitative and quantitative information about an object or object system from images obtained with multiple modalities including digital radiography, computed tomography, magnetic resonance imaging, positron emission tomography, single photon emission computed tomography, and ultrasonography. Three-dimensional imaging operations may be classified under four basic headings: preprocessing, visualization, manipulation, and analysis. Preprocessing operations (volume of interest, filtering, interpolation, registration, segmentation) are aimed at extracting or improving the extraction of object information in given images. Visualization operations facilitate seeing and comprehending objects in their full dimensionality and may be either scene-based or object-based. Manipulation may be either rigid or deformable and allows alteration of object structures and of relationships between objects. Analysis operations, like visualization operations, may be either scene-based or object-based and deal with methods of quantifying object information. There are many challenges involving matters of precision, accuracy, and efficiency in 3D imaging. Nevertheless, 3D imaging is an exciting technology that promises to offer an expanding number and variety of applications.

A task-specific evaluation of three-dimensional image interpolation techniques

Image interpolation is an important operation that is widely used in medical imaging, image processing, and computer graphics. A variety of interpolation methods are available in the literature. However, their systematic evaluation is lacking. In a previous paper, we presented a framework for the task-independent comparison of interpolation methods based on certain image-derived figures of merit using a variety of medical image data pertaining to different parts of the human body taken from different modalities. In this work, we present an objective task-specific framework for evaluating interpolation techniques. The task considered is how the interpolation methods influence the accuracy of quantification of the total volume of lesions in the brain of multiple sclerosis (MS) patients. Sixty lesion-detection experiments coming from ten patient studies, two subsampling techniques and the original data, and three interpolation methods are carried out, along with a statistical analysis of the results.

An objective comparison of 3-D image interpolation methods

To aid in the display, manipulation, and analysis of biomedical image data, they usually need to he converted to data of isotropic discretization through the process of interpolation. Traditional techniques consist of direct interpolation of the grey values. When user interaction is called for in image segmentation, as a consequence of these interpolation methods, the user needs to segment a much greater (typically 4-10x) amount of data. To mitigate this problem, a method called shape-based interpolation of binary data was developed 121. Besides significantly reducing user time, this method has been shown to provide more accurate results than grey-level interpolation. We proposed an approach for the interpolation of grey data of arbitrary dimensionality that generalized the shape-based method from binary to grey data. This method has characteristics similar to those of the binary shape-based method. In particular, we showed preliminary evidence that it produced more accurate results than conventional grey-level interpolation methods. In this paper, concentrating on the three-dimensional (3-D) interpolation problem, we compare statistically the accuracy of eight different methods: nearest-neighbor, linear grey-level, grey-level cubic spline, grey-level modified cubic spline, Goshtasby et al., and three methods from the grey-level shape-based class. A population of patient magnetic resonance and computed tomography images, corresponding to different parts of the human anatomy, coming from different three-dimensional imaging applications, are utilized for comparison. Each slice in these data sets is estimated by each interpolation method and compared to the original slice at the same location using three measures: mean-squared difference, number of sites of disagreement, and largest difference. The methods are statistically compared pairwise based on these measures. The shape-based methods statistically significantly outperformed all other methods in all measures in all applications considered here with a statistical relevance ranging from 10% to 32% (mean = 15%) for mean-squared difference.

Nonlinear filtering approach to 3-D gray-scale image interpolation

Three-dimensional (3-D) images are now common in radiology. A 3-D image is formed by stacking a contiguous sequence of two-dimensional cross-sectional images, or slices. Typically, the spacing between known slices is greater than the spacing between known points on a slice. Many visualization and image-analysis tasks, however, require the 3-D image to have equal sample spacing in all directions. To meet this requirement, one applies an interpolation technique to the known 3-D image to generate a new uniformly sampled 3-D image. The authors propose a nonlinear-filter-based approach to gray-scale interpolation of 3-D images. The method, referred to as column-fitting interpolation, is reminiscent of the maximum-homogeneity filter used for image enhancement. The authors also draw upon the paradigm of relaxation labeling to devise an improved column-fitting interpolator. Both methods are typically more effective than traditional gray-scale interpolation techniques.