Registration-based interpolation

An Efficient Muscle Segmentation Method via Bayesian Fusion of Probabilistic Shape Modeling and Deep Edge Detection

OBJECTIVE

Paraspinal muscle segmentation and reconstruction from MR images are critical to implement quantitative assessment of chronic and recurrent low back pains. Due to unclear muscle boundaries and shape variations, current segmentation methods demonstrate suboptimal performance with insufficient training samples. This work proposes a novel approach to modeling and inferring muscle shapes that enhances segmentation accuracy and efficiency with few training data.

METHODS

Firstly, a probabilistic shape model (PSM) based on Fourier basis functions and Gaussian processes (GPs) is designed to encode 3D muscle shapes, where anatomical meanings are attributed to the model's geometric parameters. Muscle shape variations and correlations are described by the GPs of the geometric parameters, which allow a small size of parameters to model the distribution of muscle shapes. Secondly, a Bayesian framework is developed to achieve entire muscle segmentation by posterior estimations. The framework fuses the geometric prior of the PSM with observations of deep-learning-based edge detections (DED) and sparse manual annotations, by which issues of unclear boundaries and shape variations can be compensated.

RESULTS AND CONCLUSION

Experiments on public and clinical datasets demonstrate that, with just three manually annotated slices, our method achieves a Dice similarity coefficient exceeding 90%, which outperforms other methods. Meanwhile, our method needs only a small training dataset and offers rapid inference speeds in clinical applications.

SIGNIFICANCE

Our study enables precise assessment of paraspinal muscles in 2D and 3D, aiding clinicians and researchers in understanding muscle changes in various conditions, potentially enhancing treatment outcomes.

Beyond automatic medical image segmentation-the spectrum between fully manual and fully automatic delineation

Semi-automatic and fully automatic contouring tools have emerged as an alternative to fully manual segmentation to reduce time spent contouring and to increase contour quality and consistency. Particularly, fully automatic segmentation has seen exceptional improvements through the use of deep learning in recent years. These fully automatic methods may not require user interactions, but the resulting contours are often not suitable to be used in clinical practice without a review by the clinician. Furthermore, they need large amounts of labelled data to be available for training. This review presents alternatives to manual or fully automatic segmentation methods along the spectrum of variable user interactivity and data availability. The challenge lies to determine how much user interaction is necessary and how this user interaction can be used most effectively. While deep learning is already widely used for fully automatic tools, interactive methods are just at the starting point to be transformed by it. Interaction between clinician and machine, via artificial intelligence, can go both ways and this review will present the avenues that are being pursued to improve medical image segmentation.

Sparse self-attention aggregation networks for neural sequence slice interpolation

Background

Microscopic imaging is a crucial technology for visualizing neural and tissue structures. Large-area defects inevitably occur during the imaging process of electron microscope (EM) serial slices, which lead to reduced registration and semantic segmentation, and affect the accuracy of 3D reconstruction. The continuity of biological tissue among serial EM images makes it possible to recover missing tissues utilizing inter-slice interpolation. However, large deformation, noise, and blur among EM images remain the task challenging. Existing flow-based and kernel-based methods have to perform frame interpolation on images with little noise and low blur. They also cannot effectively deal with large deformations on EM images.

Results

In this paper, we propose a sparse self-attention aggregation network to synthesize pixels following the continuity of biological tissue. First, we develop an attention-aware layer for consecutive EM images interpolation that implicitly adopts global perceptual deformation. Second, we present an adaptive style-balance loss taking the style differences of serial EM images such as blur and noise into consideration. Guided by the attention-aware module, adaptively synthesizing each pixel aggregated from the global domain further improves the performance of pixel synthesis. Quantitative and qualitative experiments show that the proposed method is superior to the state-of-the-art approaches.

Conclusions

The proposed method can be considered as an effective strategy to model the relationship between each pixel and other pixels from the global domain. This approach improves the algorithm’s robustness to noise and large deformation, and can accurately predict the effective information of the missing region, which will greatly promote the data analysis of neurobiological research.

Volumetric interpolation of tomographic sequences for accurate 3D reconstruction of anatomical parts

BACKGROUND AND OBJECTIVE

Tomographic sequences of biomedical images are commonly used to achieve a three-dimensional visualization of the human anatomy. In some cases, the number of images contained in the sequence is limited, e.g., in low-dose computed tomography acquired on neonatal patients, resulting in a coarse and inaccurate 3D reconstruction.

METHODS

In this paper, volumetric image interpolation methods, devised to increase the axial resolution of tomographic sequences and achieve a refined 3D reconstruction, are proposed and compared. The techniques taken into consideration are based on motion-compensated frame-interpolation concepts, which have been developed for video applications, mainly frame-rate conversion.

RESULTS

The performance of the proposed methods is quantitatively assessed by using sequences with a simulated low axial resolution obtained from the decimation of standard high-resolution computed tomography sequences. Real data with an actual low axial resolution have been used as well for a qualitative evaluation of the proposed methods.

CONCLUSIONS

The experimental results demonstrate that the proposed methods enable an effective slice interpolation and that the achievable 3D models clearly benefit from the increased axial resolution.

3D Visualization of Brain Tumors Using MR Images: A Survey

BACKGROUND

Three-Dimensional visualization of brain tumors is very useful in both diagnosis and treatment stages of brain cancer.

DISCUSSION

It helps the oncologist/neurosurgeon to take the best decision in Radiotherapy and/or surgical resection techniques. 3D visualization involves two main steps; tumor segmentation and 3D modeling.

CONCLUSION

In this article, we illustrate the most widely used segmentation and 3D modeling techniques for brain tumors visualization. We also survey the public databases available for evaluation of the mentioned techniques.

A Study on Robustness of Various Deformable Image Registration Algorithms on Image Reconstruction Using 4DCT Thoracic Images

Background

Medical image interpolation is recently introduced as a helpful tool to obtain further information via initial available images taken by tomography systems. To do this, deformable image registration algorithms are mainly utilized to perform image interpolation using tomography images.

Materials and Methods

In this work, 4DCT thoracic images of five real patients provided by DIR-lab group were utilized. Four implemented registration algorithms as 1) Original Horn-Schunck, 2) Inverse consistent Horn-Schunck, 3) Original Demons and 4) Fast Demons were implemented by means of DIRART software packages. Then, the calculated vector fields are processed to reconstruct 4DCT images at any desired time using optical flow based on interpolation method. As a comparative study, the accuracy of interpolated image obtained by each strategy is measured by calculating mean square error between the interpolated image and real middle image as ground truth dataset.

Results

Final results represent the ability to accomplish image interpolation among given two-paired images. Among them, Inverse Consistent Horn-Schunck algorithm has the best performance to reconstruct interpolated image with the highest accuracy while Demons method had the worst performance.

Conclusion

Since image interpolation is affected by increasing the distance between two given available images, the performance accuracy of four different registration algorithms is investigated concerning this issue. As a result, Inverse Consistent Horn-Schunck does not essentially have the best performance especially in facing large displacements happened due to distance increment.

A Fast Segmentation and Efficient Slice Reconstruction Technique for Head CT Images

The three-dimensional (3D) reconstruction of medical images usually requires hundreds of two-dimensional (2D) scan images. Segmentation, an obligatory part in reconstruction, needs to be performed for all the slices consuming enormous storage space and time. To reduce storage space and time, this paper proposes a three-stage procedure, namely, slice selection, segmentation and interpolation. The methodology will have the potential to 3D reconstruct the human head from minimum selected slices. The first stage of slice selection is based on structural similarity measurement, discarding the most similar slices with none or minimal impact on details. The second stage of segmentation of the selected slices is performed using our proposed phase-field segmentation method. Validation of our segmentation results is done via comparison with other deformable models, and results show that the proposed method provides fast and accurate segmentation. The third stage of interpolation is based on modified curvature registration-based interpolation, and it is applied to re-create the discarded slices. This method is compared to both standard linear interpolation and registration-based interpolation in 100 tomographic data sets. Results show that the modified curvature registration-based interpolation reconstructs missing slices with 96% accuracy and shows an improvement in sensitivity (95.802%) on par with specificity (95.901%).

Slice interpolation of medical images using enhanced fuzzy radial basis function neural networks

Volume data composed of complete slice images play an indispensable role in medical diagnoses. However, system or human factors often lead to the loss of slice images. In recent years, various interpolation algorithms have been proposed to solve these problems. Although these algorithms are effective, the interpolated images have some shortcomings, such as less accurate recovery and missing details. In this study, we propose a new method based on an enhanced fuzzy radial basis function neural network to improve the performance of the interpolation method. The neural network includes an input layer (six input neurons), three hidden layers of neurons, and the output layer (one output neuron), and we propose a patch matching method to select the input variables of the neural network. Accordingly, we use two normal pending images to be interpolated as the input. Final output data is obtained by applying the trained neural network. In examining four groups of medical images, the proposed method outperforms five other methods, achieving the highest similarity image metric (ESSIM) values of 0.96, 0.95, 0.94, and 0.92 and the lowest mean squared difference (MSD) values of 35.5, 41.2, 50.9, and 47.1. In addition, for a whole MRI brain volume data experiment, the average MSD and ESSIM values of the proposed method and other methods are (41.62, 0.95) and (57.13, 0.90), respectively. The results indicate that the proposed method is superior to the other methods.

An Optical Flow-Based Approach for Minimally Divergent Velocimetry Data Interpolation

Three-dimensional (3D) biomedical image sets are often acquired with in-plane pixel spacings that are far less than the out-of-plane spacings between images. The resultant anisotropy, which can be detrimental in many applications, can be decreased using image interpolation. Optical flow and/or other registration-based interpolators have proven useful in such interpolation roles in the past. When acquired images are comprised of signals that describe the flow velocity of fluids, additional information is available to guide the interpolation process. In this paper, we present an optical-flow based framework for image interpolation that also minimizes resultant divergence in the interpolated data.

Reducing Navigators in Free-Breathing Abdominal MRI via Temporal Interpolation Using Convolutional Neural Networks

Navigated 2-D multi-slice dynamic magnetic resonance imaging (MRI) acquisitions are essential for MR guided therapies. This technique yields time-resolved volumetric images during free-breathing, which are ideal for visualizing and quantifying breathing induced motion. To achieve this, navigated dynamic imaging requires acquiring multiple navigator slices. Reducing the number of navigator slices would allow for acquiring more data slices in the same time, and hence, increasing through-plane resolution or alternatively the overall acquisition time can be reduced while keeping resolution unchanged. To this end, we propose temporal interpolation of navigator slices using convolutional neural networks (CNNs). Our goal is to acquire fewer navigators and replace the missing ones with interpolation. We evaluate the proposed method on abdominal navigated dynamic MRI sequences acquired from 14 subjects. Investigations with several CNN architectures and training loss functions show favorable results for cost and a simple feed-forward network with no skip connections. When compared with interpolation by non-linear registration, the proposed method achieves higher interpolation accuracy on average as quantified in terms of root mean square error and residual motion. Analysis of the differences shows that the better performance is due to more accurate interpolation at peak exhalation and inhalation positions. Furthermore, the CNN-based approach requires substantially lower execution times than that of the registration-based method. At last, experiments on dynamic volume reconstruction reveal minimal differences between reconstructions with acquired and interpolated navigator slices.

Digital Topology and Geometry in Medical Imaging: A Survey

Digital topology and geometry refers to the use of topologic and geometric properties and features for images defined in digital grids. Such methods have been widely used in many medical imaging applications, including image segmentation, visualization, manipulation, interpolation, registration, surface-tracking, object representation, correction, quantitative morphometry etc. Digital topology and geometry play important roles in medical imaging research by enriching the scope of target outcomes and by adding strong theoretical foundations with enhanced stability, fidelity, and efficiency. This paper presents a comprehensive yet compact survey on results, principles, and insights of methods related to digital topology and geometry with strong emphasis on understanding their roles in various medical imaging applications. Specifically, this paper reviews methods related to distance analysis and path propagation, connectivity, surface-tracking, image segmentation, boundary and centerline detection, topology preservation and local topological properties, skeletonization, and object representation, correction, and quantitative morphometry. A common thread among the topics reviewed in this paper is that their theory and algorithms use the principle of digital path connectivity, path propagation, and neighborhood analysis.

MRI upsampling using feature-based nonlocal means approach

In magnetic resonance imaging (MRI), spatial resolution is limited by several factors such as acquisition time, short physiological phenomena, and organ motion. The acquired image usually has higher resolution in two dimensions (the acquisition plane) in comparison with the third dimension, resulting in highly anisotropic voxel size. Interpolation of these low resolution (LR) images using standard techniques, such as linear or spline interpolation, results in distorted edges in the planes perpendicular to the acquisition plane. This poses limitation on conducting quantitative analyses of LR images, particularly on their voxel-wise analysis and registration. We have proposed a new non-local means feature-based technique that uses structural information of a high resolution (HR) image with a different contrast and interpolates the LR image. In this approach, the similarity between voxels is estimated using a feature vector that characterizes the laminar pattern of the brain structures, resulting in a more accurate similarity measure in comparison with conventional patch-based approach. This technique can be applied to LR images with both anisotropic and isotropic voxel sizes. Experimental results conducted on brain MRI scans of patients with brain tumors, multiple sclerosis, epilepsy, as well as schizophrenic patients and normal controls show that the proposed method is more accurate, requires fewer computations, and thus is significantly faster than a previous state-of-the-art patch-based technique. We also show how the proposed method may be used to upsample regions of interest drawn on LR images.

A new formula for bivariate Hermite interpolation on variable step grids and its application to image interpolation

In this paper, we present a novel formula of the bivariate Hermite interpolating (BHI) polynomial in the case of support points arranged on a grid with variable step. This expression is applicable when interpolation of a bivariate function is required, given its value and the values of its partial derivatives of arbitrarily high order, at the support points. The proposed formula is a generalization of an existing formula for the bivariate Hermite polynomial. It is also algebraically much simpler, thus can be computed more efficiently. In order to apply Hermite interpolation to image interpolation, we simplify the proposed (BHI) to handle support points on a regular unit-step grid. The values of image partial derivatives are arithmetically approximated using compact finite differences. The proposed method is being assessed in a number of image interpolation experiments that include a synthetic image, for which the values of the partial derivatives are computed analytically, as well as a collection of images from different medical modalities. The proposed BHI with up to second-order image partial derivatives, outperforms the convolution-based interpolation methods, as well as generalized interpolation methods with the same number of support points that was compared with, in the majority of image interpolation experiments. The computational load of the proposed BHI is calculated and its behaviour with respect to its controlling parameters is investigated.

In silico study of low-frequency transcranial ultrasound fields in acute ischemic stroke patients

Ultrasound in the sub-megahertz range enhances thrombolysis and may be applied transcranially to ischemic stroke patients. The consistency of transcranial insonification needs to be evaluated. Acoustic and thermal simulations based on computed-tomography (CT) scans of 20 patients were performed. An unfocused 120-kHz transducer allowed homogeneous insonification of the thrombus, and positioning based on external landmarks performed similarly to an optimized placement based on CT data. With a weakly focused 500-kHz transducer, the landmark-based positioning underperformed. The predicted inter-patient variation of in situ acoustic pressure was similar with both the 120 and 500-kHz transducers for the optimized placement (18.0-26.4% relative standard deviation). The simulated maximum acoustic pressure in intervening tissues was 2.6 ± 0.6 and 2.0 ± 0.7 times the pressure in the thrombus for the 120-kHz and 500-kHz transducers, respectively. A 1 W/cm(2) insonification of the thrombus caused a 3.8 ± 2.2 °C increase in the bone for the 120-kHz transducer, and a 13.4 ± 3.3 °C increase for the 500-kHz transducer. Contralateral local maxima up to 1.1 times the pressure amplitude in the targeted zone were predicted for the 120-kHz transducer. We established two transducer placement approaches, one based on analysis of a head CT and the other using simple external, visible landmarks. Both approaches allowed consistent insonification of the thrombus.

An effective non-rigid registration approach for ultrasound image based on "demons" algorithm

Medical image registration is an important component of computer-aided diagnosis system in diagnostics, therapy planning, and guidance of surgery. Because of its low signal/noise ratio (SNR), ultrasound (US) image registration is a difficult task. In this paper, a fully automatic non-rigid image registration algorithm based on demons algorithm is proposed for registration of ultrasound images. In the proposed method, an "inertia force" derived from the local motion trend of pixels in a Moore neighborhood system is produced and integrated into optical flow equation to estimate the demons force, which is helpful to handle the speckle noise and preserve the geometric continuity of US images. In the experiment, a series of US images and several similarity measure metrics are utilized for evaluating the performance. The experimental results demonstrate that the proposed method can register ultrasound images efficiently, robust to noise, quickly and automatically.

An automatic service for the personalization of ventricular cardiac meshes

Computational cardiac physiology has great potential to improve the management of cardiovascular diseases. One of the main bottlenecks in this field is the customization of the computational model to the anatomical and physiological status of the patient. We present a fully automatic service for the geometrical personalization of cardiac ventricular meshes with high-order interpolation from segmented images. The method is versatile (able to work with different species and disease conditions) and robust (fully automatic results fulfilling accuracy and quality requirements in 87% of 255 cases). Results also illustrate the capability to minimize the impact of segmentation errors, to overcome the sparse resolution of dynamic studies and to remove the sometimes unnecessary anatomical detail of papillary and trabecular structures. The smooth meshes produced can be used to simulate cardiac function, and in particular mechanics, or can be used as diagnostic descriptors of anatomical shape by cardiologists. This fully automatic service is deployed in a cloud infrastructure, and has been made available and accessible to the scientific community.

Data-adapted moving least squares method for 3-D image interpolation

In this paper, we present a nonlinear three-dimensional interpolation scheme for gray-level medical images. The scheme is based on the moving least squares method but introduces a fundamental modification. For a given evaluation point, the proposed method finds the local best approximation by reproducing polynomials of a certain degree. In particular, in order to obtain a better match to the local structures of the given image, we employ locally data-adapted least squares methods that can improve the classical one. Some numerical experiments are presented to demonstrate the performance of the proposed method. Five types of data sets are used: MR brain, MR foot, MR abdomen, CT head, and CT foot. From each of the five types, we choose five volumes. The scheme is compared with some well-known linear methods and other recently developed nonlinear methods. For quantitative comparison, we follow the paradigm proposed by Grevera and Udupa (1998). (Each slice is first assumed to be unknown then interpolated by each method. The performance of each interpolation method is assessed statistically.) The PSNR results for the estimated volumes are also provided. We observe that the new method generates better results in both quantitative and visual quality comparisons.

Volumetric three-dimensional intravascular ultrasound visualization using shape-based nonlinear interpolation

Background

Intravascular ultrasound (IVUS) is a standard imaging modality for identification of plaque formation in the coronary and peripheral arteries. Volumetric three-dimensional (3D) IVUS visualization provides a powerful tool to overcome the limited comprehensive information of 2D IVUS in terms of complex spatial distribution of arterial morphology and acoustic backscatter information. Conventional 3D IVUS techniques provide sub-optimal visualization of arterial morphology or lack acoustic information concerning arterial structure due in part to low quality of image data and the use of pixel-based IVUS image reconstruction algorithms. In the present study, we describe a novel volumetric 3D IVUS reconstruction algorithm to utilize IVUS signal data and a shape-based nonlinear interpolation.

Methods

We developed an algorithm to convert a series of IVUS signal data into a fully volumetric 3D visualization. Intermediary slices between original 2D IVUS slices were generated utilizing the natural cubic spline interpolation to consider the nonlinearity of both vascular structure geometry and acoustic backscatter in the arterial wall. We evaluated differences in image quality between the conventional pixel-based interpolation and the shape-based nonlinear interpolation methods using both virtual vascular phantom data and in vivo IVUS data of a porcine femoral artery. Volumetric 3D IVUS images of the arterial segment reconstructed using the two interpolation methods were compared.

Results

In vitro validation and in vivo comparative studies with the conventional pixel-based interpolation method demonstrated more robustness of the shape-based nonlinear interpolation algorithm in determining intermediary 2D IVUS slices. Our shape-based nonlinear interpolation demonstrated improved volumetric 3D visualization of the in vivo arterial structure and more realistic acoustic backscatter distribution compared to the conventional pixel-based interpolation method.

Conclusions

This novel 3D IVUS visualization strategy has the potential to improve ultrasound imaging of vascular structure information, particularly atheroma determination. Improved volumetric 3D visualization with accurate acoustic backscatter information can help with ultrasound molecular imaging of atheroma component distribution.

Hierarchical patch-based sparse representation--a new approach for resolution enhancement of 4D-CT lung data

4D-CT plays an important role in lung cancer treatment because of its capability in providing a comprehensive characterization of respiratory motion for high-precision radiation therapy. However, due to the inherent high-dose exposure associated with CT, dense sampling along superior-inferior direction is often not practical, thus resulting in an inter-slice thickness that is much greater than in-plane voxel resolutions. As a consequence, artifacts such as lung vessel discontinuity and partial volume effects are often observed in 4D-CT images, which may mislead dose administration in radiation therapy. In this paper, we present a novel patch-based technique for resolution enhancement of 4D-CT images along the superior-inferior direction. Our working premise is that anatomical information that is missing in one particular phase can be recovered from other phases. Based on this assumption, we employ a hierarchical patch-based sparse representation mechanism to enhance the superior-inferior resolution of 4D-CT by reconstructing additional intermediate CT slices. Specifically, for each spatial location on an intermediate CT slice that we intend to reconstruct, we first agglomerate a dictionary of patches from images of all other phases in the 4D-CT. We then employ a sparse combination of patches from this dictionary, with guidance from neighboring (upper and lower) slices, to reconstruct a series of patches, which we progressively refine in a hierarchical fashion to reconstruct the final intermediate slices with significantly enhanced anatomical details. Our method was extensively evaluated using a public dataset. In all experiments, our method outperforms the conventional linear and cubic-spline interpolation methods in preserving image details and also in suppressing misleading artifacts, indicating that our proposed method can potentially be applied to better image-guided radiation therapy of lung cancer in the future.

A Markov random field approach for topology-preserving registration: application to object-based tomographic image interpolation

This paper proposes a topology-preserving multiresolution elastic registration method based on a discrete Markov random field of deformations and a block-matching procedure. The method is applied to the object-based interpolation of tomographic slices. For that purpose, the fidelity of a given deformation to the data is established by a block-matching strategy based on intensity- and gradient-related features, the smoothness of the transformation is favored by an appropriate prior on the field, and the deformation is guaranteed to maintain the topology by imposing some hard constraints on the local configurations of the field. The resulting deformation is defined as the maximum a posteriori configuration. Additionally, the relative influence of the fidelity and smoothness terms is weighted by the unsupervised estimation of the field parameters. In order to obtain an unbiased interpolation result, the registration is performed both in the forward and backward directions, and the resulting transformations are combined by using the local information content of the deformation. The method is applied to magnetic resonance and computed tomography acquisitions of the brain and the torso. Quantitative comparisons offer an overall improvement in performance with respect to related works in the literature. Additionally, the application of the interpolation method to cardiac magnetic resonance images has shown that the removal of any of the main components of the algorithm results in a decrease in performance which has proven to be statistically significant.

Constrained reverse diffusion for thick slice interpolation of 3D volumetric MRI images

Due to physical limitations inherent in magnetic resonance imaging scanners, three dimensional volumetric scans are often acquired with anisotropic voxel resolution. We investigate several interpolation approaches to reduce the anisotropy and present a novel approach - constrained reverse diffusion for thick slice interpolation. This technique was compared to common methods: linear and cubic B-Spline interpolation and a technique based on non-rigid registration of neighboring slices. The methods were evaluated on artificial MR phantoms and real MR scans of human brain. The constrained reverse diffusion approach delivered promising results and provides an alternative for thick slice interpolation, especially for higher anisotropy factors.

Curvature interpolation method for image zooming

We introduce a novel image zooming algorithm, called the curvature interpolation method (CIM), which is partial-differential-equation (PDE)-based and easy to implement. In order to minimize artifacts arising in image interpolation such as image blur and the checkerboard effect, the CIM first evaluates the curvature of the low-resolution image. After interpolating the curvature to the high-resolution image domain, the CIM constructs the high-resolution image by solving a linearized curvature equation, incorporating the interpolated curvature as an explicit driving force. It has been numerically verified that the new zooming method can produce clear images of sharp edges which are already denoised and superior to those obtained from linear methods and PDE-based methods of no curvature information. Various results are given to prove effectiveness and reliability of the new method.

Context-based interpolation of 3-D medical images

A context-based 3D interpolation technique is proposed to enhance the out-of plane resolution of 3D medical images. The proposed technique represents a new approach of aiding 3D interpolation and improving its performance by efficient use of domain knowledge about the anatomy, objects and imaging modalities. In the new approach a family of adaptive 3D interpolation filters are designed and conditioned on different spatial contexts (classes of feature vectors). Training is used to incorporate the domain knowledge into the design of these interpolators. Experimental results show significant improvement of the new approach over some existing 3D interpolation techniques.

Fusion of multi-planar images for improved three-dimensional object reconstruction

Due to the scan time limitation, our MRI studies of the human tongue can acquire only a limited number of contiguous two-dimensional (2D) slices to form a volumetric data set in a given series. An interpolated three-dimensional (3D) reconstruction using images acquired in a single plane presents artifacts. To address this issue, we developed a wavelet-based bidirectional linear fusion method that uses slices acquired from sagittal and coronal planes to estimate the unknown values of the inter-slice voxels. We use an interpolation method to estimate the voxel value based on neighboring fiducial voxels in the bounding slices. This interpolation is followed by a wavelet fusion to recover image details by integrating prominent coefficients from the interpolated images. Our method was evaluated using 2D MR images and 3D phantoms. Experiments demonstrated that our method reduces interpolation artifacts and greatly improves the 3D reconstruction accuracy. The advantage of our method casts new light on MR imaging and image processing and permits us to achieve high resolution and short acquisition time simultaneously.

Improved tensor scale computation with application to medical image interpolation

Tensor scale (t-scale) is a parametric representation of local structure morphology that simultaneously describes its orientation, shape and isotropic scale. At any image location, t-scale represents the largest ellipse (an ellipsoid in three dimensions) centered at that location and contained in the same homogeneous region. Here, we present an improved algorithm for t-scale computation and study its application to image interpolation. Specifically, the t-scale computation algorithm is improved by: (1) enhancing the accuracy of identifying local structure boundary and (2) combining both algebraic and geometric approaches in ellipse fitting. In the context of interpolation, a closed form solution is presented to determine the interpolation line at each image location in a gray level image using t-scale information of adjacent slices. At each location on an image slice, the method derives normal vector from its t-scale that yields trans-orientation of the local structure and points to the closest edge point. Normal vectors at the matching two-dimensional locations on two adjacent slices are used to compute the interpolation line using a closed form equation. The method has been applied to BrainWeb data sets and to several other images from clinical applications and its accuracy and response to noise and other image-degrading factors have been examined and compared with those of current state-of-the-art interpolation methods. Experimental results have established the superiority of the new t-scale based interpolation method as compared to existing interpolation algorithms. Also, a quantitative analysis based on the paired t-test of residual errors has ascertained that the improvements observed using the t-scale based interpolation are statistically significant.

A non-local approach for image super-resolution using intermodality priors

Image enhancement is of great importance in medical imaging where image resolution remains a crucial point in many image analysis algorithms. In this paper, we investigate brain hallucination (Rousseau, 2008), or generating a high-resolution brain image from an input low-resolution image, with the help of another high-resolution brain image. We propose an approach for image super-resolution by using anatomical intermodality priors from a reference image. Contrary to interpolation techniques, in order to be able to recover fine details in images, the reconstruction process is based on a physical model of image acquisition. Another contribution to this inverse problem is a new regularization approach that uses an example-based framework integrating non-local similarity constraints to handle in a better way repetitive structures and texture. The effectiveness of our approach is demonstrated by experiments on realistic Brainweb Magnetic Resonance images and on clinical images from ADNI, generating automatically high-quality brain images from low-resolution input.

Modified control grid interpolation for the volumetric reconstruction of fluid flows

Complex applications in fluid dynamics research often require more highly resolved velocity data than direct measurements or simulations provide. The advent of stereo PIV and PCMR techniques has advanced the state-of-the-art in flow velocity measurement, but 3D spatial resolution remains limited. Here a new technique is proposed for velocity data interpolation to address this problem. The new method performs with higher quality than competing solutions from the literature in terms of accurately interpolating velocities, maintaining fluid structure and domain boundaries, and preserving coherent structures.

A morphology-based approach for interslice interpolation of anatomical slices from volumetric images

This paper proposes a new morphology-based approach for the interslice interpolation of current transformer (CT) and MRI datasets composed of parallel slices. Our approach is object based and accepts as input data binary slices belonging to the same anatomical structure. Such slices may contain one or more regions, since topological changes between two adjacent slices may occur. Our approach handles explicitly interslice topology changes by decomposing a many-to-many correspondence into three fundamental cases: one-to-one, one-to-many, and zero-to-one correspondences. The proposed interpolation process is iterative. One iteration of this process computes a transition sequence between a pair of corresponding input slices, and selects the element located at equal distance from the input slices. This algorithmic design yields a gradual, smooth change of shape between the input slices. Therefore, the main contribution of our approach is its ability to interpolate between two anatomic shapes by creating a smooth, gradual change of shape, and without generating over-smoothed interpolated shapes.

Subject-specific models for image-guided cardiac surgery

Three-dimensional visualization for planning and guidance is still not routinely available for minimally invasive cardiac surgery (MICS). This can be addressed by providing the surgeon with subject-specific geometric models derived from 3D preoperative images for planning of port locations or to rehearse the procedure. For guidance purposes, these models can also be registered to the subject using intraoperative images. In this paper, we present a method for extracting subject-specific heart geometry from preoperative MR images. The main obstacle we face is the low quality of clinical data in terms of resolution, signal-to-noise ratio, and presence of artefacts. Instead of using these images directly, we approach the problem in three steps: (1) generate a high quality template model, (2) register the template with the preoperative data, and (3) animate the result over the cardiac cycle. Validation of this approach showed that dynamic subject-specific models can be generated with a mean error of 3.6+/-1.1 mm from low resolution target images (6 mm slices). Thus, the models are sufficiently accurate for MICS training and procedure planning. In terms of guidance, we also demonstrate how the resulting models may be adapted to the operating room using intraoperative ultrasound imaging.

Respiratory motion correction in 3-D PET data with advanced optical flow algorithms

The problem of motion is well known in positron emission tomography (PET) studies. The PET images are formed over an elongated period of time. As the patients cannot hold breath during the PET acquisition, spatial blurring and motion artifacts are the natural result. These may lead to wrong quantification of the radioactive uptake. We present a solution to this problem by respiratory-gating the PET data and correcting the PET images for motion with optical flow algorithms. The algorithm is based on the combined local and global optical flow algorithm with modifications to allow for discontinuity preservation across organ boundaries and for application to 3-D volume sets. The superiority of the algorithm over previous work is demonstrated on software phantom and real patient data.

A new method for registration-based medical image interpolation

A new technique is presented for interpolating between grey-scale images in a medical data set. Registration between neighboring slices is achieved with a modified control grid interpolation algorithm that selectively accepts displacement field updates in a manner optimized for performance. A cubic interpolator is then applied to pixel intensities correlated by the displacement fields. Special considerations are made for efficiency, interpolation quality, and compression in the implementation of the algorithm. Experimental results show that the new method achieves good quality, while offering dramatic improvement in efficiency relative to the best competing method.

Optical flow based volumetric spatio-temporal interpolation

Acquisition of three-dimensional medical images with excellent spatial resolution is achieved almost always at the cost-dictating expense of increased acquisition time. This can be mitigated using shape-respecting inter-slice interpolation of sparsely acquired datasets. Existing shape and registration based interpolation techniques are suboptimal. This paper proposes a new optical flow based approach to interpolate between sparse data. The Horn and Schunck technique is used to estimate the pixel-wise flow vectors across the slices; intermediate data planes can be reconstructed from these flow vectors. The proposed technique is accurate and computationally fast. Coupled with anatomy aware sparse acquisitions, the proposed technique shows promise of judiciously managing the patient-specific acquisition.

Volume reconstruction based on non-rigid registration

Volume reconstruction is one of the key problems in 3D image rendering and analysis. Inter slice interpolation methods have been widely discussed in the literature and object-based algorithms have been shown to well behave. In this paper, we present a non-rigid registration based strategy to improve the volume reconstruction. A level set evolution technique is proposed to yield the deformation between adjacent slices. A modified bilinear interpolation method is then designed to generate propagating image. A multi-resolution scheme is applied to decrease the computation time and support large deformation. The resulting images show good results on regions enclosing different anatomic structures.

Probe trajectory interpolation for 3D reconstruction of freehand ultrasound

Three-dimensional (3D) freehand ultrasound uses the acquisition of non-parallel B-scans localized in 3D by a tracking system (optic, mechanical or magnetic). Using the positions of the irregularly spaced B-scans, a regular 3D lattice volume can be reconstructed, to which conventional 3D computer vision algorithms (registration and segmentation) can be applied. This paper presents a new 3D reconstruction method which explicitly accounts for the probe trajectory. Experiments were conducted on phantom and intra-operative datasets using various probe motion types and varied slice-to-slice B-scan distances. Results suggest that this technique improves on classical methods at the expense of computational time.

A new orientation-adaptive interpolation method

We propose an isophote-oriented, orientation-adaptive interpolation method. The proposed method employs an interpolation kernel that adapts to the local orientation of isophotes, and the pixel values are obtained through an oriented, bilinear interpolation. We show that, by doing so, the curvature of the interpolated isophotes is reduced, and, thus, zigzagging artifacts are largely suppressed. Analysis and experiments show that images interpolated using the proposed method are visually pleasing and almost artifact free.

An optical flow based method for improved reconstruction of 4D CT data sets acquired during free breathing

Respiratory motion degrades anatomic position reproducibility and leads to issues affecting image acquisition, treatment planning, and radiation delivery. Four-dimensional (4D) computer tomography (CT) image acquisition can be used to measure the impact of organ motion and to explicitly account for respiratory motion during treatment planning and radiation delivery. Modern CT scanners can only scan a limited region of the body simultaneously and patients have to be scanned in segments consisting of multiple slices. A respiratory signal (spirometer signal or surface tracking) is used to reconstruct a 4D data set by sorting the CT scans according to the couch position and signal coherence with predefined respiratory phases. But artifacts can occur if there are no acquired data segments for exactly the same respiratory state for all couch positions. These artifacts are caused by device-dependent limitations of gantry rotation, image reconstruction times and by the variability of the patient's respiratory pattern. In this paper an optical flow based method for improved reconstruction of 4D CT data sets from multislice CT scans is presented. The optical flow between scans at neighboring respiratory states is estimated by a non-linear registration method. The calculated velocity field is then used to reconstruct a 4D CT data set by interpolating data at exactly the predefined respiratory phase. Our reconstruction method is compared with the usually used reconstruction based on amplitude sorting. The procedures described were applied to reconstruct 4D CT data sets for four cancer patients and a qualitative and quantitative evaluation of the optical flow based reconstruction method was performed. Evaluation results show a relevant reduction of reconstruction artifacts by our technique. The reconstructed 4D data sets were used to quantify organ displacements and to visualize the abdominothoracic organ motion.

3D freehand ultrasound reconstruction based on probe trajectory

3D freehand ultrasound imaging is a very attractive technique in medical examinations and intra-operative stage for its cost and field of view capacities. This technique produces a set of non parallel B-scans which are irregularly distributed in the space. Reconstruction amounts to computing a regular lattice volume and is needed to apply conventional computer vision algorithms like registration. In this paper, a new 3D reconstruction method is presented, taking explicitly into account the probe trajectory. Experiments were conducted on different data sets with various probe motion types and indicate that this technique outperforms classical methods, especially on low acquisition frame rate.

Partial volume reduction by interpolation with reverse diffusion

Many medical images suffer from the partial volume effect where a boundary between two structures of interest falls in the midst of a voxel giving a signal value that is a mixture of the two. We propose a method to restore the ideal boundary by splitting a voxel into subvoxels and reapportioning the signal into the subvoxels. Each voxel is divided by nearest neighbor interpolation. The gray level of each subvoxel is considered as “material” able to move between subvoxels but not between voxels. A partial differential equation is written to allow the material to flow towards the highest gradient direction, creating a “reverse” diffusion process. Flow is subject to constraints that tend to create step edges. Material is conserved in the process thereby conserving signal. The method proceeds until the flow decreases to a low value. To test the method, synthetic images were downsampled to simulate the partial volume artifact and restored. Corrected images were remarkably closer both visually and quantitatively to the original images than those obtained from common interpolation methods: on simulated data standard deviation of the errors were 3.8%, 6.6%, and 7.1% of the dynamic range for the proposed method, bicubic, and bilinear interpolation, respectively. The method was relatively insensitive to noise. On gray level, scanned text, MRI physical phantom, and brain images, restored images processed with the new method were visually much closer to high-resolution counterparts than those obtained with common interpolation methods.

Inter-breath-hold Registration for the Production of High Resolution Cardiac MR Volumes

High resolution MRI images of the beating heart permit observation of detailed anatomical features and enable quantification of small changes in metrics of cardiac function. To obtain approximately isotropic sampling with an adequate spatial and temporal resolution, these images need to be acquired in multiple breath-holds. They are, therefore, often affected by through-plane discontinuities due to inconsistent breath-hold positions. This paper presents a method to correct for these discontinuities by performing breath-hold-by-breath-hold registration of high resolution 3D data to radial long axis images. The corrected images appear free of discontinuities, and it was found that they could be delineated more reproducibly than uncorrected images. This reduces the sample size required to detect systematic changes in blood pool volume by 57% at end systole and 78% at end diastole.