Shape-based interpolation of multidimensional grey-level images

Unsupervised shape-and-texture-based generative adversarial tuning of pre-trained networks for carotid segmentation from 3D ultrasound images

BACKGROUND

Vessel-wall volume and localized three-dimensional ultrasound (3DUS) metrics are sensitive to the change of carotid atherosclerosis in response to medical/dietary interventions. Manual segmentation of the media-adventitia boundary (MAB) and lumen-intima boundary (LIB) required to obtain these metrics is time-consuming and prone to observer variability. Although supervised deep-learning segmentation models have been proposed, training of these models requires a sizeable manually segmented training set, making larger clinical studies prohibitive.

PURPOSE

We aim to develop a method to optimize pre-trained segmentation models without requiring manual segmentation to supervise the fine-tuning process.

METHODS

We developed an adversarial framework called the unsupervised shape-and-texture generative adversarial network (USTGAN) to fine-tune a convolutional neural network (CNN) pre-trained on a source dataset for accurate segmentation of a target dataset. The network integrates a novel texture-based discriminator with a shape-based discriminator, which together provide feedback for the CNN to segment the target images in a similar way as the source images. The texture-based discriminator increases the accuracy of the CNN in locating the artery, thereby lowering the number of failed segmentations. Failed segmentation was further reduced by a self-checking mechanism to flag longitudinal discontinuity of the artery and by self-correction strategies involving surface interpolation followed by a case-specific tuning of the CNN. The U-Net was pre-trained by the source dataset involving 224 3DUS volumes with 136, 44, and 44 volumes in the training, validation and testing sets. The training of USTGAN involved the same training group of 136 volumes in the source dataset and 533 volumes in the target dataset. No segmented boundaries for the target cohort were available for training USTGAN. The validation and testing of USTGAN involved 118 and 104 volumes from the target cohort, respectively. The segmentation accuracy was quantified by Dice Similarity Coefficient (DSC), and incorrect localization rate (ILR). Tukey's Honestly Significant Difference multiple comparison test was employed to quantify the difference of DSCs between models and settings, wherep ≤ 0.05 $p\,\le \,0.05$ was considered statistically significant.

RESULTS

USTGAN attained a DSC of85.7 ± 13.0 $85.7\,\pm \,13.0$ % in LIB and86.2 ± 10.6 ${86.2}\,\pm \,{10.6}$ % in MAB, improving from the baseline performance of74.6 ± 30.7 ${74.6}\,\pm \,{30.7}$ % in LIB (p< 10 - 12 $<10^{-12}$ ) and75.7 ± 28.9 ${75.7}\,\pm \,{28.9}$ % in MAB (p< 10 - 12 $<10^{-12}$ ). Our approach outperformed six state-of-the-art domain-adaptation models (MAB:p ≤ 3.63 × 10 - 7 $p \le 3.63\,\times \,10^{-7}$ , LIB:p ≤ 9.34 × 10 - 8 $p\,\le \,9.34\,\times \,10^{-8}$ ). The proposed USTGAN also had the lowest ILR among the methods compared (LIB: 2.5%, MAB: 1.7%).

CONCLUSION

Our framework improves segmentation generalizability, thereby facilitating efficient carotid disease monitoring in multicenter trials and in clinics with less expertise in 3DUS imaging.

Development of a Three-Dimensional Carotid Ultrasound Image Segmentation Workflow for Improved Efficiency, Reproducibility and Accuracy in Measuring Vessel Wall and Plaque Volume and Thickness

Automated segmentation of carotid lumen-intima boundary (LIB) and media-adventitia boundary (MAB) by deep convolutional neural networks (CNN) from three-dimensional ultrasound (3DUS) images has made assessment and monitoring of carotid atherosclerosis more efficient than manual segmentation. However, training of CNN still requires manual segmentation of LIB and MAB. Therefore, there is a need to improve the efficiency of manual segmentation and develop strategies to improve segmentation accuracy by the CNN for serial monitoring of carotid atherosclerosis. One strategy to reduce segmentation time is to increase the interslice distance (ISD) between segmented axial slices of a 3DUS image while maintaining the segmentation reliability. We, for the first time, investigated the effect of ISD on the reproducibility of MAB and LIB segmentations. The intra-observer reproducibility of LIB and MAB segmentations at ISDs of 1 mm and 2 mm was not statistically significantly different, whereas the reproducibility at ISD = 3 mm was statistically lower. Therefore, we conclude that segmentation with an ISD of 2 mm provides sufficient reliability for CNN training. We further proposed training the CNN by the baseline images of the entire cohort of patients for automatic segmentation of the follow-up images acquired for the same cohort. We validated that segmentation with this time-based partitioning approach is more accurate than that produced by patient-based partitioning, especially at the carotid bifurcation. This study forms the basis for an efficient, reproducible, and accurate 3DUS workflow for serial monitoring of carotid atherosclerosis useful in risk stratification of cardiovascular events and in evaluating the efficacy of new treatments.

Contour interpolation by deep learning approach

Purpose

Contour interpolation is an important tool for expediting manual segmentation of anatomical structures. The process allows users to manually contour on discontinuous slices and then automatically fill in the gaps, therefore saving time and efforts. The most used conventional shape-based interpolation (SBI) algorithm, which operates on shape information, often performs suboptimally near the superior and inferior borders of organs and for the gastrointestinal structures. In this study, we present a generic deep learning solution to improve the robustness and accuracy for contour interpolation, especially for these historically difficult cases.

Approach

A generic deep contour interpolation model was developed and trained using 16,796 publicly available cases from 5 different data libraries, covering 15 organs. The network inputs were a 128 × 128 × 5 image patch and the two-dimensional contour masks for the top and bottom slices of the patch. The outputs were the organ masks for the three middle slices. The performance was evaluated on both dice scores and distance-to-agreement (DTA) values.

Results

The deep contour interpolation model achieved a dice score of 0.95 ± 0.05 and a mean DTA value of 1.09 ± 2.30    mm , averaged on 3167 testing cases of all 15 organs. In a comparison, the results by the conventional SBI method were 0.94 ± 0.08 and 1.50 ± 3.63    mm , respectively. For the difficult cases, the dice score and DTA value were 0.91 ± 0.09 and 1.68 ± 2.28    mm by the deep interpolator, compared with 0.86 ± 0.13 and 3.43 ± 5.89    mm by SBI. The t-test results confirmed that the performance improvements were statistically significant ( p < 0.05 ) for all cases in dice scores and for small organs and difficult cases in DTA values. Ablation studies were also performed.

Conclusions

A deep learning method was developed to enhance the process of contour interpolation. It could be useful for expediting the tasks of manual segmentation of organs and structures in the medical images.

Improved neonatal brain MRI segmentation by interpolation of motion corrupted slices

BACKGROUND AND PURPOSE

To apply and evaluate an intensity-based interpolation technique, enabling segmentation of motion-affected neonatal brain MRI.

METHODS

Moderate-late preterm infants were enrolled in a prospective cohort study (Brain Imaging in Moderate-late Preterm infants "BIMP-study") between August 2017 and November 2019. T2-weighted MRI was performed around term equivalent age on a 3T MRI. Scans without motion (n = 27 [24%], control group) and with moderate-severe motion (n = 33 [29%]) were included. Motion-affected slices were re-estimated using intensity-based shape-preserving cubic spline interpolation, and automatically segmented in eight structures. Quality of interpolation and segmentation was visually assessed for errors after interpolation. Reliability was tested using interpolated control group scans (18/54 axial slices). Structural similarity index (SSIM) was used to compare T2-weighted scans, and Sørensen-Dice was used to compare segmentation before and after interpolation. Finally, volumes of brain structures of the control group were used assessing sensitivity (absolute mean fraction difference) and bias (confidence interval of mean difference).

RESULTS

Visually, segmentation of 25 scans (22%) with motion artifacts improved with interpolation, while segmentation of eight scans (7%) with adjacent motion-affected slices did not improve. Average SSIM was .895 and Sørensen-Dice coefficients ranged between .87 and .97. Absolute mean fraction difference was ≤0.17 for less than or equal to five interpolated slices. Confidence intervals revealed a small bias for cortical gray matter (0.14-3.07 cm3 ), cerebrospinal fluid (0.39-1.65 cm3 ), deep gray matter (0.74-1.01 cm3 ), and brainstem volumes (0.07-0.28 cm3 ) and a negative bias in white matter volumes (-4.47 to -1.65 cm3 ).

CONCLUSION

According to qualitative and quantitative assessment, intensity-based interpolation reduced the percentage of discarded scans from 29% to 7%.

Segmentation of common and internal carotid arteries from 3D ultrasound images based on adaptive triple loss

PURPOSE

Vessel wall volume (VWV) and localized vessel-wall-plus-plaque thickness (VWT) measured from three-dimensional (3D) ultrasound (US) carotid images are sensitive to anti-atherosclerotic effects of medical/dietary treatments. VWV and VWT measurements require the lumen-intima (LIB) and media-adventitia boundaries (MAB) at the common and internal carotid arteries (CCA and ICA). However, most existing segmentation techniques were capable of segmenting the CCA only. An approach capable of segmenting the MAB and LIB from the CCA and ICA was required to accelerate VWV and VWT quantification.

METHODS

Segmentation for CCA and ICA was performed independently using the proposed two-channel U-Net, which was driven by a novel loss function known as the adaptive triple Dice loss (ADTL) function. The training set was augmented by interpolating manual segmentation along the longitudinal direction, thereby taking continuity of the artery into account. A test-time augmentation (TTA) approach was applied, in which segmentation was performed three times based on the input axial images and its flipped versions; the final segmentation was generated by pixel-wise majority voting.

RESULTS

Experiments involving 224 3DUS volumes produce a Dice similarity coefficient (DSC) of 95.1% ± 4.1% and 91.6% ± 6.6% for the MAB and LIB, in the CCA, respectively, and 94.2% ± 3.3% and 89.0% ± 8.1% for the MAB and LIB, in the ICA, respectively. TTA and ATDL independently contributed to a statistically significant improvement to all boundaries except the LIB in ICA.

CONCLUSIONS

The proposed two-channel U-Net with ADTL and TTA can segment the CCA and ICA accurately and efficiently from the 3DUS volume. Our approach has the potential to accelerate the transition of 3DUS measurements of carotid atherosclerosis to clinical research.

Sparse self-attention aggregation networks for neural sequence slice interpolation

Background

Microscopic imaging is a crucial technology for visualizing neural and tissue structures. Large-area defects inevitably occur during the imaging process of electron microscope (EM) serial slices, which lead to reduced registration and semantic segmentation, and affect the accuracy of 3D reconstruction. The continuity of biological tissue among serial EM images makes it possible to recover missing tissues utilizing inter-slice interpolation. However, large deformation, noise, and blur among EM images remain the task challenging. Existing flow-based and kernel-based methods have to perform frame interpolation on images with little noise and low blur. They also cannot effectively deal with large deformations on EM images.

Results

In this paper, we propose a sparse self-attention aggregation network to synthesize pixels following the continuity of biological tissue. First, we develop an attention-aware layer for consecutive EM images interpolation that implicitly adopts global perceptual deformation. Second, we present an adaptive style-balance loss taking the style differences of serial EM images such as blur and noise into consideration. Guided by the attention-aware module, adaptively synthesizing each pixel aggregated from the global domain further improves the performance of pixel synthesis. Quantitative and qualitative experiments show that the proposed method is superior to the state-of-the-art approaches.

Conclusions

The proposed method can be considered as an effective strategy to model the relationship between each pixel and other pixels from the global domain. This approach improves the algorithm’s robustness to noise and large deformation, and can accurately predict the effective information of the missing region, which will greatly promote the data analysis of neurobiological research.

A Fast Segmentation and Efficient Slice Reconstruction Technique for Head CT Images

The three-dimensional (3D) reconstruction of medical images usually requires hundreds of two-dimensional (2D) scan images. Segmentation, an obligatory part in reconstruction, needs to be performed for all the slices consuming enormous storage space and time. To reduce storage space and time, this paper proposes a three-stage procedure, namely, slice selection, segmentation and interpolation. The methodology will have the potential to 3D reconstruct the human head from minimum selected slices. The first stage of slice selection is based on structural similarity measurement, discarding the most similar slices with none or minimal impact on details. The second stage of segmentation of the selected slices is performed using our proposed phase-field segmentation method. Validation of our segmentation results is done via comparison with other deformable models, and results show that the proposed method provides fast and accurate segmentation. The third stage of interpolation is based on modified curvature registration-based interpolation, and it is applied to re-create the discarded slices. This method is compared to both standard linear interpolation and registration-based interpolation in 100 tomographic data sets. Results show that the modified curvature registration-based interpolation reconstructs missing slices with 96% accuracy and shows an improvement in sensitivity (95.802%) on par with specificity (95.901%).

Application of Time-Frequency Domain Transform to Three-Dimensional Interpolation of Medical Images

Medical image three-dimensional (3D) interpolation is an important means to improve the image effect in 3D reconstruction. In image processing, the time-frequency domain transform is an efficient method. In this article, several time-frequency domain transform methods are applied and compared in 3D interpolation. And a Sobel edge detection and 3D matching interpolation method based on wavelet transform is proposed. We combine wavelet transform, traditional matching interpolation methods, and Sobel edge detection together in our algorithm. What is more, the characteristics of wavelet transform and Sobel operator are used. They deal with the sub-images of wavelet decomposition separately. Sobel edge detection 3D matching interpolation method is used in low-frequency sub-images under the circumstances of ensuring high frequency undistorted. Through wavelet reconstruction, it can get the target interpolation image. In this article, we make 3D interpolation of the real computed tomography (CT) images. Compared with other interpolation methods, our proposed method is verified to be effective and superior.

A framework for quantification and visualization of segmentation accuracy and variability in 3D lateral ventricle ultrasound images of preterm neonates

PURPOSE

Intraventricular hemorrhage (IVH) is a major cause of brain injury in preterm neonates. Three dimensional ultrasound (US) imaging systems have been developed to visualize 3D anatomical structure of preterm neonatal intracranial ventricular system with IVH and ventricular dilation. To allow quantitative analysis, the ventricle system is required to be segmented accurately and efficiently from 3D US images. Although semiautomatic segmentation algorithms have been developed, local segmentation accuracy and variability associated with these algorithms should be evaluated statistically before they can be applied in clinical settings. This work proposes a statistical framework to quantify the local accuracy and variability and performs statistical tests to identify locations where the semiautomatically segmented surfaces are significantly different from manually segmented surfaces.

METHODS

Three dimensional lateral ventricle US images of preterm neonates were each segmented six times manually and using a semiautomated segmentation algorithm. The local difference between manually and algorithmically segmented surfaces as well as the segmentation variability for each method was computed and superimposed on the ventricular surface of each subject. To summarize the segmentation performance for a whole group of subjects, the subject-specific local difference and standard deviation maps were registered onto a 3D template ventricular surface using a nonrigid registration algorithm. Pointwise, intersubject average accuracy and pooled variability for the whole group of subjects can be computed and visualized on the template surface, providing a summary of performance of the segmentation algorithm for the whole group of ventricles with highly variable geometry. In addition to pointwise statistical analysis performed on the template surface, statistical conclusion regarding the accuracy of the segmentation algorithm was made for subregions and the whole ventricle with the spatial correlation of pointwise accuracy taken into account.

RESULTS

Ten 3D US images were involved in this study. Pointwise local difference, ΔS, its absolute value |ΔS| as well as the standard deviations of the manual and algorithm segmentations were computed and superimposed on the each ventricle surface. Regions with lower segmentation accuracy and higher segmentation variability can be identified from these maps, and the localized information was applied to improve the accuracy of the algorithm. Intersubject average ΔS and |ΔS| as well as pooled standard deviations was computed on the template surface. Intersubject average ΔS and |ΔS| indicated that the algorithm underestimated regions in the neighborhood of the tips of anterior, inferior, and posterior horns. Intersubject pooled standard deviations indicated that manual segmentation had a higher segmentation variability than algorithm segmentation over the whole ventricle. Statistical analysis on the template surface showed that there was significant difference between algorithm and manual methods for segmenting the right lateral ventricle but not for the left lateral ventricle.

CONCLUSIONS

A framework was proposed for evaluating, visualizing, and summarizing the local accuracy and variability of a segmentation algorithm. This framework can be used for improving the accuracy of segmentation algorithms, as well as providing useful feedback to improve the manual segmentation performance. More importantly, this framework can be applied for longitudinal monitoring of local ventricular changes of neonates with IVH.

Objective Quality Assessment of Interpolated Natural Images

Image interpolation techniques that create high-resolution images from low-resolution (LR) images are widely used in real world applications, but how to evaluate the quality of interpolated images is not a well-resolved issue. Subjective assessment methods are useful and reliable, but are also slow and expensive. Here, we propose an objective method to assess the quality of an interpolated natural image using the available LR image as a reference. Our method adopts a natural scene statistics (NSS) framework, where image quality degradation is gauged by the deviation of its statistical features from the NSS models trained upon high-quality natural images. Two distortion measures are proposed, namely, interpolated natural image distortion (IND) and weighted IND. Validations by subjective tests show that the proposed approach performs statistically equivalent or sometimes better than an average human subject. Moreover, we demonstrate the potential application of the proposed method in parameter tuning of image interpolation algorithms.

Digital Topology and Geometry in Medical Imaging: A Survey

Digital topology and geometry refers to the use of topologic and geometric properties and features for images defined in digital grids. Such methods have been widely used in many medical imaging applications, including image segmentation, visualization, manipulation, interpolation, registration, surface-tracking, object representation, correction, quantitative morphometry etc. Digital topology and geometry play important roles in medical imaging research by enriching the scope of target outcomes and by adding strong theoretical foundations with enhanced stability, fidelity, and efficiency. This paper presents a comprehensive yet compact survey on results, principles, and insights of methods related to digital topology and geometry with strong emphasis on understanding their roles in various medical imaging applications. Specifically, this paper reviews methods related to distance analysis and path propagation, connectivity, surface-tracking, image segmentation, boundary and centerline detection, topology preservation and local topological properties, skeletonization, and object representation, correction, and quantitative morphometry. A common thread among the topics reviewed in this paper is that their theory and algorithms use the principle of digital path connectivity, path propagation, and neighborhood analysis.

Population-based studies of myocardial hypertrophy: high resolution cardiovascular magnetic resonance atlases improve statistical power

BACKGROUND

Cardiac phenotypes, such as left ventricular (LV) mass, demonstrate high heritability although most genes associated with these complex traits remain unidentified. Genome-wide association studies (GWAS) have relied on conventional 2D cardiovascular magnetic resonance (CMR) as the gold-standard for phenotyping. However this technique is insensitive to the regional variations in wall thickness which are often associated with left ventricular hypertrophy and require large cohorts to reach significance. Here we test whether automated cardiac phenotyping using high spatial resolution CMR atlases can achieve improved precision for mapping wall thickness in healthy populations and whether smaller sample sizes are required compared to conventional methods.

METHODS

LV short-axis cine images were acquired in 138 healthy volunteers using standard 2D imaging and 3D high spatial resolution CMR. A multi-atlas technique was used to segment and co-register each image. The agreement between methods for end-diastolic volume and mass was made using Bland-Altman analysis in 20 subjects. The 3D and 2D segmentations of the LV were compared to manual labeling by the proportion of concordant voxels (Dice coefficient) and the distances separating corresponding points. Parametric and nonparametric data were analysed with paired t-tests and Wilcoxon signed-rank test respectively. Voxelwise power calculations used the interstudy variances of wall thickness.

RESULTS

The 3D volumetric measurements showed no bias compared to 2D imaging. The segmented 3D images were more accurate than 2D images for defining the epicardium (Dice: 0.95 vs 0.93, P<0.001; mean error 1.3 mm vs 2.2 mm, P<0.001) and endocardium (Dice 0.95 vs 0.93, P<0.001; mean error 1.1 mm vs 2.0 mm, P<0.001). The 3D technique resulted in significant differences in wall thickness assessment at the base, septum and apex of the LV compared to 2D (P<0.001). Fewer subjects were required for 3D imaging to detect a 1 mm difference in wall thickness (72 vs 56, P<0.001).

CONCLUSIONS

High spatial resolution CMR with automated phenotyping provides greater power for mapping wall thickness than conventional 2D imaging and enables a reduction in the sample size required for studies of environmental and genetic determinants of LV wall thickness.

Data-adapted moving least squares method for 3-D image interpolation

In this paper, we present a nonlinear three-dimensional interpolation scheme for gray-level medical images. The scheme is based on the moving least squares method but introduces a fundamental modification. For a given evaluation point, the proposed method finds the local best approximation by reproducing polynomials of a certain degree. In particular, in order to obtain a better match to the local structures of the given image, we employ locally data-adapted least squares methods that can improve the classical one. Some numerical experiments are presented to demonstrate the performance of the proposed method. Five types of data sets are used: MR brain, MR foot, MR abdomen, CT head, and CT foot. From each of the five types, we choose five volumes. The scheme is compared with some well-known linear methods and other recently developed nonlinear methods. For quantitative comparison, we follow the paradigm proposed by Grevera and Udupa (1998). (Each slice is first assumed to be unknown then interpolated by each method. The performance of each interpolation method is assessed statistically.) The PSNR results for the estimated volumes are also provided. We observe that the new method generates better results in both quantitative and visual quality comparisons.

Volumetric three-dimensional intravascular ultrasound visualization using shape-based nonlinear interpolation

Background

Intravascular ultrasound (IVUS) is a standard imaging modality for identification of plaque formation in the coronary and peripheral arteries. Volumetric three-dimensional (3D) IVUS visualization provides a powerful tool to overcome the limited comprehensive information of 2D IVUS in terms of complex spatial distribution of arterial morphology and acoustic backscatter information. Conventional 3D IVUS techniques provide sub-optimal visualization of arterial morphology or lack acoustic information concerning arterial structure due in part to low quality of image data and the use of pixel-based IVUS image reconstruction algorithms. In the present study, we describe a novel volumetric 3D IVUS reconstruction algorithm to utilize IVUS signal data and a shape-based nonlinear interpolation.

Methods

We developed an algorithm to convert a series of IVUS signal data into a fully volumetric 3D visualization. Intermediary slices between original 2D IVUS slices were generated utilizing the natural cubic spline interpolation to consider the nonlinearity of both vascular structure geometry and acoustic backscatter in the arterial wall. We evaluated differences in image quality between the conventional pixel-based interpolation and the shape-based nonlinear interpolation methods using both virtual vascular phantom data and in vivo IVUS data of a porcine femoral artery. Volumetric 3D IVUS images of the arterial segment reconstructed using the two interpolation methods were compared.

Results

In vitro validation and in vivo comparative studies with the conventional pixel-based interpolation method demonstrated more robustness of the shape-based nonlinear interpolation algorithm in determining intermediary 2D IVUS slices. Our shape-based nonlinear interpolation demonstrated improved volumetric 3D visualization of the in vivo arterial structure and more realistic acoustic backscatter distribution compared to the conventional pixel-based interpolation method.

Conclusions

This novel 3D IVUS visualization strategy has the potential to improve ultrasound imaging of vascular structure information, particularly atheroma determination. Improved volumetric 3D visualization with accurate acoustic backscatter information can help with ultrasound molecular imaging of atheroma component distribution.

Combined image processing techniques for characterization of MRI cartilage of the knee

A common manifestation of osteoarthritis (OA) of the knee is the morphological degeneration of articular cartilage. In vivo magnetic resonance imaging (MRI) offers the potential to visualize and analyze quantitatively morphology such as cartilage thickness and volume. The purpose of this work was the development of new image processing techniques and application of existing ones for the intra and inter-subject quantitative analysis of cartilage of the knee. The process consists of MRI acquisition, cartilage segmentation, shape-based interpolation of segmented cartilage, segmentation of bone, volume registration based on bone structures, analysis, and visualization. The process is semi-automatic, the segmentation which is based on Bezier splines and edge detection requires interaction. Different shape interpolation methods were compared. The registration is based on shape matching and can be rigid-body and elastic. The analysis comprises cartilage volume and thickness calculations. The visualization allows the depiction of cartilage thickness maps overlaid on MR images or in three dimensions (3D). The cartilage segmentation and shape-based interpolation techniques were validated visually and based on the volumetric measurements of images of porcine knees which cartilage volume were directly measured using a saline displacement method. The registration technique was validated visually and using manual landmark registration.

Automated liver volumetry in orthotopic liver transplantation using multiphase acquisitions on MDCT

OBJECTIVE

The aim of this study was to test a new automated hepatic volumetry technique by comparing the accuracies and postprocessing times of manual and automated liver volume segmentation methods in a patient population undergoing orthotopic liver transplantation so that liver volume could be determined on pathology as the standard of reference.

CONCLUSION

Both manual and automated multiphase MDCT-based volume measurements were strongly correlated to liver volume (Pearson correlation coefficient, r = 0.87 [p < 0.0001] and 0.90 [p < 0.0001], respectively). Automated multiphase segmentation was significantly more rapid than manual segmentation (mean time, 16 ± 5 [SD] and 86 ± 3 seconds, respectively; p = 0.01). Overall, automated liver volumetry based on multiphase CT acquisitions is feasible and more rapid than manual segmentation.

Remodeling after acute myocardial infarction: mapping ventricular dilatation using three dimensional CMR image registration

BACKGROUND

Progressive heart failure due to remodeling is a major cause of morbidity and mortality following myocardial infarction. Conventional clinical imaging measures global volume changes, and currently there is no means of assessing regional myocardial dilatation in relation to ischemic burden. Here we use 3D co-registration of Cardiovascular Magnetic Resonance (CMR) images to assess the long-term effects of ischemia-reperfusion injury on left ventricular structure after acute ST-elevation myocardial infarction (STEMI).

METHODS

Forty six patients (age range 33-77 years) underwent CMR imaging within 7 days following primary percutaneous coronary intervention (PPCI) for acute STEMI with follow-up at one year. Functional cine imaging and Late Gadolinium Enhancement (LGE) were segmented and co-registered. Local left ventricular wall dilatation was assessed by using intensity-based similarities to track the structural changes in the heart between baseline and follow-up. Results are expressed as means, standard errors and 95% confidence interval (CI) of the difference.

RESULTS

Local left ventricular remodeling within infarcted myocardium was greater than in non-infarcted myocardium (1.6%±1.0 vs 0.3%±0.9, 95% CI: -2.4% - -0.2%, P=0.02). One-way ANOVA revealed that transmural infarct thickness had a significant effect on the degree of local remodeling at one year (P<0.0001) with greatest wall dilatation observed when infarct transmurality exceeded 50%. Infarct remodeling was more severe when microvascular obstruction (MVO) was present (3.8%±1.3 vs -1.6%±1.4, 95% CI: -9.1% - -1.5%, P=0.007) and when end-diastolic volume had increased by >20% (4.8%±1.4 vs -0.15%±1.2, 95% CI: -8.9% - -0.9%, P=0.017).

CONCLUSIONS

The severity of ischemic injury has a significant effect on local ventricular wall remodeling with only modest dilatation observed within non-ischemic myocardium. Limitation of chronic remodeling may therefore depend on therapies directed at modulating ischemia-reperfusion injury. CMR co-registration has potential for assessing dynamic changes in ventricular structure in relation to therapeutic interventions.

A comprehensive cardiac motion estimation framework using both untagged and 3-D tagged MR images based on nonrigid registration

In this paper, we present a novel technique based on nonrigid image registration for myocardial motion estimation using both untagged and 3-D tagged MR images. The novel aspect of our technique is its simultaneous usage of complementary information from both untagged and 3-D tagged MR images. To estimate the motion within the myocardium, we register a sequence of tagged and untagged MR images during the cardiac cycle to a set of reference tagged and untagged MR images at end-diastole. The similarity measure is spatially weighted to maximize the utility of information from both images. In addition, the proposed approach integrates a valve plane tracker and adaptive incompressibility into the framework. We have evaluated the proposed approach on 12 subjects. Our results show a clear improvement in terms of accuracy compared to approaches that use either 3-D tagged or untagged MR image information alone. The relative error compared to manually tracked landmarks is less than 15% throughout the cardiac cycle. Finally, we demonstrate the automatic analysis of cardiac function from the myocardial deformation fields.

A Markov random field approach for topology-preserving registration: application to object-based tomographic image interpolation

This paper proposes a topology-preserving multiresolution elastic registration method based on a discrete Markov random field of deformations and a block-matching procedure. The method is applied to the object-based interpolation of tomographic slices. For that purpose, the fidelity of a given deformation to the data is established by a block-matching strategy based on intensity- and gradient-related features, the smoothness of the transformation is favored by an appropriate prior on the field, and the deformation is guaranteed to maintain the topology by imposing some hard constraints on the local configurations of the field. The resulting deformation is defined as the maximum a posteriori configuration. Additionally, the relative influence of the fidelity and smoothness terms is weighted by the unsupervised estimation of the field parameters. In order to obtain an unbiased interpolation result, the registration is performed both in the forward and backward directions, and the resulting transformations are combined by using the local information content of the deformation. The method is applied to magnetic resonance and computed tomography acquisitions of the brain and the torso. Quantitative comparisons offer an overall improvement in performance with respect to related works in the literature. Additionally, the application of the interpolation method to cardiac magnetic resonance images has shown that the removal of any of the main components of the algorithm results in a decrease in performance which has proven to be statistically significant.

Adaptive-weighted cubic B-spline using lookup tables for fast and efficient axial resampling of 3D confocal microscopy images

Confocal laser scanning microscopy has become a most powerful tool to visualize and analyze the dynamic behavior of cellular molecules. Photobleaching of fluorochromes is a major problem with confocal image acquisition that will lead to intensity attenuation. Photobleaching effect can be reduced by optimizing the collection efficiency of the confocal image by fast z-scanning. However, such images suffer from distortions, particularly in the z dimension, which causes disparities in the x, y, and z directions of the voxels with the original image stacks. As a result, reliable segmentation and feature extraction of these images may be difficult or even impossible. Image interpolation is especially needed for the correction of undersampling artifact in the axial plane of three-dimensional images generated by a confocal microscope to obtain cubic voxels. In this work, we present an adaptive cubic B-spline-based interpolation with the aid of lookup tables by deriving adaptive weights based on local gradients for the sampling nodes in the interpolation formulae. Thus, the proposed method enhances the axial resolution of confocal images by improving the accuracy of the interpolated value simultaneously with great reduction in computational cost. Numerical experimental results confirm the effectiveness of the proposed interpolation approach and demonstrate its superiority both in terms of accuracy and speed compared to other interpolation algorithms.

Efficient computation of enclosed volume and surface area from the same triangulated surface representation

We demonstrate that the volume enclosed by triangulated surfaces can be computed efficiently in the same elegant way the volume enclosed by digital surfaces can be computed by digital surface integration. Although digital surfaces are effective and efficient for visualization and volume measurement, their drawback is that surface area measurements derived from them are inaccurate. On the other hand, triangulated surfaces give more accurate surface area measurements, but volume measurements and visualization are less efficient. Our data structure (called t-shell) for representing triangulated digital surfaces retains advantages and overcomes difficulties of both the digital and the triangulated surfaces. We create a lookup table with area and volume contributions for each of the 256 Marching Cubes configurations. When scanning the shell (e.g., while creating it), the surface area and volume are incrementally computed by using the lookup table and the current x co-ordinate, where the sign of the x component of the triangle normal indicates the sign of the volume contribution. We have computed surface area and volume for digitized mathematical phantoms, physical phantoms, and real objects. The experiments show that triangulated surface area is more accurate, triangulated volume follows digital volume closely, and that the values get closer to the true value with decreasing voxel size.

Fusion of multi-planar images for improved three-dimensional object reconstruction

Due to the scan time limitation, our MRI studies of the human tongue can acquire only a limited number of contiguous two-dimensional (2D) slices to form a volumetric data set in a given series. An interpolated three-dimensional (3D) reconstruction using images acquired in a single plane presents artifacts. To address this issue, we developed a wavelet-based bidirectional linear fusion method that uses slices acquired from sagittal and coronal planes to estimate the unknown values of the inter-slice voxels. We use an interpolation method to estimate the voxel value based on neighboring fiducial voxels in the bounding slices. This interpolation is followed by a wavelet fusion to recover image details by integrating prominent coefficients from the interpolated images. Our method was evaluated using 2D MR images and 3D phantoms. Experiments demonstrated that our method reduces interpolation artifacts and greatly improves the 3D reconstruction accuracy. The advantage of our method casts new light on MR imaging and image processing and permits us to achieve high resolution and short acquisition time simultaneously.

Topomorphologic separation of fused isointensity objects via multiscale opening: separating arteries and veins in 3-D pulmonary CT

A novel multiscale topomorphologic approach for opening of two isointensity objects fused at different locations and scales is presented and applied to separating arterial and venous trees in 3-D pulmonary multidetector X-ray computed tomography (CT) images. Initialized with seeds, the two isointensity objects (arteries and veins) grow iteratively while maintaining their spatial exclusiveness and eventually form two mutually disjoint objects at convergence. The method is intended to solve the following two fundamental challenges: how to find local size of morphological operators and how to trace continuity of locally separated regions. These challenges are met by combining fuzzy distance transform (FDT), a morphologic feature with a topologic fuzzy connectivity, and a new morphological reconstruction step to iteratively open finer and finer details starting at large scales and progressing toward smaller scales. The method employs efficient user intervention at locations where local morphological separability assumption does not hold due to imaging ambiguities or any other reason. The approach has been validated on mathematically generated tubular objects and applied to clinical pulmonary noncontrast CT data for separating arteries and veins. The tradeoff between accuracy and the required user intervention for the method has been quantitatively examined by comparing with manual outlining. The experimental study, based on a blind seed selection strategy, has demonstrated that above 95% accuracy may be achieved using 25-40 seeds for each of arteries and veins. Our method is very promising for semiautomated separation of arteries and veins in pulmonary CT images even when there is no object-specific intensity variation at conjoining locations.

Modified control grid interpolation for the volumetric reconstruction of fluid flows

Complex applications in fluid dynamics research often require more highly resolved velocity data than direct measurements or simulations provide. The advent of stereo PIV and PCMR techniques has advanced the state-of-the-art in flow velocity measurement, but 3D spatial resolution remains limited. Here a new technique is proposed for velocity data interpolation to address this problem. The new method performs with higher quality than competing solutions from the literature in terms of accurately interpolating velocities, maintaining fluid structure and domain boundaries, and preserving coherent structures.

An integral conservative gridding--algorithm using Hermitian curve interpolation

The problem of re-sampling spatially distributed data organized into regular or irregular grids to finer or coarser resolution is a common task in data processing. This procedure is known as 'gridding' or 're-binning'. Depending on the quantity the data represents, the gridding-algorithm has to meet different requirements. For example, histogrammed physical quantities such as mass or energy have to be re-binned in order to conserve the overall integral. Moreover, if the quantity is positive definite, negative sampling values should be avoided. The gridding process requires a re-distribution of the original data set to a user-requested grid according to a distribution function. The distribution function can be determined on the basis of the given data by interpolation methods. In general, accurate interpolation with respect to multiple boundary conditions of heavily fluctuating data requires polynomial interpolation functions of second or even higher order. However, this may result in unrealistic deviations (overshoots or undershoots) of the interpolation function from the data. Accordingly, the re-sampled data may overestimate or underestimate the given data by a significant amount. The gridding-algorithm presented in this work was developed in order to overcome these problems. Instead of a straightforward interpolation of the given data using high-order polynomials, a parametrized Hermitian interpolation curve was used to approximate the integrated data set. A single parameter is determined by which the user can control the behavior of the interpolation function, i.e. the amount of overshoot and undershoot. Furthermore, it is shown how the algorithm can be extended to multidimensional grids. The algorithm was compared to commonly used gridding-algorithms using linear and cubic interpolation functions. It is shown that such interpolation functions may overestimate or underestimate the source data by about 10-20%, while the new algorithm can be tuned to significantly reduce these interpolation errors. The accuracy of the new algorithm was tested on a series of x-ray CT-images (head and neck, lung, pelvis). The new algorithm significantly improves the accuracy of the sampled images in terms of the mean square error and a quality index introduced by Wang and Bovik (2002 IEEE Signal Process. Lett. 9 81-4).

Subject-specific models for image-guided cardiac surgery

Three-dimensional visualization for planning and guidance is still not routinely available for minimally invasive cardiac surgery (MICS). This can be addressed by providing the surgeon with subject-specific geometric models derived from 3D preoperative images for planning of port locations or to rehearse the procedure. For guidance purposes, these models can also be registered to the subject using intraoperative images. In this paper, we present a method for extracting subject-specific heart geometry from preoperative MR images. The main obstacle we face is the low quality of clinical data in terms of resolution, signal-to-noise ratio, and presence of artefacts. Instead of using these images directly, we approach the problem in three steps: (1) generate a high quality template model, (2) register the template with the preoperative data, and (3) animate the result over the cardiac cycle. Validation of this approach showed that dynamic subject-specific models can be generated with a mean error of 3.6+/-1.1 mm from low resolution target images (6 mm slices). Thus, the models are sufficiently accurate for MICS training and procedure planning. In terms of guidance, we also demonstrate how the resulting models may be adapted to the operating room using intraoperative ultrasound imaging.

A new method for registration-based medical image interpolation

A new technique is presented for interpolating between grey-scale images in a medical data set. Registration between neighboring slices is achieved with a modified control grid interpolation algorithm that selectively accepts displacement field updates in a manner optimized for performance. A cubic interpolator is then applied to pixel intensities correlated by the displacement fields. Special considerations are made for efficiency, interpolation quality, and compression in the implementation of the algorithm. Experimental results show that the new method achieves good quality, while offering dramatic improvement in efficiency relative to the best competing method.

Optical flow based volumetric spatio-temporal interpolation

Acquisition of three-dimensional medical images with excellent spatial resolution is achieved almost always at the cost-dictating expense of increased acquisition time. This can be mitigated using shape-respecting inter-slice interpolation of sparsely acquired datasets. Existing shape and registration based interpolation techniques are suboptimal. This paper proposes a new optical flow based approach to interpolate between sparse data. The Horn and Schunck technique is used to estimate the pixel-wise flow vectors across the slices; intermediate data planes can be reconstructed from these flow vectors. The proposed technique is accurate and computationally fast. Coupled with anatomy aware sparse acquisitions, the proposed technique shows promise of judiciously managing the patient-specific acquisition.

Registration and machine learning-based automated segmentation of subcortical and cerebellar brain structures

The large amount of imaging data collected in several ongoing multi-center studies requires automated methods to delineate brain structures of interest. We have previously reported on using artificial neural networks (ANN) to define subcortical brain structures. Here we present several automated segmentation methods using multidimensional registration. A direct comparison between template, probability, artificial neural network (ANN) and support vector machine (SVM)-based automated segmentation methods is presented. Three metrics for each segmentation method are reported in the delineation of subcortical and cerebellar brain regions. Results show that the machine learning methods outperform the template and probability-based methods. Utilization of these automated segmentation methods may be as reliable as manual raters and require no rater intervention.

CAVASS: a computer-assisted visualization and analysis software system

The Medical Image Processing Group at the University of Pennsylvania has been developing (and distributing with source code) medical image analysis and visualization software systems for a long period of time. Our most recent system, 3DVIEWNIX, was first released in 1993. Since that time, a number of significant advancements have taken place with regard to computer platforms and operating systems, networking capability, the rise of parallel processing standards, and the development of open-source toolkits. The development of CAVASS by our group is the next generation of 3DVIEWNIX. CAVASS will be freely available and open source, and it is integrated with toolkits such as Insight Toolkit and Visualization Toolkit. CAVASS runs on Windows, Unix, Linux, and Mac but shares a single code base. Rather than requiring expensive multiprocessor systems, it seamlessly provides for parallel processing via inexpensive clusters of work stations for more time-consuming algorithms. Most importantly, CAVASS is directed at the visualization, processing, and analysis of 3-dimensional and higher-dimensional medical imagery, so support for digital imaging and communication in medicine data and the efficient implementation of algorithms is given paramount importance.

Polygon interpolation for serial cross sections

In this paper, a new technique for contour interpolation between slices is presented. We assumed that contour interpolation is equivalent to the interpolation of a polygon that approximates the object shape. The location of each polygon vertex is characterized by a set of parameters. Polygon interpolation can be performed on these parameters. These interpolated parameters are then used to reconstruct the vertices of the new polygon. Finally, the contour is approximated from this polygon using a cubic spline interpolation. This new technique takes into account the shape, the translation, the size, and the orientation of the object's contours. A comparison with regular shape-based interpolation is made on several object contours. The preliminary results show that this new method yields a better contour and is computationally more efficient than shape-based interpolation. This technique can be applied to gray-level images too. The interpolation result of an MR image does not show artifact of intermediate substance commonly seen in a typical linear gray-level interpolation.

Inter-subject comparison of MRI knee cartilage thickness

In this paper, we present the development and application of current image processing techniques to perform MRI inter-subject comparison of knee cartilage thickness based on the registration of bone structures. Each point in the bone surface which is part of the bone-cartilage interface is assigned a cartilage thickness value. Cartilage and corresponding bone structures are segmented and their shapes interpolated to create isotropic voxels. Cartilage thicknesses are computed for each point in the bone-cartilage interfaces and transferred to the bone surfaces. Corresponding anatomic points are then computed for bone surfaces based on shape matching using 3D shape descriptors called shape contexts to register bones with affine and elastic transformations, and then perform a point to point comparison of cartilage thickness values. An alternative technique for cartilage shape interpolation using a morphing technique is also presented. The cartilage segmentation and morphing were validated visually, based on volumetric measurements of porcine knee images which cartilage volumes were measured using a water displacement method, and based on digital thickness values computed with an established technique. Shape matching using 3D shape contexts was validated visually and against manual shape matching performed by a radiologist. The reproducibility of intra- and inter-subject cartilage thickness comparisons was established, as well as the feasibility of using the proposed technique to build a mean femoral shape, cartilage thickness map, and cartilage coverage map. Results showed that the proposed technique is robust, accurate, and reproducible to perform point to point inter-subject comparison of knee cartilage thickness values.

High-quality two-stage resampling for 3-D volumes in medical imaging

This paper introduces a simple method of two-stage resampling where Fourier-domain up-sampling is followed by traditional resampling. Practical aspects as well as efficient implementation techniques are considered. A new version of pruned FFT algorithms to calculate the up-sampling stage is also introduced. The suggested two-stage resampling method provides very high-quality results exceeding those of the previous algorithms. It excels with higher dimensional datasets due to its ability to employ small-support kernels. The applied FFT algorithms make the method most efficient with dataset sizes of powers of two. These reasons and the importance of minimal resampling artifacts make the suggested method especially suitable for 3-D volumes in medical imaging. Furthermore, for repeated uses, only the second stage is recalculated allowing an increase in performance for motion correction applications in functional magnetic resonance imaging (fMRI), for example.

A new orientation-adaptive interpolation method

We propose an isophote-oriented, orientation-adaptive interpolation method. The proposed method employs an interpolation kernel that adapts to the local orientation of isophotes, and the pixel values are obtained through an oriented, bilinear interpolation. We show that, by doing so, the curvature of the interpolated isophotes is reduced, and, thus, zigzagging artifacts are largely suppressed. Analysis and experiments show that images interpolated using the proposed method are visually pleasing and almost artifact free.

Comparison of scene-based interpolation methods applied to CT abdominal images

Three-dimensional (3-D) interpolation from 2-D image slices is widely used to aid the display, analysis and other biomedical image processing. We investigate the performance of 5 scene-based interpolation methods: linear, cubic spline, modified cubic spline and sine-based functions (Dirichlet apodization and Hanning apodization). We test our methods on four sets of computed tomography (CT) abdominal images, which have more organs in them compared to other biomedical images. Results show that, contrary to the 1-D or 2-D cases, linear interpolation acts as well as, even slightly better than all the other methods in the sense of signal to noise ratio in most cases, while the computational load of linear interpolation is only about half of the other methods. The reason for the relative high performance of linear interpolation is probably the large distance between consecutive images, which indicates low inter-slice correlation.

An optical flow based method for improved reconstruction of 4D CT data sets acquired during free breathing

Respiratory motion degrades anatomic position reproducibility and leads to issues affecting image acquisition, treatment planning, and radiation delivery. Four-dimensional (4D) computer tomography (CT) image acquisition can be used to measure the impact of organ motion and to explicitly account for respiratory motion during treatment planning and radiation delivery. Modern CT scanners can only scan a limited region of the body simultaneously and patients have to be scanned in segments consisting of multiple slices. A respiratory signal (spirometer signal or surface tracking) is used to reconstruct a 4D data set by sorting the CT scans according to the couch position and signal coherence with predefined respiratory phases. But artifacts can occur if there are no acquired data segments for exactly the same respiratory state for all couch positions. These artifacts are caused by device-dependent limitations of gantry rotation, image reconstruction times and by the variability of the patient's respiratory pattern. In this paper an optical flow based method for improved reconstruction of 4D CT data sets from multislice CT scans is presented. The optical flow between scans at neighboring respiratory states is estimated by a non-linear registration method. The calculated velocity field is then used to reconstruct a 4D CT data set by interpolating data at exactly the predefined respiratory phase. Our reconstruction method is compared with the usually used reconstruction based on amplitude sorting. The procedures described were applied to reconstruct 4D CT data sets for four cancer patients and a qualitative and quantitative evaluation of the optical flow based reconstruction method was performed. Evaluation results show a relevant reduction of reconstruction artifacts by our technique. The reconstructed 4D data sets were used to quantify organ displacements and to visualize the abdominothoracic organ motion.

Shape-based averaging

A new method for averaging multidimensional images is presented, which is based on signed Euclidean distance maps computed for each of the pixel values. We refer to the algorithm as "shape-based averaging" (SBA) because of its similarity to Raya and Udupa's shape-based interpolation method. The new method does not introduce pixel intensities that were not present in the input data, which makes it suitable for averaging nonnumerical data such as label maps (segmentations). Using segmented human brain magnetic resonance images, SBA is compared to label voting for the purpose of averaging image segmentations in a multiclassifier fashion. SBA, on average, performed as well as label voting in terms of recognition rates of the averaged segmentations. SBA produced more regular and contiguous structures with less fragmentation than did label voting. SBA also was more robust for small numbers of atlases and for low atlas resolutions, in particular, when combined with shape-based interpolation. We conclude that SBA improves the contiguity and accuracy of averaged image segmentations.