MRI and PET image fusion using the nonparametric density model and the theory of variable-weight

Pareto Optimized Adaptive Learning with Transposed Convolution for Image Fusion Alzheimer's Disease Classification

Alzheimer's disease (AD) is a neurological condition that gradually weakens the brain and impairs cognition and memory. Multimodal imaging techniques have become increasingly important in the diagnosis of AD because they can help monitor disease progression over time by providing a more complete picture of the changes in the brain that occur over time in AD. Medical image fusion is crucial in that it combines data from various image modalities into a single, better-understood output. The present study explores the feasibility of employing Pareto optimized deep learning methodologies to integrate Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) images through the utilization of pre-existing models, namely the Visual Geometry Group (VGG) 11, VGG16, and VGG19 architectures. Morphological operations are carried out on MRI and PET images using Analyze 14.0 software and after which PET images are manipulated for the desired angle of alignment with MRI image using GNU Image Manipulation Program (GIMP). To enhance the network's performance, transposed convolution layer is incorporated into the previously extracted feature maps before image fusion. This process generates feature maps and fusion weights that facilitate the fusion process. This investigation concerns the assessment of the efficacy of three VGG models in capturing significant features from the MRI and PET data. The hyperparameters of the models are tuned using Pareto optimization. The models' performance is evaluated on the ADNI dataset utilizing the Structure Similarity Index Method (SSIM), Peak Signal-to-Noise Ratio (PSNR), Mean-Square Error (MSE), and Entropy (E). Experimental results show that VGG19 outperforms VGG16 and VGG11 with an average of 0.668, 0.802, and 0.664 SSIM for CN, AD, and MCI stages from ADNI (MRI modality) respectively. Likewise, an average of 0.669, 0.815, and 0.660 SSIM for CN, AD, and MCI stages from ADNI (PET modality) respectively.

Multimodal fusion of tomographic sequences of medical images: MRE spatially enhanced by MRI

BACKGROUND AND OBJECTIVE

In biomedical fields, image analysis is often necessary for an accurate diagnosis. In order to obtain all the information needed to form an in-depth clinical picture, it may be useful to combine the contents of images taken under different diagnostic modes. Multimodal medical image fusion techniques enable complementary information acquired by different imaging devices to be automatically combined into a unique image.

METHODS

In this paper, multimodal medical images fusion method based on multiresolution analysis (MRA) is proposed, with the aim to combine the high geometric content of magnetic resonance imaging (MRI) and the elasticity information of magnetic resonance elastography (MRE), simultaneously acquired on the same organs of a patient. First, the slices of MRE are volumetrically interpolated to exactly overlap, each with a slice of MRI. Then, the spatial details of MRI are extracted by means of MRA and injected into the corresponding slices of MRE. Due to the intrinsic dissimilarity between corresponding slices of MRE and MRI, the spatial details of MRI are modulated by local or global matching functions.

RESULTS

The performance of the proposed method is quantitatively assessed considering radiometric and geometric consistency of the fused images with respect to their originals, in a comparison with two popular methods from the literature. For a qualitative evaluation, a visual inspection is carried out.

CONCLUSIONS

The results show that the proposed method enables an effective MRI-MRE fusion that allows the elasticity information and geometric details of the examined organs to be evaluated in a single image.

Multimodal Medical Image Fusion of Positron Emission Tomography and Magnetic Resonance Imaging Using Generative Adversarial Networks

Multimodal medical image fusion is a current technique applied in the applications related to medical field to combine images from the same modality or different modalities to improve the visual content of the image to perform further operations like image segmentation. Biomedical research and medical image analysis highly demand medical image fusion to perform higher level of medical analysis. Multimodal medical fusion assists medical practitioners to visualize the internal organs and tissues. Multimodal medical fusion of brain image helps to medical practitioners to simultaneously visualize hard portion like skull and soft portion like tissue. Brain tumor segmentation can be accurately performed by utilizing the image obtained after multimodal medical image fusion. The area of the tumor can be accurately located with the information obtained from both Positron Emission Tomography and Magnetic Resonance Image in a single fused image. This approach increases the accuracy in diagnosing the tumor and reduces the time consumed in diagnosing and locating the tumor. The functional information of the brain is available in the Positron Emission Tomography while the anatomy of the brain tissue is available in the Magnetic Resonance Image. Thus, the spatial characteristics and functional information can be obtained from a single image using a robust multimodal medical image fusion model. The proposed approach uses a generative adversarial network to fuse Positron Emission Tomography and Magnetic Resonance Image into a single image. The results obtained from the proposed approach can be used for further medical analysis to locate the tumor and plan for further surgical procedures. The performance of the GAN based model is evaluated using two metrics, namely, structural similarity index and mutual information. The proposed approach achieved a structural similarity index of 0.8551 and a mutual information of 2.8059.

Gaussian Filter for Brain SPECT Imaging

Background. The presence of a noise component on 3D images of single-photon emission computed tomo­graphy (SPECT) of a brain significantly distorts the probability distribution function (PD) of the radioactive count rate in the images. The presence of noise and further filtering of the data, based on a subjective assessment of image quality, have a significant impact on the calculation of volumetric cerebral blood flow and the values of the uptake asymmetry of the radiopharmaceutical in a brain. Objective. We are aimed to develop a method for optimal SPECT filtering of brain images with lipophilic radiopharmaceuticals, based on a Gaussian filter (GF), for subsequent image segmentation by the threshold method.  Methods. SPECT images of the water phantom and the brain of patients with 99mTc-HMPAO were used. We have developed a technique for artificial addition of speckle noise to conditionally flawless data in order to determine the optimal parameters for smoothing SPECT, based on a GF. The quantitative criterion for optimal smoothing was the standard deviation between the PD of radioactive count rate of the smoothed image and conditionally ideal one. Results. It was shown that the maximum radioactive count rate of the SPECT image has an extremum by changing the standard deviation of the GF in the range of 0.3–0.4 pixels. The greater the noise component in the SPECT image, the more quasi-linearly the corresponding rate changes. This dependence allows determining the optimal smoothing parameters. The application of the developed smoothing technique allows restoring the probability distribution function of the radioactive count rate (distribution histogram) with an accuracy up to 5–10%. This provides the possibility to standardize SPECT images of brain. Conclusions. The research results of work solve a specific applied problem: restoration of the histogram of a radiopharmaceuticals distribution in a brain for correct quantitative assessment of regional cerebral blood flow. In contrast to the well-known publications on the filtration of SPECT data, the work takes into account that the initial tomographic data are 3D, rather than 2D slices, and contain not only uniform random Gaussian noise, but also a pronounced speckle component.

Two-Scale Multimodal Medical Image Fusion Based on Structure Preservation

Medical image fusion has an indispensable value in the medical field. Taking advantage of structure-preserving filter and deep learning, a structure preservation-based two-scale multimodal medical image fusion algorithm is proposed. First, we used a two-scale decomposition method to decompose source images into base layer components and detail layer components. Second, we adopted a fusion method based on the iterative joint bilateral filter to fuse the base layer components. Third, a convolutional neural network and local similarity of images are used to fuse the components of the detail layer. At the last, the final fused result is got by using two-scale image reconstruction. The contrast experiments display that our algorithm has better fusion results than the state-of-the-art medical image fusion algorithms.

Magnetic Resonance Image under Variable Model Algorithm in Diagnosis of Patients with Spinal Metastatic Tumors

The aim of this study was to explore the adoption of the variable model algorithm in magnetic resonance imaging (MRI) image analysis and evaluate the effect of the algorithm-based MRI in the diagnosis of spinal metastatic tumor diseases. 100 patients with spinal metastatic tumors who were treated in hospital were recruited as the research objects. All patients were randomly divided into the experimental group (MRI image analysis based on variable model) and the control group (conventional MRI image diagnosis), and the MRI of the experimental group was segmented using the conventional algorithm with variable model and the improved algorithm with GVF force field. The accuracy index (Dice coefficient D) values were used to evaluate the vertebral segmentation effect of the improved variable model algorithm with the introduction of GVF force field, and the recognition rate, sensitivity, and specificity indexes were used to evaluate the effects of the two algorithms on the recognition of MRI image features of spinal metastatic tumors. The results showed that the mean D value of the variable model improvement algorithm for the segmentation of five vertebrae of spinal metastatic tumors was significantly improved relative to the conventional variable model algorithm, and the difference was statistically significant (P < 0.05). At the number of 80 iterations, the recognition rate, sensitivity, and specificity of MRI image segmentation of the traditional variable model algorithm processing group were 89.32%, 74.88%, and 86.27%, respectively, while the recognition rate, sensitivity, and specificity of MRI image segmentation of the variable model improvement algorithm processing group were 97.89%, 96.75%, and 96.45%, respectively. The results of the latter were significantly better than those of the former, and the differences were statistically significant (P < 0.05); and the comparison of MRI images showed that the variable model improvement algorithm was more rapid and accurate in identifying the focal sites of patients with spinal metastases. The accuracy of MRI images based on the variable model algorithm increased from 69.5% to 92%, and the difference was statistically significant (P < 0.05). In short, MRI image analysis based on the variable model algorithm had great adoption potential in the clinical diagnosis of spinal metastatic tumors and was worthy of clinical promotion.

An adaptive two-scale biomedical image fusion method with statistical comparisons

Two-scale image representation of base and detail in the spatial-domain is a well-known decomposition scheme for its lower computational complexity than that performed in the transform-domain in the field of image fusion. Unfortunately, for a pseudo-colour input image, the base and detail images in the spatial-domain obtained via image decomposition scheme always display in greyscale. In this paper, a two-scale image fusion method with adaptive threshold obtained by Otsu's method is proposed for pseudo-colour image in the colour space domain. For greyscale image, detail and base image are obtained using structural information extracted from the difference image between a global and a local patch size. Consequently, local edge-preserving filter for preserving luminance information and local energy with the discussed window size are adopted to combine base and detail image. Experimental results show that structural and luminance information has been better preserved in terms of subjective and objective evaluations for medical image and protein image fusion. Specially, a two-step non-parametric statistical test (Friedman test and Nemenyi post-hoc test) with p-values is adopted to analysis the statistical significant of the relative difference between the proposed and compared methods in terms of values of objective metrics including 30 co-registered pairs of imaging data.

A diffeomorphic unsupervised method for deformable soft tissue image registration

BACKGROUND AND OBJECTIVES

The image registration methods for deformable soft tissues utilize nonlinear transformations to align a pair of images precisely. In some situations, when there is huge gray scale difference or large deformation between the images to be registered, the deformation field tends to fold at some local voxels, which will result in the breakdown of the one-to-one mapping between images and the reduction of invertibility of the deformation field. In order to address this issue, a novel registration approach based on unsupervised learning is presented for deformable soft tissue image registration.

METHODS

A novel unsupervised learning based registration approach, which consists of a registration network, a velocity field integration module and a grid sampling module, is presented for deformable soft tissue image registration. The main contributions are: (1) A novel encoder-decoder network is presented for the evaluation of stationary velocity field. (2) A Jacobian determinant based penalty term (Jacobian loss) is developed to reduce the folding voxels and to improve the invertibility of the deformation field.

RESULTS AND CONCLUSIONS

The experimental results show that a new pair of images can be accurately registered using the trained registration model. In comparison with the conventional state-of-the-art method, SyN, the invertibility of the deformation field, accuracy and speed are all improved. Compared with the deep learning based method, VoxelMorph, the proposed method improves the invertibility of the deformation field.