Multi-modality image fusion based on enhanced fuzzy radial basis function neural networks

A New General Maximum Intensity Projection Technology via the Hybrid of U-Net and Radial Basis Function Neural Network

Maximum intensity projection (MIP) technology is a computer visualization method that projects three-dimensional spatial data on a visualization plane. According to the specific purposes, the specific lab thickness and direction can be selected. This technology can better show organs, such as blood vessels, arteries, veins, and bronchi and so forth, from different directions, which could bring more intuitive and comprehensive results for doctors in the diagnosis of related diseases. However, in this traditional projection technology, the details of the small projected target are not clearly visualized when the projected target is not much different from the surrounding environment, which could lead to missed diagnosis or misdiagnosis. Therefore, it is urgent to develop a new technology that can better and clearly display the angiogram. However, to the best of our knowledge, research in this area is scarce. To fill this gap in the literature, in the present study, we propose a new method based on the hybrid of convolutional neural network (CNN) and radial basis function neural network (RBFNN) to synthesize the projection image. We first adopted the U-net to obtain feature or enhanced images to be projected; subsequently, the RBF neural network performed further synthesis processing for these data; finally, the projection images were obtained. For experimental data, in order to increase the robustness of the proposed algorithm, the following three different types of datasets were adopted: the vascular projection of the brain, the bronchial projection of the lung parenchyma, and the vascular projection of the liver. In addition, radiologist evaluation and five classic metrics of image definition were implemented for effective analysis. Finally, compared to the traditional MIP technology and other structures, the use of a large number of different types of data and superior experimental results proved the versatility and robustness of the proposed method.

Multi-Modal Medical Image Fusion Based on FusionNet in YIQ Color Space

In order to obtain the physiological information and key features of source images to the maximum extent, improve the visual effect and clarity of the fused image, and reduce the computation, a multi-modal medical image fusion framework based on feature reuse is proposed. The framework consists of intuitive fuzzy processing (IFP), capture image details network (CIDN), fusion, and decoding. First, the membership function of the image is redefined to remove redundant features and obtain the image with complete features. Then, inspired by DenseNet, we proposed a new encoder to capture all the medical information features in the source image. In the fusion layer, we calculate the weight of each feature graph in the required fusion coefficient according to the trajectory of the feature graph. Finally, the filtered medical information is spliced and decoded to reproduce the required fusion image. In the encoding and image reconstruction networks, the mixed loss function of cross entropy and structural similarity is adopted to greatly reduce the information loss in image fusion. To assess performance, we conducted three sets of experiments on medical images of different grayscales and colors. Experimental results show that the proposed algorithm has advantages not only in detail and structure recognition but also in visual features and time complexity compared with other algorithms.

A diffeomorphic unsupervised method for deformable soft tissue image registration

BACKGROUND AND OBJECTIVES

The image registration methods for deformable soft tissues utilize nonlinear transformations to align a pair of images precisely. In some situations, when there is huge gray scale difference or large deformation between the images to be registered, the deformation field tends to fold at some local voxels, which will result in the breakdown of the one-to-one mapping between images and the reduction of invertibility of the deformation field. In order to address this issue, a novel registration approach based on unsupervised learning is presented for deformable soft tissue image registration.

METHODS

A novel unsupervised learning based registration approach, which consists of a registration network, a velocity field integration module and a grid sampling module, is presented for deformable soft tissue image registration. The main contributions are: (1) A novel encoder-decoder network is presented for the evaluation of stationary velocity field. (2) A Jacobian determinant based penalty term (Jacobian loss) is developed to reduce the folding voxels and to improve the invertibility of the deformation field.

RESULTS AND CONCLUSIONS

The experimental results show that a new pair of images can be accurately registered using the trained registration model. In comparison with the conventional state-of-the-art method, SyN, the invertibility of the deformation field, accuracy and speed are all improved. Compared with the deep learning based method, VoxelMorph, the proposed method improves the invertibility of the deformation field.

Slice interpolation of medical images using enhanced fuzzy radial basis function neural networks

Volume data composed of complete slice images play an indispensable role in medical diagnoses. However, system or human factors often lead to the loss of slice images. In recent years, various interpolation algorithms have been proposed to solve these problems. Although these algorithms are effective, the interpolated images have some shortcomings, such as less accurate recovery and missing details. In this study, we propose a new method based on an enhanced fuzzy radial basis function neural network to improve the performance of the interpolation method. The neural network includes an input layer (six input neurons), three hidden layers of neurons, and the output layer (one output neuron), and we propose a patch matching method to select the input variables of the neural network. Accordingly, we use two normal pending images to be interpolated as the input. Final output data is obtained by applying the trained neural network. In examining four groups of medical images, the proposed method outperforms five other methods, achieving the highest similarity image metric (ESSIM) values of 0.96, 0.95, 0.94, and 0.92 and the lowest mean squared difference (MSD) values of 35.5, 41.2, 50.9, and 47.1. In addition, for a whole MRI brain volume data experiment, the average MSD and ESSIM values of the proposed method and other methods are (41.62, 0.95) and (57.13, 0.90), respectively. The results indicate that the proposed method is superior to the other methods.