1
|
Han Q, Jung C. Deep Selective Fusion of Visible and Near-Infrared Images Using Unsupervised U-Net. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2025; 36:4172-4183. [PMID: 35100123 DOI: 10.1109/tnnls.2022.3142780] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
In low light conditions, visible (VIS) images are of a low dynamic range (low contrast) with severe noise and color, while near-infrared (NIR) images contain clear textures without noise and color. Multispectral fusion of VIS and NIR images produces color images of high quality, rich textures, and little noise by taking both advantages of VIS and NIR images. In this article, we propose the deep selective fusion of VIS and NIR images using unsupervised U-Net. Existing image fusion methods are afflicted with the low contrast in VIS images and flash-like effect in NIR images. Thus, we adopt unsupervised U-Net to achieve deep selective fusion of multiple scale features. Due to the absence of the ground truth, we use unsupervised learning by formulating an energy function as a loss function. To deal with insufficient training data, we perform data augmentation by rotating images and adjusting their intensity. We synthesize training data by degrading clean VIS images and masking clean NIR images using a circle. First, we utilize pretrained visual geometry group (VGG) to extract features from VIS images. Second, we build an encoding network to obtain edge information from NIR images. Finally, we combine all features and feed them into a decoding network for fusion. Experimental results demonstrate that the proposed fusion network produces visually pleasing results with fine details, little noise, and natural color and it is superior to state-of-the-art methods in terms of visual quality and quantitative measurements.
Collapse
|
2
|
Li H, Yuan M, Li J, Liu Y, Lu G, Xu Y, Yu Z, Zhang D. Focus Affinity Perception and Super-Resolution Embedding for Multifocus Image Fusion. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2025; 36:4311-4325. [PMID: 38446648 DOI: 10.1109/tnnls.2024.3367782] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/08/2024]
Abstract
Despite the fact that there is a remarkable achievement on multifocus image fusion, most of the existing methods only generate a low-resolution image if the given source images suffer from low resolution. Obviously, a naive strategy is to independently conduct image fusion and image super-resolution. However, this two-step approach would inevitably introduce and enlarge artifacts in the final result if the result from the first step meets artifacts. To address this problem, in this article, we propose a novel method to simultaneously achieve image fusion and super-resolution in one framework, avoiding step-by-step processing of fusion and super-resolution. Since a small receptive field can discriminate the focusing characteristics of pixels in detailed regions, while a large receptive field is more robust to pixels in smooth regions, a subnetwork is first proposed to compute the affinity of features under different types of receptive fields, efficiently increasing the discriminability of focused pixels. Simultaneously, in order to prevent from distortion, a gradient embedding-based super-resolution subnetwork is also proposed, in which the features from the shallow layer, the deep layer, and the gradient map are jointly taken into account, allowing us to get an upsampled image with high resolution. Compared with the existing methods, which implemented fusion and super-resolution independently, our proposed method directly achieves these two tasks in a parallel way, avoiding artifacts caused by the inferior output of image fusion or super-resolution. Experiments conducted on the real-world dataset substantiate the superiority of our proposed method compared with state of the arts.
Collapse
|
3
|
Chen Y, Liu A, Liu Y, He Z, Liu C, Chen X. Multi-Dimensional Medical Image Fusion With Complex Sparse Representation. IEEE Trans Biomed Eng 2024; 71:2728-2739. [PMID: 38652633 DOI: 10.1109/tbme.2024.3391314] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/25/2024]
Abstract
In the field of medical imaging, the fusion of data from diverse modalities plays a pivotal role in advancing our understanding of pathological conditions. Sparse representation (SR), a robust signal modeling technique, has demonstrated noteworthy success in multi-dimensional (MD) medical image fusion. However, a fundamental limitation appearing in existing SR models is their lack of directionality, restricting their efficacy in extracting anatomical details from different imaging modalities. To tackle this issue, we propose a novel directional SR model, termed complex sparse representation (ComSR), specifically designed for medical image fusion. ComSR independently represents MD signals over directional dictionaries along specific directions, allowing precise analysis of intricate details of MD signals. Besides, current studies in medical image fusion mostly concentrate on addressing either 2D or 3D fusion problems. This work bridges this gap by proposing a MD medical image fusion method based on ComSR, presenting a unified framework for both 2D and 3D fusion tasks. Experimental results across six multi-modal medical image fusion tasks, involving 93 pairs of 2D source images and 20 pairs of 3D source images, substantiate the superiority of our proposed method over 11 state-of-the-art 2D fusion methods and 4 representative 3D fusion methods, in terms of both visual quality and objective evaluation.
Collapse
|
4
|
Zhao F, Zhao W, Lu H. Interactive Feature Embedding for Infrared and Visible Image Fusion. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:12810-12822. [PMID: 37040245 DOI: 10.1109/tnnls.2023.3264911] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
General deep learning-based methods for infrared and visible image fusion rely on the unsupervised mechanism for vital information retention by utilizing elaborately designed loss functions. However, the unsupervised mechanism depends on a well-designed loss function, which cannot guarantee that all vital information of source images is sufficiently extracted. In this work, we propose a novel interactive feature embedding in a self-supervised learning framework for infrared and visible image fusion, attempting to overcome the issue of vital information degradation. With the help of a self-supervised learning framework, hierarchical representations of source images can be efficiently extracted. In particular, interactive feature embedding models are tactfully designed to build a bridge between self-supervised learning and infrared and visible image fusion learning, achieving vital information retention. Qualitative and quantitative evaluations exhibit that the proposed method performs favorably against state-of-the-art methods.
Collapse
|
5
|
Xiang W, Shen J, Zhang L, Zhang Y. Infrared and Visual Image Fusion Based on a Local-Extrema-Driven Image Filter. SENSORS (BASEL, SWITZERLAND) 2024; 24:2271. [PMID: 38610482 PMCID: PMC11014052 DOI: 10.3390/s24072271] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/02/2024] [Revised: 03/30/2024] [Accepted: 04/01/2024] [Indexed: 04/14/2024]
Abstract
The objective of infrared and visual image fusion is to amalgamate the salient and complementary features of the infrared and visual images into a singular informative image. To accomplish this, we introduce a novel local-extrema-driven image filter designed to effectively smooth images by reconstructing pixel intensities based on their local extrema. This filter is iteratively applied to the input infrared and visual images, extracting multiple scales of bright and dark feature maps from the differences between continuously filtered images. Subsequently, the bright and dark feature maps of the infrared and visual images at each scale are fused using elementwise-maximum and elementwise-minimum strategies, respectively. The two base images, representing the final-scale smoothed images of the infrared and visual images, are fused using a novel structural similarity- and intensity-based strategy. Finally, our fusion image can be straightforwardly produced by combining the fused bright feature map, dark feature map, and base image together. Rigorous experimentation conducted on the widely used TNO dataset underscores the superiority of our method in fusing infrared and visual images. Our approach consistently performs on par or surpasses eleven state-of-the-art image-fusion methods, showcasing compelling results in both qualitative and quantitative assessments.
Collapse
Affiliation(s)
- Wenhao Xiang
- Department of Electronic Engineering, Tsinghua University, Beijing 100084, China; (W.X.); (J.S.); (L.Z.)
| | - Jianjun Shen
- Department of Electronic Engineering, Tsinghua University, Beijing 100084, China; (W.X.); (J.S.); (L.Z.)
| | - Li Zhang
- Department of Electronic Engineering, Tsinghua University, Beijing 100084, China; (W.X.); (J.S.); (L.Z.)
| | - Yu Zhang
- School of Astronautics, Beihang University, Beijing 102206, China
| |
Collapse
|
6
|
Behrouzi Y, Basiri A, Pourgholi R, Kiaei AA. Fusion of medical images using Nabla operator; Objective evaluations and step-by-step statistical comparisons. PLoS One 2023; 18:e0284873. [PMID: 37585476 PMCID: PMC10431637 DOI: 10.1371/journal.pone.0284873] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Accepted: 04/11/2023] [Indexed: 08/18/2023] Open
Abstract
Since vectors include direction and magnitude, they have more information than scalars. So, converting the scalar images into the vector field leads achieving much information about the images that have been hidden in the spatial domain. In this paper, the proposed method fuses images after transforming the scalar field of images to a vector one. To transform the field, it uses Nabla operator. After that, the inverse transform is implemented to reconstruct the fused medical image. To show the performance of the proposed method and to evaluate it, different experiments and statistical comparisons were accomplished. Comparing the experimental results with the previous works, shows the effectiveness of the proposed method.
Collapse
Affiliation(s)
- Yasin Behrouzi
- School of Mathematics and Computer Science, Damghan University, Damghan, Iran
| | - Abdolali Basiri
- School of Mathematics and Computer Science, Damghan University, Damghan, Iran
| | - Reza Pourgholi
- School of Mathematics and Computer Science, Damghan University, Damghan, Iran
| | - Ali Akbar Kiaei
- Department of Computer Engineering, Bu-ali Sina University, Hamedan, Iran
| |
Collapse
|
7
|
Fu J, He B, Yang J, Liu J, Ouyang A, Wang Y. CDRNet: Cascaded dense residual network for grayscale and pseudocolor medical image fusion. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 234:107506. [PMID: 37003041 DOI: 10.1016/j.cmpb.2023.107506] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/04/2022] [Revised: 03/18/2023] [Accepted: 03/22/2023] [Indexed: 06/19/2023]
Abstract
OBJECTIVE Multimodal medical fusion images have been widely used in clinical medicine, computer-aided diagnosis and other fields. However, the existing multimodal medical image fusion algorithms generally have shortcomings such as complex calculations, blurred details and poor adaptability. To solve this problem, we propose a cascaded dense residual network and use it for grayscale and pseudocolor medical image fusion. METHODS The cascaded dense residual network uses a multiscale dense network and a residual network as the basic network architecture, and a multilevel converged network is obtained through cascade. The cascaded dense residual network contains 3 networks, the first-level network inputs two images with different modalities to obtain a fused Image 1, the second-level network uses fused Image 1 as the input image to obtain fused Image 2 and the third-level network uses fused Image 2 as the input image to obtain fused Image 3. The multimodal medical image is trained through each level of the network, and the output fusion image is enhanced step-by-step. RESULTS As the number of networks increases, the fusion image becomes increasingly clearer. Through numerous fusion experiments, the fused images of the proposed algorithm have higher edge strength, richer details, and better performance in the objective indicators than the reference algorithms. CONCLUSION Compared with the reference algorithms, the proposed algorithm has better original information, higher edge strength, richer details and an improvement of the four objective SF, AG, MZ and EN indicator metrics.
Collapse
Affiliation(s)
- Jun Fu
- School of Information Engineering, Zunyi Normal University, Zunyi, Guizhou, 563006, China.
| | - Baiqing He
- Nanchang Institute of Technology, Nanchang, Jiangxi, 330044, China
| | - Jie Yang
- School of Information Engineering, Zunyi Normal University, Zunyi, Guizhou, 563006, China
| | - Jianpeng Liu
- School of Science, East China Jiaotong University, Nanchang, Jiangxi, 330013, China
| | - Aijia Ouyang
- School of Information Engineering, Zunyi Normal University, Zunyi, Guizhou, 563006, China
| | - Ya Wang
- School of Information Engineering, Zunyi Normal University, Zunyi, Guizhou, 563006, China
| |
Collapse
|
8
|
Li J, Han D, Wang X, Yi P, Yan L, Li X. Multi-Sensor Medical-Image Fusion Technique Based on Embedding Bilateral Filter in Least Squares and Salient Detection. SENSORS (BASEL, SWITZERLAND) 2023; 23:3490. [PMID: 37050552 PMCID: PMC10098979 DOI: 10.3390/s23073490] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Revised: 03/21/2023] [Accepted: 03/23/2023] [Indexed: 06/19/2023]
Abstract
A multi-sensor medical-image fusion technique, which integrates useful information from different single-modal images of the same tissue and provides a fused image that is more comprehensive and objective than a single-source image, is becoming an increasingly important technique in clinical diagnosis and treatment planning. The salient information in medical images often visually describes the tissue. To effectively embed salient information in the fused image, a multi-sensor medical image fusion method is proposed based on an embedding bilateral filter in least squares and salient detection via a deformed smoothness constraint. First, source images are decomposed into base and detail layers using a bilateral filter in least squares. Then, the detail layers are treated as superpositions of salient regions and background information; a fusion rule for this layer based on the deformed smoothness constraint and guided filtering was designed to successfully conserve the salient structure and detail information of the source images. A base-layer fusion rule based on modified Laplace energy and local energy is proposed to preserve the energy information of these source images. The experimental results demonstrate that the proposed method outperformed nine state-of-the-art methods in both subjective and objective quality assessments on the Harvard Medical School dataset.
Collapse
Affiliation(s)
- Jiangwei Li
- Guangdong-Hong Kong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528225, China
| | - Dingan Han
- Guangdong-Hong Kong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528225, China
| | - Xiaopan Wang
- Guangdong Province Graduate Joint Training Base (Foshan), Foshan University, Foshan 528225, China
| | - Peng Yi
- Jiangsu Shuguang Photoelectric Co., Ltd., Yangzhou 225009, China
| | - Liang Yan
- Jiangsu Shuguang Photoelectric Co., Ltd., Yangzhou 225009, China
| | - Xiaosong Li
- Guangdong-Hong Kong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528225, China
| |
Collapse
|
9
|
Zhang Y, Nie R, Cao J, Ma C. Self-Supervised Fusion for Multi-Modal Medical Images via Contrastive Auto-Encoding and Convolutional Information Exchange. IEEE COMPUT INTELL M 2023. [DOI: 10.1109/mci.2022.3223487] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
|
10
|
Zhang G, Nie X, Liu B, Yuan H, Li J, Sun W, Huang S. A multimodal fusion method for Alzheimer's disease based on DCT convolutional sparse representation. Front Neurosci 2023; 16:1100812. [PMID: 36685238 PMCID: PMC9853298 DOI: 10.3389/fnins.2022.1100812] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2022] [Accepted: 12/07/2022] [Indexed: 01/07/2023] Open
Abstract
Introduction The medical information contained in magnetic resonance imaging (MRI) and positron emission tomography (PET) has driven the development of intelligent diagnosis of Alzheimer's disease (AD) and multimodal medical imaging. To solve the problems of severe energy loss, low contrast of fused images and spatial inconsistency in the traditional multimodal medical image fusion methods based on sparse representation. A multimodal fusion algorithm for Alzheimer' s disease based on the discrete cosine transform (DCT) convolutional sparse representation is proposed. Methods The algorithm first performs a multi-scale DCT decomposition of the source medical images and uses the sub-images of different scales as training images, respectively. Different sparse coefficients are obtained by optimally solving the sub-dictionaries at different scales using alternating directional multiplication method (ADMM). Secondly, the coefficients of high-frequency and low-frequency subimages are inverse DCTed using an improved L1 parametric rule combined with improved spatial frequency novel sum-modified SF (NMSF) to obtain the final fused images. Results and discussion Through extensive experimental results, we show that our proposed method has good performance in contrast enhancement, texture and contour information retention.
Collapse
Affiliation(s)
- Guo Zhang
- School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing, China,School of Medical Information and Engineering, Southwest Medical University, Luzhou, China
| | - Xixi Nie
- Chongqing Key Laboratory of Image Cognition, College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Bangtao Liu
- School of Medical Information and Engineering, Southwest Medical University, Luzhou, China
| | - Hong Yuan
- School of Medical Information and Engineering, Southwest Medical University, Luzhou, China
| | - Jin Li
- School of Medical Information and Engineering, Southwest Medical University, Luzhou, China
| | - Weiwei Sun
- School of Optoelectronic Engineering, Chongqing University of Posts and Telecommunications, Chongqing, China,*Correspondence: Weiwei Sun,
| | - Shixin Huang
- School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing, China,Department of Scientific Research, The People’s Hospital of Yubei District of Chongqing City, Yubei, China,Shixin Huang,
| |
Collapse
|
11
|
Chen J, Ding J, Yu Y, Gong W. THFuse: An Infrared and Visible Image Fusion Network using Transformer and Hybrid Feature Extractor. Neurocomputing 2023. [DOI: 10.1016/j.neucom.2023.01.033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
|
12
|
Zhu P, Ouyang W, Guo Y, Zhou X. A Two-To-One Deep Learning General Framework for Image Fusion. Front Bioeng Biotechnol 2022; 10:923364. [PMID: 35979172 PMCID: PMC9376963 DOI: 10.3389/fbioe.2022.923364] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2022] [Accepted: 06/09/2022] [Indexed: 11/28/2022] Open
Abstract
The image fusion algorithm has great application value in the domain of computer vision, which makes the fused image have a more comprehensive and clearer description of the scene, and is beneficial to human eye recognition and automatic mechanical detection. In recent years, image fusion algorithms have achieved great success in different domains. However, it still has huge challenges in terms of the generalization of multi-modal image fusion. In reaction to this problem, this paper proposes a general image fusion framework based on an improved convolutional neural network. Firstly, the feature information of the input image is captured by the multiple feature extraction layers, and then multiple feature maps are stacked along the number of channels to acquire the feature fusion map. Finally, feature maps, which are derived from multiple feature extraction layers, are stacked in high dimensions by skip connection and convolution filtering for reconstruction to produce the final result. In this paper, multi-modal images are gained from multiple datasets to produce a large sample space to adequately train the network. Compared with the existing convolutional neural networks and traditional fusion algorithms, the proposed model not only has generality and stability but also has some strengths in subjective visualization and objective evaluation, while the average running time is at least 94% faster than the reference algorithm based on neural network.
Collapse
Affiliation(s)
- Pan Zhu
- Key Laboratory of Metallurgical Equipment and Control Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan, China
- Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, China
- Precision Manufacturing Institute, Wuhan University of Science and Technology, Wuhan, China
| | - Wanqi Ouyang
- Key Laboratory of Metallurgical Equipment and Control Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan, China
- Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, China
- Precision Manufacturing Institute, Wuhan University of Science and Technology, Wuhan, China
- *Correspondence: Wanqi Ouyang,
| | - Yongxing Guo
- Key Laboratory of Metallurgical Equipment and Control Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan, China
- Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, China
- Precision Manufacturing Institute, Wuhan University of Science and Technology, Wuhan, China
| | - Xinglin Zhou
- Key Laboratory of Metallurgical Equipment and Control Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan, China
- Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, China
- Precision Manufacturing Institute, Wuhan University of Science and Technology, Wuhan, China
| |
Collapse
|
13
|
Wang C, Wu J, Zhu Z, Chen H. MSFNet: MultiStage Fusion Network for infrared and visible image fusion. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.07.048] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
|
14
|
Kong W, Miao Q, Lei Y, Ren C. Guided filter random walk and improved spiking cortical model based image fusion method in NSST domain. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2021.11.060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
15
|
An object-based sparse representation model for spatiotemporal image fusion. Sci Rep 2022; 12:5021. [PMID: 35322054 PMCID: PMC8943014 DOI: 10.1038/s41598-022-08728-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2021] [Accepted: 03/10/2022] [Indexed: 11/12/2022] Open
Abstract
Many algorithms have been proposed for spatiotemporal image fusion on simulated data, yet only a few deal with spectral changes in real satellite images. An innovative spatiotemporal sparse representation (STSR) image fusion approach is introduced in this study to generate global dense high spatial and temporal resolution images from real satellite images. It aimed to minimize the data gap, especially when fine spatial resolution images are unavailable for a specific period. The proposed approach uses a set of real coarse- and fine-spatial resolution satellite images acquired simultaneously and another coarse image acquired at a different time to predict the corresponding unknown fine image. During the fusion process, pixels located between object classes with different spectral responses are more vulnerable to spectral distortion. Therefore, firstly, a rule-based fuzzy classification algorithm is used in STSR to classify input data and extract accurate edge candidates. Then, an object-based estimation of physical constraints and brightness shift between input data is utilized to construct the proposed sparse representation (SR) model that can deal with real input satellite images. Initial rules to adjust spatial covariance and equalize spectral response of object classes between input images are introduced as prior information to the model, followed by an optimization step to improve the STSR approach. The proposed method is applied to real fine Sentinel-2 and coarse Landsat-8 satellite data. The results showed that introducing objects in the fusion process improved spatial detail, especially over the edge candidates, and eliminated spectral distortion by preserving the spectral continuity of extracted objects. Experiments revealed the promising performance of the proposed object-based STSR image fusion approach based on its quantitative results, where it preserved almost 96.9% and 93.8% of the spectral detail over the smooth and urban areas, respectively.
Collapse
|
16
|
Image fusion algorithm based on unsupervised deep learning-optimized sparse representation. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103140] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
17
|
Hybrid Sharpening Transformation Approach for Multifocus Image Fusion Using Medical and Nonmedical Images. JOURNAL OF HEALTHCARE ENGINEERING 2021; 2021:7000991. [PMID: 34931139 PMCID: PMC8684527 DOI: 10.1155/2021/7000991] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/02/2021] [Accepted: 10/18/2021] [Indexed: 12/01/2022]
Abstract
In this study, we introduced a preprocessing novel transformation approach for multifocus image fusion. In the multifocus image, fusion has generated a high informative image by merging two source images with different areas or objects in focus. Acutely the preprocessing means sharpening performed on the images before applying fusion techniques. In this paper, along with the novel concept, a new sharpening technique, Laplacian filter + discrete Fourier transform (LF + DFT), is also proposed. The LF is used to recognize the meaningful discontinuities in an image. DFT recognizes that the rapid change in the image is like sudden changes in the frequencies, low-frequency to high-frequency in the images. The aim of image sharpening is to highlight the key features, identifying the minor details, and sharpen the edges while the previous methods are not so effective. To validate the effectiveness the proposed method, the fusion is performed by a couple of advanced techniques such as stationary wavelet transform (SWT) and discrete wavelet transform (DWT) with both types of images like grayscale and color image. The experiments are performed on nonmedical and medical (breast medical CT and MRI images) datasets. The experimental results demonstrate that the proposed method outperforms all evaluated qualitative and quantitative metrics. Quantitative assessment is performed by eight well-known metrics, and every metric described its own feature by which it is easily assumed that the proposed method is superior. The experimental results of the proposed technique SWT (LF + DFT) are summarized for evaluation matrices such as RMSE (5.6761), PFE (3.4378), MAE (0.4010), entropy (9.0121), SNR (26.8609), PSNR (40.1349), CC (0.9978), and ERGAS (2.2589) using clock dataset.
Collapse
|
18
|
Lu Y, Wang R, Gao Q, Sun D, Zhu D. Multi-Modal Image Fusion Based on Matrix Product State of Tensor. Front Neurorobot 2021; 15:762252. [PMID: 34867257 PMCID: PMC8634473 DOI: 10.3389/fnbot.2021.762252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2021] [Accepted: 10/06/2021] [Indexed: 11/16/2022] Open
Abstract
Multi-modal image fusion integrates different images of the same scene collected by different sensors into one image, making the fused image recognizable by the computer and perceived by human vision easily. The traditional tensor decomposition is an approximate decomposition method and has been applied to image fusion. In this way, the image details may be lost in the process of fusion image reconstruction. To preserve the fine information of the images, an image fusion method based on tensor matrix product decomposition is proposed to fuse multi-modal images in this article. First, each source image is initialized into a separate third-order tensor. Then, the tensor is decomposed into a matrix product form by using singular value decomposition (SVD), and the Sigmoid function is used to fuse the features extracted in the decomposition process. Finally, the fused image is reconstructed by multiplying all the fused tensor components. Since the algorithm is based on a series of singular value decomposition, a stable closed solution can be obtained and the calculation is also simple. The experimental results show that the fusion image quality obtained by this algorithm is superior to other algorithms in both objective evaluation metrics and subjective evaluation.
Collapse
Affiliation(s)
| | | | | | | | - De Zhu
- Anhui University, Hefei, China
| |
Collapse
|
19
|
Fu J, Li W, Du J, Xu L. DSAGAN: A generative adversarial network based on dual-stream attention mechanism for anatomical and functional image fusion. Inf Sci (N Y) 2021. [DOI: 10.1016/j.ins.2021.06.083] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
20
|
Muzammil SR, Maqsood S, Haider S, Damaševičius R. CSID: A Novel Multimodal Image Fusion Algorithm for Enhanced Clinical Diagnosis. Diagnostics (Basel) 2020; 10:E904. [PMID: 33167376 PMCID: PMC7694345 DOI: 10.3390/diagnostics10110904] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2020] [Revised: 10/28/2020] [Accepted: 10/29/2020] [Indexed: 12/19/2022] Open
Abstract
Technology-assisted clinical diagnosis has gained tremendous importance in modern day healthcare systems. To this end, multimodal medical image fusion has gained great attention from the research community. There are several fusion algorithms that merge Computed Tomography (CT) and Magnetic Resonance Images (MRI) to extract detailed information, which is used to enhance clinical diagnosis. However, these algorithms exhibit several limitations, such as blurred edges during decomposition, excessive information loss that gives rise to false structural artifacts, and high spatial distortion due to inadequate contrast. To resolve these issues, this paper proposes a novel algorithm, namely Convolutional Sparse Image Decomposition (CSID), that fuses CT and MR images. CSID uses contrast stretching and the spatial gradient method to identify edges in source images and employs cartoon-texture decomposition, which creates an overcomplete dictionary. Moreover, this work proposes a modified convolutional sparse coding method and employs improved decision maps and the fusion rule to obtain the final fused image. Simulation results using six datasets of multimodal images demonstrate that CSID achieves superior performance, in terms of visual quality and enriched information extraction, in comparison with eminent image fusion algorithms.
Collapse
Affiliation(s)
- Shah Rukh Muzammil
- Department of Computer Science, City University of Science and Information Technology, Peshawar 25000, Pakistan; (S.R.M.); (S.H.)
| | - Sarmad Maqsood
- Department of Software Engineering, Kaunas University of Technology, Kaunas 51368, Lithuania;
| | - Shahab Haider
- Department of Computer Science, City University of Science and Information Technology, Peshawar 25000, Pakistan; (S.R.M.); (S.H.)
| | - Robertas Damaševičius
- Department of Software Engineering, Kaunas University of Technology, Kaunas 51368, Lithuania;
| |
Collapse
|
21
|
Gao C, Liu F, Yan H. Infrared and visible image fusion using dual-tree complex wavelet transform and convolutional sparse representation. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2020. [DOI: 10.3233/jifs-200554] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Infrared and visible image fusion refers to the technology that merges the visual details of visible images and thermal feature information of infrared images; it has been extensively adopted in numerous image processing fields. In this study, a dual-tree complex wavelet transform (DTCWT) and convolutional sparse representation (CSR)-based image fusion method was proposed. In the proposed method, the infrared images and visible images were first decomposed by dual-tree complex wavelet transform to characterize their high-frequency bands and low-frequency band. Subsequently, the high-frequency bands were enhanced by guided filtering (GF), while the low-frequency band was merged through convolutional sparse representation and choose-max strategy. Lastly, the fused images were reconstructed by inverse DTCWT. In the experiment, the objective and subjective comparisons with other typical methods proved the advantage of the proposed method. To be specific, the results achieved using the proposed method were more consistent with the human vision system and contained more texture detail information.
Collapse
Affiliation(s)
- Chengrui Gao
- Sichuan University, The College of Electronics Information and Engineering, Chengdu, China
| | - Feiqiang Liu
- Sichuan University, The College of Electronics Information and Engineering, Chengdu, China
| | - Hua Yan
- Sichuan University, The College of Electronics Information and Engineering, Chengdu, China
| |
Collapse
|
22
|
Ma H, Liao Q, Zhang J, Liu S, Xue JH. An α-Matte Boundary Defocus Model-Based Cascaded Network for Multi-focus Image Fusion. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2020; PP:8668-8679. [PMID: 32845840 DOI: 10.1109/tip.2020.3018261] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Capturing an all-in-focus image with a single camera is difficult since the depth of field of the camera is usually limited. An alternative method to obtain the all-in-focus image is to fuse several images that are focused at different depths. However, existing multi-focus image fusion methods cannot obtain clear results for areas near the focused/defocused boundary (FDB). In this paper, a novel α-matte boundary defocus model is proposed to generate realistic training data with the defocus spread effect precisely modeled, especially for areas near the FDB. Based on this α-matte defocus model and the generated data, a cascaded boundary-aware convolutional network termed MMF-Net is proposed and trained, aiming to achieve clearer fusion results around the FDB. Specifically, the MMF-Net consists of two cascaded subnets for initial fusion and boundary fusion. These two subnets are designed to first obtain a guidance map of FDB and then refine the fusion near the FDB. Experiments demonstrate that with the help of the new α-matte boundary defocus model, the proposed MMF-Net outperforms the state-of-the-art methods both qualitatively and quantitatively.
Collapse
|
23
|
|
24
|
Li H, He X, Yu Z, Luo J. Noise-robust image fusion with low-rank sparse decomposition guided by external patch prior. Inf Sci (N Y) 2020. [DOI: 10.1016/j.ins.2020.03.009] [Citation(s) in RCA: 32] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
25
|
Yang M, Qu Q, Shen Y, Lei K, Zhu J. Cross-domain aspect/sentiment-aware abstractive review summarization by combining topic modeling and deep reinforcement learning. Neural Comput Appl 2020. [DOI: 10.1007/s00521-018-3825-2] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
26
|
Li J, Guo X, Lu G, Zhang B, Xu Y, Wu F, Zhang D. DRPL: Deep Regression Pair Learning For Multi-Focus Image Fusion. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2020; 29:4816-4831. [PMID: 32142440 DOI: 10.1109/tip.2020.2976190] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
In this paper, a novel deep network is proposed for multi-focus image fusion, named Deep Regression Pair Learning (DRPL). In contrast to existing deep fusion methods which divide the input image into small patches and apply a classifier to judge whether the patch is in focus or not, DRPL directly converts the whole image into a binary mask without any patch operation, subsequently tackling the difficulty of the blur level estimation around the focused/defocused boundary. Simultaneously, a pair learning strategy, which takes a pair of complementary source images as inputs and generates two corresponding binary masks, is introduced into the model, greatly imposing the complementary constraint on each pair and making a large contribution to the performance improvement. Furthermore, as the edge or gradient does exist in the focus part while there is no similar property for the defocus part, we also embed a gradient loss to ensure the generated image to be all-in-focus. Then the structural similarity index (SSIM) is utilized to make a trade-off between the reference and fused images. Experimental results conducted on the synthetic and real-world datasets substantiate the effectiveness and superiority of DRPL compared with other state-of-the-art approaches. The testing code can be found in https://github.com/sasky1/DPRL.
Collapse
|
27
|
Bouzos O, Andreadis I, Mitianoudis N. Conditional Random Field Model for Robust Multi-Focus Image Fusion. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2019; 28:5636-5648. [PMID: 31217116 DOI: 10.1109/tip.2019.2922097] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
In this paper, a novel multi-focus image fusion algorithm based on conditional random field optimization (mf-CRF) is proposed. It is based on an unary term that includes the combined activity estimation of both high and low frequencies of the input images, while a spatially varying smoothness term is introduced, in order to align the graph-cut solution with boundaries of focused and defocused pixels. The proposed model retains the advantages of both spatial-domain methods and multi-spectral methods and by solving an energy minimization problem and finds an optimal solution for the multi-focus image fusion problem. Experimental results demonstrate the effectiveness of the proposed method that outperforms current state-of-the-art multi-focus image fusion algorithms in both qualitative and quantitative comparisons. In this paper, the successful application of the mf-CRF model in multi-modal image fusion (visible-infrared and medical) is also presented.
Collapse
|
28
|
Chen WB, Hu M, Zhou L, Gu H, Zhang X. Fusion Algorithm of Multi-focus Images with Weighted Ratios and Weighted Gradient Based on Wavelet Transform. JOURNAL OF INTELLIGENT SYSTEMS 2019. [DOI: 10.1515/jisys-2017-0078] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
Abstract
Abstract
Multi-focus image fusion means fusing a completely clear image with a set of images of the same scene and under the same imaging conditions with different focus points. In order to get a clear image that contains all relevant objects in an area, the multi-focus image fusion algorithm is proposed based on wavelet transform. Firstly, the multi-focus images were decomposed by wavelet transform. Secondly, the wavelet coefficients of the approximant and detail sub-images are fused respectively based on the fusion rule. Finally, the fused image was obtained by using the inverse wavelet transform. Among them, for the low-frequency and high-frequency coefficients, we present a fusion rule based on the weighted ratios and the weighted gradient with the improved edge detection operator. The experimental results illustrate that the proposed algorithm is effective for retaining the detailed images.
Collapse
|
29
|
Ma J, Zhou Z, Wang B, Miao L, Zong H. Multi-focus image fusion using boosted random walks-based algorithm with two-scale focus maps. Neurocomputing 2019. [DOI: 10.1016/j.neucom.2019.01.048] [Citation(s) in RCA: 46] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
30
|
Li J, Yuan G, Fan H. Multifocus Image Fusion Using Wavelet-Domain-Based Deep CNN. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2019; 2019:4179397. [PMID: 30915109 PMCID: PMC6402241 DOI: 10.1155/2019/4179397] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/11/2018] [Revised: 01/05/2019] [Accepted: 01/20/2019] [Indexed: 11/17/2022]
Abstract
Multifocus image fusion is the merging of images of the same scene and having multiple different foci into one all-focus image. Most existing fusion algorithms extract high-frequency information by designing local filters and then adopt different fusion rules to obtain the fused images. In this paper, a wavelet is used for multiscale decomposition of the source and fusion images to obtain high-frequency and low-frequency images. To obtain clearer and complete fusion images, this paper uses a deep convolutional neural network to learn the direct mapping between the high-frequency and low-frequency images of the source and fusion images. In this paper, high-frequency and low-frequency images are used to train two convolutional networks to encode the high-frequency and low-frequency images of the source and fusion images. The experimental results show that the method proposed in this paper can obtain a satisfactory fusion image, which is superior to that obtained by some advanced image fusion algorithms in terms of both visual and objective evaluations.
Collapse
Affiliation(s)
- Jinjiang Li
- School of Computer Science and Technology, Shandong Technology and Business University, Yantai 264005, China
- Co-innovation Center of Shandong Colleges and Universities: Future Intelligent Computing, Yantai 264005, China
| | - Genji Yuan
- School of Computer Science and Technology, Shandong Technology and Business University, Yantai 264005, China
- Co-innovation Center of Shandong Colleges and Universities: Future Intelligent Computing, Yantai 264005, China
| | - Hui Fan
- School of Computer Science and Technology, Shandong Technology and Business University, Yantai 264005, China
- Co-innovation Center of Shandong Colleges and Universities: Future Intelligent Computing, Yantai 264005, China
| |
Collapse
|
31
|
Long M, Li Z, Xie X, Li G, Wang Z. Adaptive Image Enhancement Based on Guide Image and Fraction-Power Transformation for Wireless Capsule Endoscopy. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2018; 12:993-1003. [PMID: 30346276 DOI: 10.1109/tbcas.2018.2869530] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Good image quality of the wireless capsule endoscopy (WCE) is the key for doctors to diagnose gastrointestinal (GI) tract diseases. However, the poor illumination, limited performance of the camera in WCE, and complex environment in the GI tract usually result in low-quality endoscopic images. Existing image enhancement methods only use the information of the image itself or multiple images of the same scene to accomplish the enhancement. In this paper, we propose an adaptive image enhancement method based on guide image and fraction-power transformation. First, intensities of endoscopic images are analyzed to assess the illumination conditions. Second, images captured under poor illumination conditions are enhanced by a brand-new image enhancement method called adaptive guide image based enhancement (AGIE). AGIE enhances low-quality images by using the information of a good quality image of the similar scene. Otherwise, images are enhanced by the proposed adaptive fraction-power transformation. Experimental results show that the proposed method improves the average intensity of endoscopic images by 64.20% and the average local entropy by 31.25%, which outperforms the state-of-art methods.
Collapse
|
32
|
Du B, Wang S, Xu C, Wang N, Zhang L, Tao D. Multi-Task Learning for Blind Source Separation. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2018; 27:4219-4231. [PMID: 29870343 DOI: 10.1109/tip.2018.2836324] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Blind source separation (BSS) aims to discover the underlying source signals from a set of linear mixture signals without any prior information of the mixing system, which is a fundamental problem in signal and image processing field. Most of the state-of-the-art algorithms have independently handled the decompositions of mixture signals. In this paper, we propose a new algorithm named multi-task sparse model to solve the BSS problem. Source signals are characterized via sparse techniques. Meanwhile, we regard the decomposition of each mixture signal as a task and employ the idea of multi-task learning to discover connections between tasks for the accuracy improvement of the source signal separation. Theoretical analyses on the optimization convergence and sample complexity of the proposed algorithm are provided. Experimental results based on extensive synthetic and real-world data demonstrate the necessity of exploiting connections between mixture signals and the effectiveness of the proposed algorithm.
Collapse
|
33
|
Qi G, Zhang Q, Zeng F, Wang J, Zhu Z. Multi‐focus image fusion via morphological similarity‐based dictionary construction and sparse representation. CAAI TRANSACTIONS ON INTELLIGENCE TECHNOLOGY 2018. [DOI: 10.1049/trit.2018.0011] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
Affiliation(s)
- Guanqiu Qi
- College of Automation, Chongqing University of Posts and TelecommunicationsChongqing400065People's Republic of China
- School of Computing, Informatics, and Decision Systems EngineeringArizona State UniversityTempeAZ85287USA
| | - Qiong Zhang
- College of Automation, Chongqing University of Posts and TelecommunicationsChongqing400065People's Republic of China
| | - Fancheng Zeng
- College of Automation, Chongqing University of Posts and TelecommunicationsChongqing400065People's Republic of China
| | - Jinchuan Wang
- College of Automation, Chongqing University of Posts and TelecommunicationsChongqing400065People's Republic of China
| | - Zhiqin Zhu
- College of Automation, Chongqing University of Posts and TelecommunicationsChongqing400065People's Republic of China
| |
Collapse
|
34
|
Li C, Zhang X, Wu H. Multifocus image fusion method for image acquisition of 3D objects. APPLIED OPTICS 2018; 57:4514-4523. [PMID: 29877399 DOI: 10.1364/ao.57.004514] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/21/2017] [Accepted: 04/25/2018] [Indexed: 06/08/2023]
Abstract
We propose a multifocus image fusion method for achieving all-in-focus images of three-dimensional objects based on the combination of transform domain and spatial domain techniques. First, the source images are decomposed into low-frequency and high-frequency components by the discrete wavelet transform technique. Next, a correlation coefficient is employed to obtain the maximum similarity among low-frequency components. Then, in order not to interrupt the correlations among decomposition layers, the comparison among high-frequency components is executed by transforming them to spatial domain. In addition, a sliding window is used to evaluate the local saliency of the pixels more accurately. Finally, the fused image is synthesized from source images and the saliency map. The variance, entropy, spatial frequency, mutual information, edge intensity, and similarity measure (QAB/F) are used as metrics to evaluate the sharpness of the fused image. Experimental results demonstrate that the fusion performance of the proposed method is enhanced compared with that of the other widely used techniques. In the application of three-dimensional surface optical detection, the proposed method is suitable for obtaining the complete image at varying distances in the same scene, so as to prepare for subsequent defect identification.
Collapse
|
35
|
Chen Y, Guan J, Cham WK. Robust Multi-Focus Image Fusion Using Edge Model and Multi-Matting. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2018; 27:1526-1541. [PMID: 29990190 DOI: 10.1109/tip.2017.2779274] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
An effective multi-focus image fusion method is proposed to generate an all-in-focus image with all objects in focus by merging multiple images. The proposed method first estimates focus maps using a novel combination of edge model and a traditional block-based focus measure. Then, a propagation process is conducted to obtain accurate weight maps based on a novel multi-matting model that makes full use of the spatial information. The fused all-in-focus image is finally generated based on a weighted-sum strategy. Experimental results demonstrate that the proposed method has state-of-the-art performance for multi-focus image fusion under various situations encountered in practice, even in cases with obvious misregistration.
Collapse
|
36
|
Yan X, Qin H, Li J. Multi-focus image fusion based on dictionary learning with rolling guidance filter. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2017; 34:432-440. [PMID: 28248370 DOI: 10.1364/josaa.34.000432] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
We present a new multi-focus image fusion method based on dictionary learning with a rolling guidance filter to fusion of multi-focus images with registration and mis-registration. First, we learn a dictionary via several classical multi-focus images blurred by a rolling guidance filter. Subsequently, we present a new model for focus regions identification via applying the learned dictionary to input images to obtain the corresponding focus feature maps. Then, we determine the initial decision map via comparing the difference of the focus feature maps. The latter is to optimize the initial decision map and perform it on input images to obtain fused images. Experimental results demonstrate that the suggested algorithm is competitive with the current state of the art and superior to some representative methods when input images are well registered and mis-registered.
Collapse
|
37
|
|