Conditional Random Field Model for Robust Multi-Focus Image Fusion

Multi-focus image fusion with parameter adaptive dual channel dynamic threshold neural P systems

Multi-focus image fusion (MFIF) is an important technique that aims to combine the focused regions of multiple source images into a fully clear image. Decision-map methods are widely used in MFIF to maximize the preservation of information from the source images. While many decision-map methods have been proposed, they often struggle with difficulties in determining focus and non-focus boundaries, further affecting the quality of the fused images. Dynamic threshold neural P (DTNP) systems are computational models inspired by biological spiking neurons, featuring dynamic threshold and spiking mechanisms to better distinguish focused and unfocused regions for decision map generation. However, original DTNP systems require manual parameter configuration and have only one stimulus. Therefore, they are not suitable to be used directly for generating high-precision decision maps. To overcome these limitations, we propose a variant called parameter adaptive dual channel DTNP (PADCDTNP) systems. Inspired by the spiking mechanisms of PADCDTNP systems, we further develop a new MFIF method. As a new neural model, PADCDTNP systems adaptively estimate parameters according to multiple external inputs to produce decision maps with robust boundaries, resulting in high-quality fusion results. Comprehensive experiments on the Lytro/MFFW/MFI-WHU dataset show that our method achieves advanced performance and yields comparable results to the fourteen representative MFIF methods. In addition, compared to the standard DTNP systems, PADCDTNP systems improve the fusion performance and fusion efficiency on the three datasets by 5.69% and 86.03%, respectively. The codes for both the proposed method and the comparison methods are released at https://github.com/MorvanLi/MFIF-PADCDTNP.

Conditional Random Field-Guided Multi-Focus Image Fusion

Multi-Focus image fusion is of great importance in order to cope with the limited Depth-of-Field of optical lenses. Since input images contain noise, multi-focus image fusion methods that support denoising are important. Transform-domain methods have been applied to image fusion, however, they are likely to produce artifacts. In order to cope with these issues, we introduce the Conditional Random Field (CRF) CRF-Guided fusion method. A novel Edge Aware Centering method is proposed and employed to extract the low and high frequencies of the input images. The Independent Component Analysis—ICA transform is applied to high-frequency components and a Conditional Random Field (CRF) model is created from the low frequency and the transform coefficients. The CRF model is solved efficiently with the α-expansion method. The estimated labels are used to guide the fusion of the low-frequency components and the transform coefficients. Inverse ICA is then applied to the fused transform coefficients. Finally, the fused image is the addition of the fused low frequency and the fused high frequency. CRF-Guided fusion does not introduce artifacts during fusion and supports image denoising during fusion by applying transform domain coefficient shrinkage. Quantitative and qualitative evaluation demonstrate the superior performance of CRF-Guided fusion compared to state-of-the-art multi-focus image fusion methods.

Deep Learning-Based Multi-Focus Image Fusion: A Survey and a Comparative Study

Multi-focus image fusion (MFIF) is an important area in image processing. Since 2017, deep learning has been introduced to the field of MFIF and various methods have been proposed. However, there is a lack of survey papers that discuss deep learning-based MFIF methods in detail. In this study, we fill this gap by giving a detailed survey on deep learning-based MFIF algorithms, including methods, datasets and evaluation metrics. To the best of our knowledge, this is the first survey paper that focuses on deep learning-based approaches in the field of MFIF. Besides, extensive experiments have been conducted to compare the performance of deep learning-based MFIF algorithms with conventional MFIF approaches. By analyzing qualitative and quantitative results, we give some observations on the current status of MFIF and discuss some future prospects of this field.

A Self-Supervised Residual Feature Learning Model for Multifocus Image Fusion

Multi-focus image fusion (MFIF) attempts to achieve an "all-focused" image from multiple source images with the same scene but different focused objects. Given the lack of multi-focus image sets for network training, we propose a self-supervised residual feature learning model in this paper. The model consists of a feature extraction network and a fusion module. We select image super-resolution as a pretext task in the MFIF field, which is supported by a new residual gradient prior discovered by our theoretical study for low- and high-resolution (LR-HR) image pairs, as well as for multi-focus images. In the pretext task, our network's training set is LR-HR image pairs generated from natural images, and HR images can be regarded as pseudo-labels of LR images. In the fusion task, the trained network extracts residual features of multi-focus images firstly. Secondly, the fusion module, consisting of an activity level measurement and a new boundary refinement method, is leveraged for the features to generated decision maps. Experimental results, both subjective evaluations and objective evaluations, demonstrate that our approach outperforms other state-of-the-art fusion algorithms.

Exploiting Superpixels for Multi-Focus Image Fusion

Multi-focus image fusion is the process of combining focused regions of two or more images to obtain a single all-in-focus image. It is an important research area because a fused image is of high quality and contains more details than the source images. This makes it useful for numerous applications in image enhancement, remote sensing, object recognition, medical imaging, etc. This paper presents a novel multi-focus image fusion algorithm that proposes to group the local connected pixels with similar colors and patterns, usually referred to as superpixels, and use them to separate the focused and de-focused regions of an image. We note that these superpixels are more expressive than individual pixels, and they carry more distinctive statistical properties when compared with other superpixels. The statistical properties of superpixels are analyzed to categorize the pixels as focused or de-focused and to estimate a focus map. A spatial consistency constraint is ensured on the initial focus map to obtain a refined map, which is used in the fusion rule to obtain a single all-in-focus image. Qualitative and quantitative evaluations are performed to assess the performance of the proposed method on a benchmark multi-focus image fusion dataset. The results show that our method produces better quality fused images than existing image fusion techniques.

An α-Matte Boundary Defocus Model-Based Cascaded Network for Multi-focus Image Fusion

Capturing an all-in-focus image with a single camera is difficult since the depth of field of the camera is usually limited. An alternative method to obtain the all-in-focus image is to fuse several images that are focused at different depths. However, existing multi-focus image fusion methods cannot obtain clear results for areas near the focused/defocused boundary (FDB). In this paper, a novel α-matte boundary defocus model is proposed to generate realistic training data with the defocus spread effect precisely modeled, especially for areas near the FDB. Based on this α-matte defocus model and the generated data, a cascaded boundary-aware convolutional network termed MMF-Net is proposed and trained, aiming to achieve clearer fusion results around the FDB. Specifically, the MMF-Net consists of two cascaded subnets for initial fusion and boundary fusion. These two subnets are designed to first obtain a guidance map of FDB and then refine the fusion near the FDB. Experiments demonstrate that with the help of the new α-matte boundary defocus model, the proposed MMF-Net outperforms the state-of-the-art methods both qualitatively and quantitatively.