Deep Graph-Convolutional Image Denoising

Overview of Research on Digital Image Denoising Methods

During image collection, images are often polluted by noise because of imaging conditions and equipment limitations. Images are also disturbed by external noise during compression and transmission, which adversely affects consequent processing, like image segmentation, target recognition, and text detection. A two-dimensional amplitude image is one of the most common image categories, which is widely used in people's daily life and work. Research on this kind of image-denoising algorithm is a hotspot in the field of image denoising. Conventional denoising methods mainly use the nonlocal self-similarity of images and sparser representatives in the converted domain for image denoising. In particular, the three-dimensional block matching filtering (BM3D) algorithm not only effectively removes the image noise but also better retains the detailed information in the image. As artificial intelligence develops, the deep learning-based image-denoising method has become an important research direction. This review provides a general overview and comparison of traditional image-denoising methods and deep neural network-based image-denoising methods. First, the essential framework of classic traditional denoising and deep neural network denoising approaches is presented, and the denoising approaches are classified and summarized. Then, existing denoising methods are compared with quantitative and qualitative analyses on a public denoising dataset. Finally, we point out some potential challenges and directions for future research in the field of image denoising. This review can help researchers clearly understand the differences between various image-denoising algorithms, which not only helps them to choose suitable algorithms or improve and innovate on this basis but also provides research ideas and directions for subsequent research in this field.

GroupCDL: Interpretable Denoising and Compressed Sensing MRI via Learned Group-Sparsity and Circulant Attention

Nonlocal self-similarity within images has become an increasingly popular prior in deep-learning models. Despite their successful image restoration performance, such models remain largely uninterpretable due to their black-box construction. Our previous studies have shown that interpretable construction of a fully convolutional denoiser (CDLNet), with performance on par with state-of-the-art black-box counterparts, is achievable by unrolling a convolutional dictionary learning algorithm. In this manuscript, we seek an interpretable construction of a convolutional network with a nonlocal self-similarity prior that performs on par with black-box nonlocal models. We show that such an architecture can be effectively achieved by up-grading theℓ 1 sparsity prior (soft-thresholding) of CDLNet to an image-adaptive group-sparsity prior (group-thresholding). The proposed learned group-thresholding makes use of nonlocal attention to perform spatially varying soft-thresholding on the latent representation. To enable effective training and inference on large images with global artifacts, we propose a novel circulant-sparse attention. We achieve competitive natural-image denoising performance compared to black-box nonlocal DNNs and transformers. The interpretable construction of our network allows for a straightforward extension to Compressed Sensing MRI (CS-MRI), yielding state-of-the-art performance. Lastly, we show robustness to noise-level mismatches between training and inference for denoising and CS-MRI reconstruction.

An efficient lightweight network for image denoising using progressive residual and convolutional attention feature fusion

While deep learning has become the go-to method for image denoising due to its impressive noise removal capabilities, excessive network depth often plagues existing approaches, leading to significant computational burdens. To address this critical bottleneck, we propose a novel lightweight progressive residual and attention mechanism fusion network that effectively alleviates these limitations. This architecture tackles both Gaussian and real-world image noise with exceptional efficacy. Initiated through dense blocks (DB) tasked with discerning the noise distribution, this approach substantially reduces network parameters while comprehensively extracting local image features. The network then adopts a progressive strategy, whereby shallow convolutional features are incrementally integrated with deeper features, establishing a residual fusion framework adept at extracting encompassing global features relevant to noise characteristics. The process concludes by integrating the output feature maps from each DB and the robust edge features from the convolutional attention feature fusion module (CAFFM). These combined elements are then directed to the reconstruction layer, ultimately producing the final denoised image. Empirical analyses conducted in environments characterized by Gaussian white noise and natural noise, spanning noise levels 15-50, indicate a marked enhancement in performance. This assertion is quantitatively corroborated by increased average values in metrics such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Feature Similarity Index for Color images (FSIMc), outperforming the outcomes of more than 20 existing methods across six varied datasets. Collectively, the network delineated in this research exhibits exceptional adeptness in image denoising. Simultaneously, it adeptly preserves essential image features such as edges and textures, thereby signifying a notable progression in the domain of image processing. The proposed model finds applicability in a range of image-centric domains, encompassing image processing, computer vision, video analysis, and pattern recognition.

Multi-energy CT material decomposition using graph model improved CNN

In spectral CT imaging, the coefficient image of the basis material obtained by the material decomposition technique can estimate the tissue composition, and its accuracy directly affects the disease diagnosis. Although the precision of material decomposition is increased by employing convolutional neural networks (CNN), extracting the non-local features from the CT image is restricted using the traditional CNN convolution operator. A graph model built by multi-scale non-local self-similar patterns is introduced into multi-material decomposition (MMD). We proposed a novel MMD method based on graph edge-conditioned convolution U-net (GECCU-net) to enhance material image quality. The GECCU-net focuses on developing a multi-scale encoder. At the network coding stage, three paths are applied to capture comprehensive image features. The local and non-local feature aggregation (LNFA) blocks are designed to integrate the local and non-local features from different paths. The graph edge-conditioned convolution based on non-Euclidean space excavates the non-local features. A hybrid loss function is defined to accommodate multi-scale input images and avoid over-smoothing of results. The proposed network is compared quantitatively with base CNN models on the simulated and real datasets. The material images generated by GECCU-net have less noise and artifacts while retaining more information on tissue. The Structural SIMilarity (SSIM) of the obtained abdomen and chest water maps reaches 0.9976 and 0.9990, respectively, and the RMSE reduces to 0.1218 and 0.4903 g/cm3. The proposed method can improve MMD performance and has potential applications.

All-optical image denoising using a diffractive visual processor

Image denoising, one of the essential inverse problems, targets to remove noise/artifacts from input images. In general, digital image denoising algorithms, executed on computers, present latency due to several iterations implemented in, e.g., graphics processing units (GPUs). While deep learning-enabled methods can operate non-iteratively, they also introduce latency and impose a significant computational burden, leading to increased power consumption. Here, we introduce an analog diffractive image denoiser to all-optically and non-iteratively clean various forms of noise and artifacts from input images - implemented at the speed of light propagation within a thin diffractive visual processor that axially spans <250 × λ, where λ is the wavelength of light. This all-optical image denoiser comprises passive transmissive layers optimized using deep learning to physically scatter the optical modes that represent various noise features, causing them to miss the output image Field-of-View (FoV) while retaining the object features of interest. Our results show that these diffractive denoisers can efficiently remove salt and pepper noise and image rendering-related spatial artifacts from input phase or intensity images while achieving an output power efficiency of ~30-40%. We experimentally demonstrated the effectiveness of this analog denoiser architecture using a 3D-printed diffractive visual processor operating at the terahertz spectrum. Owing to their speed, power-efficiency, and minimal computational overhead, all-optical diffractive denoisers can be transformative for various image display and projection systems, including, e.g., holographic displays.

A dual nonsubsampled contourlet network for synthesis images and infrared thermal images denoising

The most direct way to find the electrical switchgear fault is to use infrared thermal imaging technology for temperature measurement. However, infrared thermal imaging images are usually polluted by noise, and there are problems such as low contrast and blurred edges. To solve these problems, this article proposes a dual convolutional neural network model based on nonsubsampled contourlet transform (NSCT). First, the overall structure of the model is made wider by combining the two networks. Compared with the deeper convolutional neural network, the dual convolutional neural network (CNN) improves the denoising performance without increasing the computational cost too much. Secondly, the model uses NSCT and inverse NSCT to obtain more texture information and avoid the gridding effect. It achieves a good balance between noise reduction performance and detail retention. A large number of simulation experiments show that the model has the ability to deal with synthetic noise and real noise, which has high practical value.

Adaptive orthogonal directional total variation with kernel regression for CT image denoising

BACKGROUND

Low-dose computed tomography (CT) has been successful in reducing radiation exposure for patients. However, the use of reconstructions from sparse angle sampling in low-dose CT often leads to severe streak artifacts in the reconstructed images.

OBJECTIVE

In order to address this issue and preserve image edge details, this study proposes an adaptive orthogonal directional total variation method with kernel regression.

METHODS

The CT reconstructed images are initially processed through kernel regression to obtain the N-term Taylor series, which serves as a local representation of the regression function. By expanding the series to the second order, we obtain the desired estimate of the regression function and localized information on the first and second derivatives. To mitigate the noise impact on these derivatives, kernel regression is performed again to update the first and second derivatives. Subsequently, the original reconstructed image, its local approximation, and the updated derivatives are summed using a weighting scheme to derive the image used for calculating orientation information. For further removal of stripe artifacts, the study introduces the adaptive orthogonal directional total variation (AODTV) method, which denoises along both the edge direction and the normal direction, guided by the previously obtained orientation.

RESULTS

Both simulation and real experiments have obtained good results. The results of two real experiments show that the proposed method has obtained PSNR values of 34.5408 dB and 29.4634 dB, which are 1.2392-5.9333 dB and 2.828-6.7995 dB higher than the contrast denoising algorithm, respectively, indicating that the proposed method has good denoising performance.

CONCLUSIONS

The study demonstrates the effectiveness of the method in eliminating strip artifacts and preserving the fine details of the images.

Adaptive filter method in Bendlet domain for biological slice images

The biological cross-sectional images majorly consist of closed-loop structures, which are suitable to be represented by the second-order shearlet system with curvature (Bendlet). In this study, an adaptive filter method for preserving textures in the bendlet domain is proposed. The Bendlet system represents the original image as an image feature database based on image size and Bendlet parameters. This database can be divided into image high-frequency and low-frequency sub-bands separately. The low-frequency sub-bands adequately represent the closed-loop structure of the cross-sectional images and the high-frequency sub-bands accurately represent the detailed textural features of the images, which reflect the characteristics of Bendlet and can be effectively distinguished from the Shearlet system. The proposed method takes full advantage of this feature, then selects the appropriate thresholds based on the images' texture distribution characteristics in the database to eliminate noise. The locust slice images are taken as an example to test the proposed method. The experimental results show that the proposed method can significantly eliminate the low-level Gaussian noise and protect the image information compared with other popular denoising algorithms. The PSNR and SSIM obtained are better than other methods. The proposed algorithm can be effectively applied to other biological cross-sectional images.

Multimodal learning with graphs

Artificial intelligence for graphs has achieved remarkable success in modeling complex systems, ranging from dynamic networks in biology to interacting particle systems in physics. However, the increasingly heterogeneous graph datasets call for multimodal methods that can combine different inductive biases-the set of assumptions that algorithms use to make predictions for inputs they have not encountered during training. Learning on multimodal datasets presents fundamental challenges because the inductive biases can vary by data modality and graphs might not be explicitly given in the input. To address these challenges, multimodal graph AI methods combine different modalities while leveraging cross-modal dependencies using graphs. Diverse datasets are combined using graphs and fed into sophisticated multimodal architectures, specified as image-intensive, knowledge-grounded and language-intensive models. Using this categorization, we introduce a blueprint for multimodal graph learning, use it to study existing methods and provide guidelines to design new models.

MGL: Mutual Graph Learning for Camouflaged Object Detection

Camouflaged object detection, which aims to detect/segment the object(s) that blend in with their surrounding, remains challenging for deep models due to the intrinsic similarities between foreground objects and background surroundings. Ideally, an effective model should be capable of finding valuable clues from the given scene and integrating them into a joint learning framework to co-enhance the representation. Inspired by this observation, we propose a novel Mutual Graph Learning (MGL) model by shifting the conventional perspective of mutual learning from regular grids to graph domain. Specifically, an image is decoupled by MGL into two task-specific feature maps - one for finding the rough location of the target and the other for capturing its accurate boundary details. Then, the mutual benefits can be fully exploited by reasoning their high-order relations through graphs recurrently. It should be noted that our method is different from most mutual learning models that model all between-task interactions with the use of a shared function. To increase information interactions, MGL is built with typed functions for dealing with different complementary relations. To overcome the accuracy loss caused by interpolation to higher resolution and the computational redundancy resulting from recurrent learning, the S-MGL is equipped with a multi-source attention contextual recovery module, called R-MGL_v2, which uses the pixel feature information iteratively. Experiments on challenging datasets, including CHAMELEON, CAMO, COD10K, and NC4K demonstrate the effectiveness of our MGL with superior performance to existing state-of-the-art methods. The code can be found at https://github.com/fanyang587/MGL.

Equilibrium Image Denoising With Implicit Differentiation

Recent efforts on learning-based image denoising approaches use unrolled architectures with a fixed number of repeatedly stacked blocks. However, due to difficulties in training networks corresponding to deeper layers, simply stacking blocks may cause performance degradation, and the number of unrolled blocks needs to be manually tuned to find an appropriate value. To circumvent these problems, this paper describes an alternative approach with implicit models. To our best knowledge, our approach is the first attempt to model iterative image denoising through an implicit scheme. The model employs implicit differentiation to calculate gradients in the backward pass, thus avoiding the training difficulties of explicit models and elaborate selection of the iteration number. Our model is parameter-efficient and has only one implicit layer, which is a fixed-point equation that casts the desired noise feature as its solution. By simulating infinite iterations of the model, the final denoising result is given by the equilibrium that is achieved through accelerated black-box solvers. The implicit layer not only captures the non-local self-similarity prior for image denoising, but also facilitates training stability and thereby boosts the denoising performance. Extensive experiments show that our model leads to better performances than state-of-the-art explicit denoisers with enhanced qualitative and quantitative results.

ERA-WGAT: Edge-enhanced residual autoencoder with a window-based graph attention convolutional network for low-dose CT denoising

Computed tomography (CT) has become a powerful tool for medical diagnosis. However, minimizing X-ray radiation risk for the patient poses significant challenges to obtain suitable low dose CT images. Although various low-dose CT methods using deep learning techniques have produced impressive results, convolutional neural network based methods focus more on local information and hence are very limited for non-local information extraction. This paper proposes ERA-WGAT, a residual autoencoder incorporating an edge enhancement module that performs convolution with eight types of learnable operators providing rich edge information and a window-based graph attention convolutional network that combines static and dynamic attention modules to explore non-local self-similarity. We use the compound loss function that combines MSE loss and multi-scale perceptual loss to mitigate the over-smoothing problem. Compared with current low-dose CT denoising methods, ERA-WGAT confirmed superior noise suppression and perceived image quality.

Dual Encoder-Based Dynamic-Channel Graph Convolutional Network With Edge Enhancement for Retinal Vessel Segmentation

Retinal vessel segmentation with deep learning technology is a crucial auxiliary method for clinicians to diagnose fundus diseases. However, the deep learning approaches inevitably lose the edge information, which contains spatial features of vessels while performing down-sampling, leading to the limited segmentation performance of fine blood vessels. Furthermore, the existing methods ignore the dynamic topological correlations among feature maps in the deep learning framework, resulting in the inefficient capture of the channel characterization. To address these limitations, we propose a novel dual encoder-based dynamic-channel graph convolutional network with edge enhancement (DE-DCGCN-EE) for retinal vessel segmentation. Specifically, we first design an edge detection-based dual encoder to preserve the edge of vessels in down-sampling. Secondly, we investigate a dynamic-channel graph convolutional network to map the image channels to the topological space and synthesize the features of each channel on the topological map, which solves the limitation of insufficient channel information utilization. Finally, we study an edge enhancement block, aiming to fuse the edge and spatial features in the dual encoder, which is beneficial to improve the accuracy of fine blood vessel segmentation. Competitive experimental results on five retinal image datasets validate the efficacy of the proposed DE-DCGCN-EE, which achieves more remarkable segmentation results against the other state-of-the-art methods, indicating its potential clinical application.

Cloud Removal with SAR-Optical Data Fusion and Graph-Based Feature Aggregation Network

In observations of Earth, the existence of clouds affects the quality and usability of optical remote sensing images in practical applications. Many cloud removal methods have been proposed to solve this issue. Among these methods, synthetic aperture radar (SAR)-based methods have more potential than others because SAR imaging is hardly affected by clouds, and can reflect ground information differences and changes. While SAR images used as auxiliary information for cloud removal may be blurred and noisy, the similar non-local information of spectral and electromagnetic features cannot be effectively utilized by traditional cloud removal methods. To overcome these weaknesses, we propose a novel cloud removal method using SAR-optical data fusion and a graph-based feature aggregation network (G-FAN). First, cloudy optical images and contemporary SAR images are concatenated and transformed into hyper-feature maps by pre-convolution. Second, the hyper-feature maps are inputted into the G-FAN to reconstruct the missing data of the cloud-covered area by aggregating the electromagnetic backscattering information of the SAR image, and the spectral information of neighborhood and non-neighborhood pixels in the optical image. Finally, post-convolution and a long skip connection are adopted to reconstruct the final predicted cloud-free images. Both the qualitative and quantitative experimental results from the simulated data and real data experiments show that our proposed method outperforms traditional deep learning methods for cloud removal.

RatUNet: residual U-Net based on attention mechanism for image denoising

Deep convolutional neural networks (CNNs) have been very successful in image denoising. However, with the growth of the depth of plain networks, CNNs may result in performance degradation. The lack of network depth leads to the limited ability of the network to extract image features and difficults to fuse the shallow image features into the deep image information. In this work, we propose an improved deep convolutional U-Net framework (RatUNet) for image denoising. RatUNet improves Unet as follows: (1) RatUNet uses the residual blocks of ResNet to deepen the network depth, so as to avoid the network performance saturation. (2) RatUNet improves the down-sampling method, which is conducive to extracting image features. (3) RatUNet improves the up-sampling method, which is used to restore image details. (4) RatUNet improves the skip-connection method of the U-Net network, which is used to fuse the shallow feature information into the deep image details, and it is more conducive to restore the clean image. (5) In order to better process the edge information of the image, RatUNet uses depthwise and polarized self-attention mechanism to guide a CNN for image denoising. Extensive experiments show that our RatUNet is more efficient and has better performance than existing state-of-the-art denoising methods, especially in SSIM metrics, the denoising effect of the RatUNet achieves very high performance. Visualization results show that the denoised image by RatUNet is smoother and sharper than other methods.

Image denoising algorithm of social network based on multifeature fusion

A social network image denoising algorithm based on multifeature fusion is proposed. Based on the multifeature fusion theory, the process of social network image denoising is regarded as the fitting process of neural network, and a simple and efficient convolution neural structure of multifeature fusion is constructed for image denoising. The gray features of social network image are collected, and the gray values are denoising and cleaning. Based on the image features, multiple denoising is carried out to ensure the accuracy of social network image denoising algorithm and improve the accuracy of image processing. Experiments show that the average noise of the image processed by the algorithm designed in this study is reduced by 8.6905 dB, which is much larger than that of other methods, and the signal-to-noise ratio of the output image is high, which is maintained at about 30 dB, which has a high effect in the process of practical application.

Meta PID Attention Network for Flexible and Efficient Real-World Noisy Image Denoising

Recent deep convolutional neural networks for real-world noisy image denoising have shown a huge boost in performance by training a well-engineered network over external image pairs. However, most of these methods are generally trained with supervision. Once the testing data is no longer compatible with the training conditions, they can exhibit poor generalization and easily result in severe overfitting or degrading performances. To tackle this barrier, we propose a novel denoising algorithm, dubbed as Meta PID Attention Network (MPA-Net). Our MPA-Net is built based upon stacking Meta PID Attention Modules (MPAMs). In each MPAM, we utilize a second-order attention module (SAM) to exploit the channel-wise feature correlations with second-order statistics, which are then adaptively updated via a proportional-integral-derivative (PID) guided meta-learning framework. This learning framework exerts the unique property of the PID controller and meta-learning scheme to dynamically generate filter weights for beneficial update of the extracted features within a feedback control system. Moreover, the dynamic nature of the framework enables the generated weights to be flexibly tweaked according to the input at test time. Thus, MPAM not only achieves discriminative feature learning, but also facilitates a robust generalization ability on distinct noises for real images. Extensive experiments on ten datasets are conducted to inspect the effectiveness of the proposed MPA-Net quantitatively and qualitatively, which demonstrates both its superior denoising performance and promising generalization ability that goes beyond those of the state-of-the-art denoising methods.

Learning Dynamic Relationships for Facial Expression Recognition Based on Graph Convolutional Network

Facial action units (AUs) analysis plays an important role in facial expression recognition (FER). Existing deep spectral convolutional networks (DSCNs) have made encouraging performance for FER based on a set of facial local regions and a predefined graph structure. However, these regions do not have close relationships to AUs, and DSCNs cannot model the dynamic spatial dependencies of these regions for estimating different facial expressions. To tackle these issues, we propose a novel double dynamic relationships graph convolutional network (DDRGCN) to learn the strength of the edges in the facial graph by a trainable weighted adjacency matrix. We construct facial graph data by 20 regions of interest (ROIs) guided by different facial AUs. Furthermore, we devise an efficient graph convolutional network in which the inherent dependencies of vertices in the facial graph can be learned automatically during network training. Notably, the proposed model only has 110K parameters and 0.48MB model size, which is significantly less than most existing methods. Experiments on four widely-used FER datasets demonstrate that the proposed dynamic relationships graph network achieves superior results compared to existing light-weight networks, not just in terms of accuracy but also model size and speed.

Reasoning Graph Networks for Kinship Verification: From Star-Shaped to Hierarchical

In this paper, we investigate the problem of facial kinship verification by learning hierarchical reasoning graph networks. Conventional methods usually focus on learning discriminative features for each facial image of a paired sample and neglect how to fuse the obtained two facial image features and reason about the relations between them. To address this, we propose a Star-shaped Reasoning Graph Network (S-RGN). Our S-RGN first constructs a star-shaped graph where each surrounding node encodes the information of comparisons in a feature dimension and the central node is employed as the bridge for the interaction of surrounding nodes. Then we perform relational reasoning on this star graph with iterative message passing. The proposed S-RGN uses only one central node to analyze and process information from all surrounding nodes, which limits its reasoning capacity. We further develop a Hierarchical Reasoning Graph Network (H-RGN) to exploit more powerful and flexible capacity. More specifically, our H-RGN introduces a set of latent reasoning nodes and constructs a hierarchical graph with them. Then bottom-up comparative information abstraction and top-down comprehensive signal propagation are iteratively performed on the hierarchical graph to update the node features. Extensive experimental results on four widely used kinship databases show that the proposed methods achieve very competitive results.

Spatio-temporal graph convolutional network for diagnosis and treatment response prediction of major depressive disorder from functional connectivity

The pathophysiology of major depressive disorder (MDD) has been explored to be highly associated with the dysfunctional integration of brain networks. It is therefore imperative to explore neuroimaging biomarkers to aid diagnosis and treatment. In this study, we developed a spatiotemporal graph convolutional network (STGCN) framework to learn discriminative features from functional connectivity for automatic diagnosis and treatment response prediction of MDD. Briefly, dynamic functional networks were first obtained from the resting-state fMRI with the sliding temporal window method. Secondly, a novel STGCN approach was proposed by introducing the modules of spatial graph attention convolution (SGAC) and temporal fusion. A novel SGAC was proposed to improve the feature learning ability and special anatomy prior guided pooling was developed to enable the feature dimension reduction. A temporal fusion module was proposed to capture the dynamic features of functional connectivity between adjacent sliding windows. Finally, the STGCN proposed approach was utilized to the tasks of diagnosis and antidepressant treatment response prediction for MDD. Performances of the framework were comprehensively examined with large cohorts of clinical data, which demonstrated its effectiveness in classifying MDD patients and predicting the treatment response. The sound performance suggests the potential of the STGCN for the clinical use in diagnosis and treatment prediction.

Deep Concatenated Residual Networks for Improving Quality of Halftoning-Based BTC Decoded Image

This paper presents a simple technique for improving the quality of the halftoning-based block truncation coding (H-BTC) decoded image. The H-BTC is an image compression technique inspired from typical block truncation coding (BTC). The H-BTC yields a better decoded image compared to that of the classical BTC scheme under human visual observation. However, the impulsive noise commonly appears on the H-BTC decoded image. It induces an unpleasant feeling while one observes this decoded image. Thus, the proposed method presented in this paper aims to suppress the occurring impulsive noise by exploiting a deep learning approach. This process can be regarded as an ill-posed inverse imaging problem, in which the solution candidates of a given problem can be extremely huge and undetermined. The proposed method utilizes the convolutional neural networks (CNN) and residual learning frameworks to solve the aforementioned problem. These frameworks effectively reduce the impulsive noise occurrence, and at the same time, it improves the quality of H-BTC decoded images. The experimental results show the effectiveness of the proposed method in terms of subjective and objective measurements.