Selective Wavelet Attention Learning for Single Image Deraining

FDuDoCLNet: Fully dual-domain contrastive learning network for parallel MRI reconstruction

Magnetic resonance imaging (MRI) is a non-invasive medical imaging technique that is widely used for high-resolution imaging of soft tissues and organs. However, the slow speed of MRI imaging, especially in high-resolution or dynamic scans, makes MRI reconstruction an important research topic. Currently, MRI reconstruction methods based on deep learning (DL) have garnered significant attention, and they improve the reconstruction quality by learning complex image features. However, DL-based MR image reconstruction methods exhibit certain limitations. First, the existing reconstruction networks seldom account for the diverse frequency features in the wavelet domain. Second, existing dual-domain reconstruction methods may pay too much attention to the features of a single domain (such as the global information in the image domain or the local details in the wavelet domain), resulting in the loss of either critical global structures or fine details in certain regions of the reconstructed image. In this work, inspired by the lifting scheme in wavelet theory, we propose a novel Fully Dual-Domain Contrastive Learning Network (FDuDoCLNet) based on variational networks (VarNet) for accelerating PI in both the image and wavelet domains. It is composed of several cascaded dual-domain regularization units and data consistency (DC) layers, in which a novel dual-domain contrastive loss is introduced to optimize the reconstruction performance effectively. The proposed FDuDoCLNet was evaluated on the publicly available fastMRI multi-coil knee dataset under a 6× acceleration factor, achieving a PSNR of 34.439 dB and a SSIM of 0.895.

BiFFN: Bi-Frequency Guided Feature Fusion Network for Visible-Infrared Person Re-Identification

Visible-infrared person re-identification (VI-ReID) aims to minimize the modality gaps of pedestrian images across different modalities. Existing methods primarily focus on extracting cross-modality features from the spatial domain, which often limits the comprehensive extraction of useful information. Compared with conventional approaches that either focus on single-frequency components or employ simple multi-branch fusion strategies, our method fundamentally addresses the modality discrepancy through systematic frequency-space co-learning. To address this limitation, we propose a novel bi-frequency feature fusion network (BiFFN) that effectively extracts and fuses features from both high- and low-frequency domains and spatial domain features to reduce modality gaps. The network introduces a frequency-spatial enhancement (FSE) module to enhance feature representation across both domains. Additionally, the deep frequency mining (DFM) module optimizes cross-modality information utilization by leveraging distinct features of high- and low-frequency features. The cross-frequency fusion (CFF) module further aligns low-frequency features and fuses them with high-frequency features to generate middle features that incorporate critical information from each modality. To refine the distribution of identity features in the common space, we develop a unified modality center (UMC) loss, which promotes a more balanced inter-modality distribution while preserving discriminative identity information. Extensive experiments demonstrate that the proposed BiFFN achieves state-of-the-art performance in VI-ReID. Specifically, our method achieved a Rank-1 accuracy of 77.5% and an mAP of 75.9% on the SYSU-MM01 dataset under the all-search mode. Additionally, it achieved a Rank-1 accuracy of 58.5% and an mAP of 63.7% on the LLCM dataset under the IR-VIS mode. These improvements verify that our model, with the integration of feature fusion and the incorporation of frequency domains, significantly reduces modality gaps and outperforms previous methods.

Wavelet Approximation-Aware Residual Network for Single Image Deraining

It has been made great progress on single image deraining based on deep convolutional neural networks (CNNs). In most existing deep deraining methods, CNNs aim to learn a direct mapping from rainy images to clean rain-less images, and their architectures are becoming more and more complex. However, due to the limitation of mixing rain with object edges and background, it is difficult to separate rain and object/background, and the edge details of the image cannot be effectively recovered in the reconstruction process. To address this problem, we propose a novel wavelet approximation-aware residual network (WAAR), wherein rain is effectively removed from both low-frequency structures and high-frequency details at each level separately, especially in low-frequency sub-images at each level. After wavelet transform, we propose novel approximation aware (AAM) and approximation level blending (ALB) mechanisms to further aid the low-frequency networks at each level recover the structure and texture of low-frequency sub-images recursively, while the high frequency network can effectively eliminate rain streaks through block connection and achieve different degrees of edge detail enhancement by adjusting hyperparameters. In addition, we also introduce block connection to enrich the high-frequency details in the high-frequency network, which is favorable for obtaining potential interdependencies between high- and low-frequency features. Experimental results indicate that the proposed WAAR exhibits strong performance in reconstructing clean and rain-free images, recovering real and undistorted texture structures, and enhancing image edges in comparison with the state-of-the-art approaches on synthetic and real image datasets. It shows the effectiveness of our method, especially on image edges and texture details.

Memory Uncertainty Learning for Real-World Single Image Deraining

Single image deraining has witnessed dramatic improvements by training deep neural networks on large-scale synthetic data. However, due to the discrepancy between authentic and synthetic rain images, it is challenging to directly extend existing methods to real-world scenes. To address this issue, we propose a memory-uncertainty guided semi-supervised method to learn rain properties simultaneously from synthetic and real data. The key aspect is developing a stochastic memory network that is equipped with memory modules to record prototypical rain patterns. The memory modules are updated in a self-supervised way, allowing the network to comprehensively capture rainy styles without the need for clean labels. The memory items are read stochastically according to their similarities with rain representations, leading to diverse predictions and efficient uncertainty estimation. Furthermore, we present an uncertainty-aware self-training mechanism to transfer knowledge from supervised deraining to unsupervised cases. An additional target network is adopted to produce pseudo-labels for unlabeled data, of which the incorrect ones are rectified by uncertainty estimates. Finally, we construct a new large-scale image deraining dataset of 10.2 k real rain images, significantly improving the diversity of real rain scenes. Experiments show that our method achieves more appealing results for real-world rain removal than recent state-of-the-art methods.

Rain Removal From Light Field Images With 4D Convolution and Multi-Scale Gaussian Process

Existing deraining methods focus mainly on a single input image. However, with just a single input image, it is extremely difficult to accurately detect and remove rain streaks, in order to restore a rain-free image. In contrast, a light field image (LFI) embeds abundant 3D structure and texture information of the target scene by recording the direction and position of each incident ray via a plenoptic camera. LFIs are becoming popular in the computer vision and graphics communities. However, making full use of the abundant information available from LFIs, such as 2D array of sub-views and the disparity map of each sub-view, for effective rain removal is still a challenging problem. In this paper, we propose a novel method, 4D-MGP-SRRNet, for rain streak removal from LFIs. Our method takes as input all sub-views of a rainy LFI. To make full use of the LFI, it adopts 4D convolutional layers to simultaneously process all sub-views of the LFI. In the pipeline, the rain detection network, MGPDNet, with a novel Multi-scale Self-guided Gaussian Process (MSGP) module is proposed to detect high-resolution rain streaks from all sub-views of the input LFI at multi-scales. Semi-supervised learning is introduced for MSGP to accurately detect rain streaks by training on both virtual-world rainy LFIs and real-world rainy LFIs at multi-scales via computing pseudo ground truths for real-world rain streaks. We then feed all sub-views subtracting the predicted rain streaks into a 4D convolution-based Depth Estimation Residual Network (DERNet) to estimate the depth maps, which are later converted into fog maps. Finally, all sub-views concatenated with the corresponding rain streaks and fog maps are fed into a powerful rainy LFI restoring model based on the adversarial recurrent neural network to progressively eliminate rain streaks and recover the rain-free LFI. Extensive quantitative and qualitative evaluations conducted on both synthetic LFIs and real-world LFIs demonstrate the effectiveness of our proposed method.

Rain-Free and Residue Hand-in-Hand: A Progressive Coupled Network for Real-Time Image Deraining

Rainy weather is a challenge for many vision-oriented tasks (e.g., object detection and segmentation), which causes performance degradation. Image deraining is an effective solution to avoid performance drop of downstream vision tasks. However, most existing deraining methods either fail to produce satisfactory restoration results or cost too much computation. In this work, considering both effectiveness and efficiency of image deraining, we propose a progressive coupled network (PCNet) to well separate rain streaks while preserving rain-free details. To this end, we investigate the blending correlations between them and particularly devise a novel coupled representation module (CRM) to learn the joint features and the blending correlations. By cascading multiple CRMs, PCNet extracts the hierarchical features of multi-scale rain streaks, and separates the rain-free content and rain streaks progressively. To promote computation efficiency, we employ depth-wise separable convolutions and a U-shaped structure, and construct CRM in an asymmetric architecture to reduce model parameters and memory footprint. Extensive experiments are conducted to evaluate the efficacy of the proposed PCNet in two aspects: (1) image deraining on several synthetic and real-world rain datasets and (2) joint image deraining and downstream vision tasks (e.g., object detection and segmentation). Furthermore, we show that the proposed CRM can be easily adopted to similar image restoration tasks including image dehazing and low-light enhancement with competitive performance. The source code is available at https://github.com/kuijiang0802/PCNet.