Hyperspectral Images Denoising via Nonconvex Regularized Low-Rank and Sparse Matrix Decomposition

High-resolution hyperspectral video imaging using a hexagonal camera array

Retrieving the reflectance spectrum from objects is an essential task for many classification and detection problems, since many materials and processes have a unique spectral behavior. In many cases, it is highly desirable to capture hyperspectral images due to the high spectral flexibility. Often, it is even necessary to capture hyperspectral videos or at least to be able to record a hyperspectral image at once, also called snapshot hyperspectral imaging, to avoid spectral smearing. For this task, a high-resolution snapshot hyperspectral camera array using a hexagonal shape is introduced. The hexagonal array for hyperspectral imaging uses off-the-shelf hardware, which enables high flexibility regarding employed cameras, lenses, and filters. Hence, the spectral range can be easily varied by mounting a different set of filters. Moreover, the concept of using off-the-shelf hardware enables low prices in comparison to other approaches with highly specialized hardware. Since classical industrial cameras are used in this hyperspectral camera array, the spatial and temporal resolution is very high, while recording 37 hyperspectral channels in the range from 400 to 760 nm in 10 nm steps. As the cameras are at different spatial positions, a registration process is required for near-field imaging, which maps the peripheral camera views to the center view. It is shown that this combination using a hyperspectral camera array and the corresponding image registration pipeline is superior in comparison to other popular snapshot approaches. For this evaluation, a synthetic hyperspectral database is rendered. On the synthetic data, the novel approach, to our knowledge, outperforms its best competitor by more than 3 dB in reconstruction quality. This synthetic data is also used to show the superiority of the hexagonal shape in comparison to an orthogonal-spaced one. Moreover, a real-world high-resolution hyperspectral video database with 10 scenes is provided for further research in other applications.

Laplacian Gradient Consistency Prior for Flash Guided Non-Flash Image Denoising

For flash guided non-flash image denoising, the main challenge is to explore the consistency prior between the two modalities. Most existing methods attempt to model the flash/non-flash consistency in pixel level, which may easily lead to blurred edges. Different from these methods, we have an important finding in this paper, which reveals that the modality gap between flash and non-flash images conforms to the Laplacian distribution in gradient domain. Based on this finding, we establish a Laplacian gradient consistency (LGC) model for flash guided non-flash image denoising. This model is demonstrated to have faster convergence speed and denoising accuracy than the traditional pixel consistency model. Through solving the LGC model, we further design a deep network namely LGCNet. Different from existing image denoising networks, each component of the LGCNet strictly matches the solution of LGC model, giving the network good interpretability. The performance of the proposed LGCNet is evaluated on three different flash/non-flash image datasets, which demonstrates its superior denoising performance over many state-of-the-art methods both quantitatively and qualitatively. The intermediate features are also visualized to verify the effectiveness of the Laplacian gradient consistency prior. The source codes are available at https://github.com/JingyiXu404/LGCNet.

Hyperspectral Image Denoising: From Model-Driven, Data-Driven, to Model-Data-Driven

Mixed noise pollution in HSI severely disturbs subsequent interpretations and applications. In this technical review, we first give the noise analysis in different noisy HSIs and conclude crucial points for programming HSI denoising algorithms. Then, a general HSI restoration model is formulated for optimization. Later, we comprehensively review existing HSI denoising methods, from model-driven strategy (nonlocal mean, total variation, sparse representation, low-rank matrix approximation, and low-rank tensor factorization), data-driven strategy [2-D convolutional neural network (CNN), 3-D CNN, hybrid, and unsupervised networks], to model-data-driven strategy. The advantages and disadvantages of each strategy for HSI denoising are summarized and contrasted. Behind this, we present an evaluation of the HSI denoising methods for various noisy HSIs in simulated and real experiments. The classification results of denoised HSIs and execution efficiency are depicted through these HSI denoising methods. Finally, prospects of future HSI denoising methods are listed in this technical review to guide the ongoing road for HSI denoising. The HSI denoising dataset could be found at https://qzhang95.github.io.

Plug-and-Play Priors for Multi-Shot Compressive Hyperspectral Imaging

Multi-shot coded aperture snapshot spectral imaging (CASSI) uses multiple measurement snapshots to encode the three-dimensional hyperspectral image (HSI). Increasing the number of snapshots will multiply the number of measurements, making CASSI system more appropriate for detailed spatial or spectrally rich scenes. However, the reconstruction algorithms still face the challenge of being ineffective or inflexible. In this paper, we propose a plug-and-play (PnP) method that uses denoiser as priors for multi-shot CASSI. Specifically, the proposed PnP method is based on the primal-dual algorithm with linesearch (PDAL), which makes it flexible and can be used for any multi-shot CASSI mechanisms. Furthermore, a new subspaced-based nonlocal reweighted low-rank (SNRL) denoiser is presented to utilize the global spectral correlation and nonlocal self-similarity priors of HSI. By integrating the SNRL denoiser into PnP-PDAL, we show the balloons ( 512×512×31 ) in CAVE dataset recovered from two snapshots compressive measurements with MPSNR above 50 dB. Experimental results demonstrate that our proposed method leads to significant improvements compared to the current state-of-the-art methods.

Enhancing Image Quality via Robust Noise Filtering Using Redescending M-Estimators

In the field of image processing, noise represents an unwanted component that can occur during signal acquisition, transmission, and storage. In this paper, we introduce an efficient method that incorporates redescending M-estimators within the framework of Wiener estimation. The proposed approach effectively suppresses impulsive, additive, and multiplicative noise across varied densities. Our proposed filter operates on both grayscale and color images; it uses local information obtained from the Wiener filter and robust outlier rejection based on Insha and Hampel's tripartite redescending influence functions. The effectiveness of the proposed method is verified through qualitative and quantitative results, using metrics such as PSNR, MAE, and SSIM.

Synthetic hyperspectral array video database with applications to cross-spectral reconstruction and hyperspectral video coding

In this paper, a synthetic hyperspectral video database is introduced. Since it is impossible to record ground-truth hyperspectral videos, this database offers the possibility to leverage the evaluation of algorithms in diverse applications. For all scenes, depth maps are provided as well to yield the position of a pixel in all spatial dimensions as well as the reflectance in spectral dimension. Two novel algorithms for two different applications are proposed to prove the diversity of applications that can be addressed by this novel database. First, a cross-spectral image reconstruction algorithm is extended to exploit the temporal correlation between two consecutive frames. The evaluation using this hyperspectral database shows an increase in peak signal-to-noise ratio (PSNR) of up to 5.6 dB dependent on the scene. Second, a hyperspectral video coder is introduced, which extends an existing hyperspectral image coder by exploiting temporal correlation. The evaluation shows rate savings of up to 10% depending on the scene.

Cooperated Spectral Low-Rankness Prior and Deep Spatial Prior for HSI Unsupervised Denoising

Model-driven methods and data-driven methods have been widely developed for hyperspectral image (HSI) denoising. However, there are pros and cons in both model-driven and data-driven methods. To address this issue, we develop a self-supervised HSI denoising method via integrating model-driven with data-driven strategy. The proposed framework simultaneously cooperates the spectral low-rankness prior and deep spatial prior (SLRP-DSP) for HSI self-supervised denoising. SLRP-DSP introduces the Tucker factorization via orthogonal basis and reduced factor, to capture the global spectral low-rankness prior in HSI. Besides, SLRP-DSP adopts a self-supervised way to learn the deep spatial prior. The proposed method doesn't need a large number of clean HSIs as the label samples. Through the self-supervised learning, SLRP-DSP can adaptively adjust the deep spatial prior from self-spatial information for reduced spatial factor denoising. An alternating iterative optimization framework is developed to exploit the internal low-rankness prior of third-order tensors and the spatial feature extraction capacity of convolutional neural network. Compared with both existing model-driven methods and data-driven methods, experimental results manifest that the proposed SLRP-DSP outperforms on mixed noise removal in different noisy HSIs.

Denoising Single Images by Feature Ensemble Revisited

Image denoising is still a challenging issue in many computer vision subdomains. Recent studies have shown that significant improvements are possible in a supervised setting. However, a few challenges, such as spatial fidelity and cartoon-like smoothing, remain unresolved or decisively overlooked. Our study proposes a simple yet efficient architecture for the denoising problem that addresses the aforementioned issues. The proposed architecture revisits the concept of modular concatenation instead of long and deeper cascaded connections, to recover a cleaner approximation of the given image. We find that different modules can capture versatile representations, and a concatenated representation creates a richer subspace for low-level image restoration. The proposed architecture's number of parameters remains smaller than in most of the previous networks and still achieves significant improvements over the current state-of-the-art networks.

Adaptive Rank and Structured Sparsity Corrections for Hyperspectral Image Restoration

Hyperspectral images (HSIs) are inevitably contaminated by the mixed noise (such as Gaussian noise, impulse noise, deadlines, and stripes), which could influence the subsequent processing accuracy. Generally, HSI restoration can be transformed into the low-rank matrix recovery (LRMR). In the LRMR, the nuclear norm is widely used to substitute the matrix rank, but its effectiveness is still worth improving. Besides, the l₀ -norm cannot capture the sparse noise's structured sparsity property. To handle these issues, the adaptive rank and structured sparsity corrections (ARSSC) are presented for HSI restoration. The ARSSC introduces two convex regularizers, that is: 1) the rank correction (RC) and 2) the structured sparsity correction (SSC), to, respectively, approximate the matrix rank and the l_2,0 -norm. The RC and the SSC can adaptively offset the penalization of large entries from the nuclear norm and the l_2,1 -norm, respectively, where the larger the entry, the greater its offset. Therefore, the proposed ARSSC achieves a tighter approximation of the noise-free HSI low-rank structure and promotes the structured sparsity of sparse noise. An efficient alternative direction method of multipliers (ADMM) algorithm is applied to solve the resulting convex optimization problem. The superiority of the ARSSC in terms of the mixed noise removal and spatial-spectral structure information preserving, is demonstrated by several experimental results both on simulated and real datasets, compared with other state-of-the-art HSI restoration approaches.

Adaptive denoising hyperspectral data for visualization enhancement of intraoperative tissue

The vast amount of reflectance information obtained from the hyperspectral imaging devices offers great opportunities for investigating the function and structure of human tissue. However, the captured hyperspectral data often contain various noises due to the intrinsic imperfection of associated electrical and optical imaging components. This work proposed an automatic total variation algorithm to suppress the noises while preserving the details of the spectral and spatial information. The variation of spectral images at neighboring bands was calculated for regulating the total variation of hyperspectral data so that the spectral-dependent noises can be treated differentially across all bands. Experimental results demonstrated that the proposed method could effectively remove the spectral noises, especially near the ends of those extreme bands. The noise suppressed hyperspectral data could then be used for the visualization enhancement on pathophysiological conditions of intraoperative observed anatomies such as the vessels of brain tissues.

A Low-Rank Tensor Decomposition Model With Factors Prior and Total Variation for Impulsive Noise Removal

Image restoration is a long-standing problem in signal processing and low-level computer vision. Previous studies have shown that imposing a low-rank Tucker decomposition (TKD) constraint could produce impressive performances. However, the TKD-based schemes may lead to the overfitting/underfitting problem because of incorrectly predefined ranks. To address this issue, we prove that the n -rank is upper bounded by the rank of each Tucker factor matrix. Using this relationship, we propose a formulation by imposing the nuclear norm regularization on the latent factors of TKD, which can avoid the burden of rank selection and reduce the computational cost when dealing with large-scale tensors. In this formulation, we adopt the Minimax Concave Penalty to remove the impulsive noise instead of the l₁ -norm which may deviate from both the data-acquisition model and the prior model. Moreover, we employ an anisotropic total variation regularization to explore the piecewise smooth structure in both spatial and spectral domains. To solve this problem, we design the symmetric Gauss-Seidel (sGS) based alternating direction method of multipliers (ADMM) algorithm. Compared to the directly extended ADMM, our algorithm can achieve higher accuracy since more structural information is utilized. Finally, we conduct experiments on the three kinds of datasets, numerical results demonstrate the superiority of the proposed method, especially, the average PSNR of the proposed method can improve about 1~5dB for each noise level of color images.

Multimodal Magnetic Resonance Imaging to Diagnose Knee Osteoarthritis under Artificial Intelligence

This work aimed to investigate the application value of the multimodal magnetic resonance imaging (MRI) algorithm based on the low-rank decomposition denoising (LRDD) in the diagnosis of knee osteoarthritis (KOA), so as to offer a better examination method in the clinic. Seventy-eight patients with KOA were selected as the research objects, and they all underwent T1-weighted imaging (T1WI), T2-weighted imaging (T2WI), fat suppression T2WI (SE-T2WI), and fat saturation T2WI (FS-T2WI). All obtained images were processed by using the I-LRDD algorithm. According to the degree of articular cartilage lesions under arthroscopy, the patients were divided into a group I, a group II, a group III, and a group IV. The sensitivity, specificity, accuracy, and consistency of KOA diagnosis of T1WI, T2WI, SE-T2WI, and FS-T2WI were analyzed by referring to the results of arthroscopy. The results showed that the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) of the I-LRDD algorithm used in this work were higher than those of image block priori denoising (IBPD) and LRDD, and the time consumption was lower than that of IBDP and LRDD (p < 0.05). The sensitivity, specificity, accuracy, and consistency (Kappa value) of multimodal MRI in the diagnosis of KOA were 88.61%, 85.3%, 87.37%, and 0.73%, respectively, which were higher than those of T1WI, T2WI, SE-T2WI, and FS-T2WI. The sensitivity, specificity, accuracy, and consistency of multimodal MRI in diagnosing lesions in group IV were 95%, 96.10%, 95.88%, and 0.70%, respectively, which were much higher than those in groups I, II, and III (p < 0.05). In conclusion, the LRDD algorithm shows a good image processing efficacy, and the multimodal MRI showed a good diagnosis effect on KOA, which was worthy of promotion clinically.

Hyperspectral image compressed processing: Evolutionary multi-objective optimization sparse decomposition

In the compressed processing of hyperspectral images, orthogonal matching pursuit algorithm (OMP) can be used to obtain sparse decomposition results. Aimed at the time-complex and difficulty in applying real-time processing, an evolutionary multi-objective optimization sparse decomposition algorithm for hyperspectral images is proposed. Instead of using OMP for the matching process to search optimal atoms, the proposed algorithm explores the idea of reference point non-dominated sorting genetic algorithm (NSGA) to optimize the matching process of OMP. Take two objective function to establish the multi-objective sparse decomposition optimization model, including the largest inner product of matching atoms and image residuals, and the smallest correlation between atoms. Utilize NSGA-III with advantage of high accuracy to solve the optimization model, and the implementation process of NSGA-III-OMP is presented. The experimental results and analysis carried on four hyperspectral datasets demonstrate that, compared with the state-of-the-art algorithms, the proposed NSGA-III-OMP algorithm has effectively improved the sparse decomposition performance and efficiency, and can effectively solve the sparse decomposition optimization problem of hyperspectral images.

RFI Suppression for SAR via a Dictionary-Based Nonconvex Low-Rank Minimization Framework and Its Adaptive Implementation

Synthetic aperture radar (SAR) frequently suffers from radio frequency interference (RFI) due to the simultaneous presence of numerous wireless communication signals. Recently, the narrowband RFI is found to possess the low-rank property benefiting from stable frequency occupancy, hence the reconsideration of RFI suppression as a joint sparse and low-rank optimization problem. The existing methods either use the non-sparse useful signal itself as the sparse regularizer, or employ the nuclear norm to approximate the rank function, which punishes all singular values with the same penalty via singular value thresholding (SVT), resulting in the improper punishment problem. Hence, both are consequentially subject to performance limitation. In this paper, a novel dictionary-based nonconvex low-rank minimization (DNLRM) optimization framework is proposed for RFI suppression, which concurrently considers the improvements for both the sparse regularizer and the low-rank regularizer. For the former, an over-completed dictionary is constructed, for which the sparse coefficient acts as the sparse regularizer. For the latter, the rank function is more accurately approximated by innovatively introducing the nonconvex function, for which the supergradient is synchronously used to generate the weighted penalty, thus solving the improper punishment problem. The derivation of the closed-form solution and the convergence analysis are described in detail. Additionally, the adaptive selection scheme for the model parameter is uniquely proposed for further ensuring the practicality of the DNLRM framework. The superiority of the proposed method is demonstrated via not only the RFI-free real SAR data combined with the measured RFI, but the RFI-contaminated real SAR data.

Hyperspectral Image Mixed Noise Removal Using Subspace Representation and Deep CNN Image Prior

The ever-increasing spectral resolution of hyperspectral images (HSIs) is often obtained at the cost of a decrease in the signal-to-noise ratio (SNR) of the measurements. The decreased SNR reduces the reliability of measured features or information extracted from HSIs, thus calling for effective denoising techniques. This work aims to estimate clean HSIs from observations corrupted by mixed noise (containing Gaussian noise, impulse noise, and dead-lines/stripes) by exploiting two main characteristics of hyperspectral data, namely low-rankness in the spectral domain and high correlation in the spatial domain. We take advantage of the spectral low-rankness of HSIs by representing spectral vectors in an orthogonal subspace, which is learned from observed images by a new method. Subspace representation coefficients of HSIs are learned by solving an optimization problem plugged with an image prior extracted from a neural denoising network. The proposed method is evaluated on simulated and real HSIs. An exhaustive array of experiments and comparisons with state-of-the-art denoisers were carried out.

A Non-Local Superpatch-Based Algorithm Exploiting Low Rank Prior for Restoration of Hyperspectral Images

We propose a novel algorithm for the restoration of a degraded hyperspectral image. The proposed algorithm exploits the spatial as well as the spectral redundancy of a degraded hyperspectral image in order to restore it without having any prior knowledge about the type of degradation present. Our work uses superpatches to exploit the spatial and spectral redundancies. We formulate a restoration algorithm incorporating structural similarity index measure as the data fidelity term and nuclear norm as the regularization term. The proposed algorithm is able to cope with additive Gaussian noise, signal dependent Poisson noise, mixed Poisson-Gaussian noise and can restore a hyperspectral image corrupted by dead lines and stripes. As we demonstrate with the aid of extensive experiments, our algorithm is capable of recovering the spectra even in the case of severe degradation. A comparison with the state-of-the-art low rank hyperspectral image restoration methods via experiments with real world and simulated data establishes the competitiveness of the proposed algorithm with the existing methods.

Mixed Noise Estimation Model for Optimized Kernel Minimum Noise Fraction Transformation in Hyperspectral Image Dimensionality Reduction

Dimensionality reduction (DR) is of great significance for simplifying and optimizing hyperspectral image (HSI) features. As a widely used DR method, kernel minimum noise fraction (KMNF) transformation preserves the high-order structures of the original data perfectly. However, the conventional KMNF noise estimation (KMNF-NE) uses the local regression residual of neighbourhood pixels, which depends heavily on spatial information. Due to the limited spatial resolution, there are many mixed pixels in HSI, making KMNF-NE unreliable for noise estimation and leading to poor performance in KMNF for classification on HSIs with low spatial resolution. In order to overcome this problem, a mixed noise estimation model (MNEM) is proposed in this paper for optimized KMNF (OP-KMNF). The MNEM adopts the sequential and linear combination of the Gaussian prior denoising model, median filter, and Sobel operator to estimate noise. It retains more details and edge features, making it more suitable for noise estimation in KMNF. Experiments using several HSI datasets with different spatial and spectral resolutions are conducted. The results show that, compared with some other DR methods, the improvement of OP-KMNF in average classification accuracy is up to 4%. To improve the efficiency, the OP-KMNF was implemented on graphics processing units (GPU) and sped up by about 60× compared to the central processing unit (CPU) implementation. The outcome demonstrates the significant performance of OP-KMNF in terms of classification ability and execution efficiency.

Variational Low-Rank Matrix Factorization with Multi-Patch Collaborative Learning for Hyperspectral Imagery Mixed Denoising

In this study, multi-patch collaborative learning is introduced into variational low-rank matrix factorization to suppress mixed noise in hyperspectral images (HSIs). Firstly, based on the spatial consistency and nonlocal self-similarities, the HSI is partitioned into overlapping patches with a full band. The similarity metric with fusing features is exploited to select the most similar patches and construct the corresponding collaborative patches. Secondly, considering that the latent clean HSI holds the low-rank property across the spectra, whereas the noise component does not, variational low-rank matrix factorization is proposed in the Bayesian framework for each collaborative patch. Using Gaussian distribution adaptively adjusted by a gamma distribution, the noise-free data can be learned by exploring low-rank properties of collaborative patches in the spatial/spectral domain. Additionally, the Dirichlet process Gaussian mixture model is utilized to approximate the statistical characteristics of mixed noises, which is constructed by exploiting the Gaussian distribution, the inverse Wishart distribution, and the Dirichlet process. Finally, variational inference is utilized to estimate all variables and solve the proposed model using closed-form equations. Widely used datasets with different settings are adopted to conduct experiments. The quantitative and qualitative results indicate the effectiveness and superiority of the proposed method in reducing mixed noises in HSIs.