1
|
Wu T, Fan J. Smooth Tensor Product for Tensor Completion. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2024; 33:6483-6496. [PMID: 39527431 DOI: 10.1109/tip.2024.3489272] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2024]
Abstract
Low-rank tensor completion (LRTC) has shown promise in processing incomplete visual data, yet it often overlooks the inherent local smooth structures in images and videos. Recent advances in LRTC, integrating total variation regularization to capitalize on the local smoothness, have yielded notable improvements. Nonetheless, these methods are limited to exploiting local smoothness within the original data space, neglecting the latent factor space of tensors. More seriously, there is a lack of theoretical backing for the role of local smoothness in enhancing recovery performance. In response, this paper introduces an innovative tensor completion model that concurrently leverages the global low-rank structure of the original tensor and the local smooth structure of its factor tensors. Our objective is to learn a low-rank tensor that decomposes into two factor tensors, each exhibiting sufficient local smoothness. We propose an efficient alternating direction method of multipliers to optimize our model. Further, we establish generalization error bounds for smooth factor-based tensor completion methods across various decomposition frameworks. These bounds are significantly tighter than existing baselines. We conduct extensive inpainting experiments on color images, multispectral images, and videos, which demonstrate the efficacy and superiority of our method. Additionally, our approach shows a low sensitivity to hyper-parameter settings, enhancing its convenience and reliability for practical applications.
Collapse
|
2
|
Lin CH, Liu Y, Chi CY, Hsu CC, Ren H, Quek TQS. Hyperspectral Tensor Completion Using Low-Rank Modeling and Convex Functional Analysis. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:10736-10750. [PMID: 37027554 DOI: 10.1109/tnnls.2023.3243808] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Hyperspectral tensor completion (HTC) for remote sensing, critical for advancing space exploration and other satellite imaging technologies, has drawn considerable attention from recent machine learning community. Hyperspectral image (HSI) contains a wide range of narrowly spaced spectral bands hence forming unique electrical magnetic signatures for distinct materials, and thus plays an irreplaceable role in remote material identification. Nevertheless, remotely acquired HSIs are of low data purity and quite often incompletely observed or corrupted during transmission. Therefore, completing the 3-D hyperspectral tensor, involving two spatial dimensions and one spectral dimension, is a crucial signal processing task for facilitating the subsequent applications. Benchmark HTC methods rely on either supervised learning or nonconvex optimization. As reported in recent machine learning literature, John ellipsoid (JE) in functional analysis is a fundamental topology for effective hyperspectral analysis. We therefore attempt to adopt this key topology in this work, but this induces a dilemma that the computation of JE requires the complete information of the entire HSI tensor that is, however, unavailable under the HTC problem setting. We resolve the dilemma, decouple HTC into convex subproblems ensuring computational efficiency, and show state-of-the-art HTC performances of our algorithm. We also demonstrate that our method has improved the subsequent land cover classification accuracy on the recovered hyperspectral tensor.
Collapse
|
3
|
Feng K, Zeng H, Zhao Y, Kong SG, Bu Y. Unsupervised Spectral Demosaicing With Lightweight Spectral Attention Networks. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2024; 33:1655-1669. [PMID: 38386587 DOI: 10.1109/tip.2024.3364064] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/24/2024]
Abstract
This paper presents a deep learning-based spectral demosaicing technique trained in an unsupervised manner. Many existing deep learning-based techniques relying on supervised learning with synthetic images, often underperform on real-world images, especially as the number of spectral bands increases. This paper presents a comprehensive unsupervised spectral demosaicing (USD) framework based on the characteristics of spectral mosaic images. This framework encompasses a training method, model structure, transformation strategy, and a well-fitted model selection strategy. To enable the network to dynamically model spectral correlation while maintaining a compact parameter space, we reduce the complexity and parameters of the spectral attention module. This is achieved by dividing the spectral attention tensor into spectral attention matrices in the spatial dimension and spectral attention vector in the channel dimension. This paper also presents Mosaic 25 , a real 25-band hyperspectral mosaic image dataset featuring various objects, illuminations, and materials for benchmarking purposes. Extensive experiments on both synthetic and real-world datasets demonstrate that the proposed method outperforms conventional unsupervised methods in terms of spatial distortion suppression, spectral fidelity, robustness, and computational cost. Our code and dataset are publicly available at https://github.com/polwork/Unsupervised-Spectral-Demosaicing.
Collapse
|
4
|
Zeng J, Qiu Y, Ma Y, Wang A, Zhao Q. A Novel Tensor Ring Sparsity Measurement for Image Completion. ENTROPY (BASEL, SWITZERLAND) 2024; 26:105. [PMID: 38392360 PMCID: PMC10887661 DOI: 10.3390/e26020105] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/12/2023] [Revised: 01/15/2024] [Accepted: 01/22/2024] [Indexed: 02/24/2024]
Abstract
As a promising data analysis technique, sparse modeling has gained widespread traction in the field of image processing, particularly for image recovery. The matrix rank, served as a measure of data sparsity, quantifies the sparsity within the Kronecker basis representation of a given piece of data in the matrix format. Nevertheless, in practical scenarios, much of the data are intrinsically multi-dimensional, and thus, using a matrix format for data representation will inevitably yield sub-optimal outcomes. Tensor decomposition (TD), as a high-order generalization of matrix decomposition, has been widely used to analyze multi-dimensional data. In a direct generalization to the matrix rank, low-rank tensor modeling has been developed for multi-dimensional data analysis and achieved great success. Despite its efficacy, the connection between TD rank and the sparsity of the tensor data is not direct. In this work, we introduce a novel tensor ring sparsity measurement (TRSM) for measuring the sparsity of the tensor. This metric relies on the tensor ring (TR) Kronecker basis representation of the tensor, providing a unified interpretation akin to matrix sparsity measurements, wherein the Kronecker basis serves as the foundational representation component. Moreover, TRSM can be efficiently computed by the product of the ranks of the mode-2 unfolded TR-cores. To enhance the practical performance of TRSM, the folded-concave penalty of the minimax concave penalty is introduced as a nonconvex relaxation. Lastly, we extend the TRSM to the tensor completion problem and use the alternating direction method of the multipliers scheme to solve it. Experiments on image and video data completion demonstrate the effectiveness of the proposed method.
Collapse
Affiliation(s)
- Junhua Zeng
- School of Automation, Guangdong University of Technology, Guangzhou 510006, China
- RIKEN Center for Advanced Intelligence Project (AIP), Tokyo 103-0027, Japan
| | - Yuning Qiu
- School of Automation, Guangdong University of Technology, Guangzhou 510006, China
- RIKEN Center for Advanced Intelligence Project (AIP), Tokyo 103-0027, Japan
| | - Yumeng Ma
- School of Automation, Guangdong University of Technology, Guangzhou 510006, China
| | - Andong Wang
- RIKEN Center for Advanced Intelligence Project (AIP), Tokyo 103-0027, Japan
| | - Qibin Zhao
- School of Automation, Guangdong University of Technology, Guangzhou 510006, China
- RIKEN Center for Advanced Intelligence Project (AIP), Tokyo 103-0027, Japan
| |
Collapse
|
5
|
Xie M, Liu X, Yang X, Cai W. Multichannel Image Completion With Mixture Noise: Adaptive Sparse Low-Rank Tensor Subspace Meets Nonlocal Self-Similarity. IEEE TRANSACTIONS ON CYBERNETICS 2023; 53:7521-7534. [PMID: 35580099 DOI: 10.1109/tcyb.2022.3169800] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Multichannel image completion with mixture noise is a common but complex problem in the fields of machine learning, image processing, and computer vision. Most existing algorithms devote to explore global low-rank information and fail to optimize local and joint-mode structures, which may lead to oversmooth restoration results or lower quality restoration details. In this study, we propose a novel model to deal with multichannel image completion with mixture noise based on adaptive sparse low-rank tensor subspace and nonlocal self-similarity (ASLTS-NS). In the proposed model, a nonlocal similar patch matching framework cooperating with Tucker decomposition is used to explore information of global and joint modes and optimize the local structure for improving restoration quality. In order to enhance the robustness of low-rank decomposition to data missing and mixture noise, we present an adaptive sparse low-rank regularization to construct robust tensor subspace for self-weighing importance of different modes and capturing a stable inherent structure. In addition, joint tensor Frobenius and l1 regularizations are exploited to control two different types of noise. Based on alternating directions method of multipliers (ADMM), a convergent learning algorithm is designed to solve this model. Experimental results on three different types of multichannel image sets demonstrate the advantages of ASLTS-NS under five complex scenarios.
Collapse
|
6
|
Huang S, Zhang H, Xue J, Pizurica A. Heterogeneous Regularization-Based Tensor Subspace Clustering for Hyperspectral Band Selection. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2023; 34:9259-9273. [PMID: 35294365 DOI: 10.1109/tnnls.2022.3157711] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Band selection (BS) reduces effectively the spectral dimension of a hyperspectral image (HSI) by selecting relatively few representative bands, which allows efficient processing in subsequent tasks. Existing unsupervised BS methods based on subspace clustering are built on matrix-based models, where each band is reshaped as a vector. They encode the correlation of data only in the spectral mode (dimension) and neglect strong correlations between different modes, i.e., spatial modes and spectral mode. Another issue is that the subspace representation of bands is performed in the raw data space, where the dimension is often excessively high, resulting in a less efficient and less robust performance. To address these issues, in this article, we propose a tensor-based subspace clustering model for hyperspectral BS. Our model is developed on the well-known Tucker decomposition. The three factor matrices and a core tensor in our model encode jointly the multimode correlations of HSI, avoiding effectively to destroy the tensor structure and information loss. In addition, we propose well-motivated heterogeneous regularizations (HRs) on the factor matrices by taking into account the important local and global properties of HSI along three dimensions, which facilitates the learning of the intrinsic cluster structure of bands in the low-dimensional subspaces. Instead of learning the correlations of bands in the original domain, a common way for the matrix-based models, our model learns naturally the band correlations in a low-dimensional latent feature space, which is derived by the projections of two factor matrices associated with spatial dimensions, leading to a computationally efficient model. More importantly, the latent feature space is learned in a unified framework. We also develop an efficient algorithm to solve the resulting model. Experimental results on benchmark datasets demonstrate that our model yields improved performance compared to the state-of-the-art.
Collapse
|
7
|
Zhang C, Zhang Z, Yu D, Cheng Q, Shan S, Li M, Mou L, Yang X, Ma X. Unsupervised band selection of medical hyperspectral images guided by data gravitation and weak correlation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 240:107721. [PMID: 37506601 DOI: 10.1016/j.cmpb.2023.107721] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 07/06/2023] [Accepted: 07/12/2023] [Indexed: 07/30/2023]
Abstract
BACKGROUND AND OBJECTIVE Medical hyperspectral images (MHSIs) are used for a contact-free examination of patients without harmful radiation. However, high-dimensionality images contain large amounts of data that are sparsely distributed in a high-dimensional space, which leads to the "curse of dimensionality" (called Hughes' phenomenon) and increases the complexity and cost of data processing and storage. Hence, there is a need for spectral dimensionality reduction before the clinical application of MHSIs. Some dimensionality-reducing strategies have been proposed; however, they distort the data within MHSIs. METHODS To compress dimensionality without destroying the original data structure, we propose a method that involves data gravitation and weak correlation-based ranking (DGWCR) for removing bands of noise from MHSIs while clustering signal-containing bands. Band clustering is done by using the connection centre evolution (CCE) algorithm and selecting the most representative bands in each cluster based on the composite force. The bands within the clusters are ranked using the new entropy-containing matrix, and a global ranking of bands is obtained by applying an S-shaped strategy. The source code is available at https://www.github.com/zhangchenglong1116/DGWCR. RESULTS Upon feeding the reduced-dimensional images into various classifiers, the experimental results demonstrated that the small number of bands selected by the proposed DGWCR consistently achieved higher classification accuracy than the original data. Unlike other reference methods (e.g. the latest deep-learning-based strategies), DGWCR chooses the spectral bands with the least redundancy and greatest discrimination. CONCLUSION In this study, we present a method for efficient band selection for MHSIs that alleviates the "curse of dimensionality". Experiments were validated with three MHSIs in the human brain, and they outperformed several other band selection methods, demonstrating the clinical potential of DGWCR.
Collapse
Affiliation(s)
- Chenglong Zhang
- School of Control Science and Engineering, Shandong University, Jinan 250061, China
| | - Zhimin Zhang
- School of Control Science and Engineering, Shandong University, Jinan 250061, China
| | - Dexin Yu
- Radiology Department, Qilu Hospital of Shandong University, Jinan 250000, China
| | - Qiyuan Cheng
- Medical Engineering Department, Shandong Provincial Hospital affiliated to Shandong First Medical University, Jinan 250021, China
| | - Shihao Shan
- School of Control Science and Engineering, Shandong University, Jinan 250061, China
| | - Mengjiao Li
- School of Control Science and Engineering, Shandong University, Jinan 250061, China
| | - Lichao Mou
- Chair of Data Science in Earth Observation, Technical University of Munich (TUM), Munich, 80333, Germany
| | - Xiaoli Yang
- School of Control Science and Engineering, Shandong University, Jinan 250061, China; Weifang Xinli Superconducting Magnet Technology Co., Ltd, Weifang 261005, China.
| | - Xiaopeng Ma
- School of Control Science and Engineering, Shandong University, Jinan 250061, China.
| |
Collapse
|
8
|
Xue J, Zhao Y, Bu Y, Chan JCW, Kong SG. When Laplacian Scale Mixture Meets Three-Layer Transform: A Parametric Tensor Sparsity for Tensor Completion. IEEE TRANSACTIONS ON CYBERNETICS 2022; 52:13887-13901. [PMID: 35081033 DOI: 10.1109/tcyb.2021.3140148] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Recently, tensor sparsity modeling has achieved great success in the tensor completion (TC) problem. In real applications, the sparsity of a tensor can be rationally measured by low-rank tensor decomposition. However, existing methods either suffer from limited modeling power in estimating accurate rank or have difficulty in depicting hierarchical structure underlying such data ensembles. To address these issues, we propose a parametric tensor sparsity measure model, which encodes the sparsity for a general tensor by Laplacian scale mixture (LSM) modeling based on three-layer transform (TLT) for factor subspace prior with Tucker decomposition. Specifically, the sparsity of a tensor is first transformed into factor subspace, and then factor sparsity in the gradient domain is used to express the local similarity in within-mode. To further refine the sparsity, we adopt LSM by the transform learning scheme to self-adaptively depict deeper layer structured sparsity, in which the transformed sparse matrices in the sense of a statistical model can be modeled as the product of a Laplacian vector and a hidden positive scalar multiplier. We call the method as parametric tensor sparsity delivered by LSM-TLT. By a progressive transformation operator, we formulate the LSM-TLT model and use it to address the TC problem, and then the alternating direction method of multipliers-based optimization algorithm is designed to solve the problem. The experimental results on RGB images, hyperspectral images (HSIs), and videos demonstrate the proposed method outperforms state of the arts.
Collapse
|
9
|
Zhang H, Zhao XL, Jiang TX, Ng MK, Huang TZ. Multiscale Feature Tensor Train Rank Minimization for Multidimensional Image Recovery. IEEE TRANSACTIONS ON CYBERNETICS 2022; 52:13395-13410. [PMID: 34543216 DOI: 10.1109/tcyb.2021.3108847] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
The general tensor-based methods can recover missing values of multidimensional images by exploiting the low-rankness on the pixel level. However, especially when considerable pixels of an image are missing, the low-rankness is not reliable on the pixel level, resulting in some details losing in their results, which hinders the performance of subsequent image applications (e.g., image recognition and segmentation). In this article, we suggest a novel multiscale feature (MSF) tensorization by exploiting the MSFs of multidimensional images, which not only helps to recover the missing values on a higher level, that is, the feature level but also benefits subsequent image applications. By exploiting the low-rankness of the resulting MSF tensor constructed by the new tensorization, we propose the convex and nonconvex MSF tensor train rank minimization (MSF-TT) to conjointly recover the MSF tensor and the corresponding original tensor in a unified framework. We develop the alternating directional method of multipliers (ADMMs) to solve the convex MSF-TT and the proximal alternating minimization (PAM) to solve the nonconvex MSF-TT. Moreover, we establish the theoretical guarantee of convergence for the PAM algorithm. Numerical examples of real-world multidimensional images show that the proposed MSF-TT outperforms other compared approaches in image recovery and the recovered MSF tensor can benefit the subsequent image recognition.
Collapse
|
10
|
Xue J, Zhao Y, Huang S, Liao W, Chan JCW, Kong SG. Multilayer Sparsity-Based Tensor Decomposition for Low-Rank Tensor Completion. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:6916-6930. [PMID: 34143740 DOI: 10.1109/tnnls.2021.3083931] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Existing methods for tensor completion (TC) have limited ability for characterizing low-rank (LR) structures. To depict the complex hierarchical knowledge with implicit sparsity attributes hidden in a tensor, we propose a new multilayer sparsity-based tensor decomposition (MLSTD) for the low-rank tensor completion (LRTC). The method encodes the structured sparsity of a tensor by the multiple-layer representation. Specifically, we use the CANDECOMP/PARAFAC (CP) model to decompose a tensor into an ensemble of the sum of rank-1 tensors, and the number of rank-1 components is easily interpreted as the first-layer sparsity measure. Presumably, the factor matrices are smooth since local piecewise property exists in within-mode correlation. In subspace, the local smoothness can be regarded as the second-layer sparsity. To describe the refined structures of factor/subspace sparsity, we introduce a new sparsity insight of subspace smoothness: a self-adaptive low-rank matrix factorization (LRMF) scheme, called the third-layer sparsity. By the progressive description of the sparsity structure, we formulate an MLSTD model and embed it into the LRTC problem. Then, an effective alternating direction method of multipliers (ADMM) algorithm is designed for the MLSTD minimization problem. Various experiments in RGB images, hyperspectral images (HSIs), and videos substantiate that the proposed LRTC methods are superior to state-of-the-art methods.
Collapse
|
11
|
Tensor ring decomposition-based model with interpretable gradient factors regularization for tensor completion. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.110094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
|
12
|
Huang JJ, Dragotti PL. WINNet: Wavelet-Inspired Invertible Network for Image Denoising. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2022; 31:4377-4392. [PMID: 35759598 DOI: 10.1109/tip.2022.3184845] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Image denoising aims to restore a clean image from an observed noisy one. Model-based image denoising approaches can achieve good generalization ability over different noise levels and are with high interpretability. Learning-based approaches are able to achieve better results, but usually with weaker generalization ability and interpretability. In this paper, we propose a wavelet-inspired invertible network (WINNet) to combine the merits of the wavelet-based approaches and learning-based approaches. The proposed WINNet consists of K -scale of lifting inspired invertible neural networks (LINNs) and sparsity-driven denoising networks together with a noise estimation network. The network architecture of LINNs is inspired by the lifting scheme in wavelets. LINNs are used to learn a non-linear redundant transform with perfect reconstruction property to facilitate noise removal. The denoising network implements a sparse coding process for denoising. The noise estimation network estimates the noise level from the input image which will be used to adaptively adjust the soft-thresholds in LINNs. The forward transform of LINNs produces a redundant multi-scale representation for denoising. The denoised image is reconstructed using the inverse transform of LINNs with the denoised detail channels and the original coarse channel. The simulation results show that the proposed WINNet method is highly interpretable and has strong generalization ability to unseen noise levels. It also achieves competitive results in the non-blind/blind image denoising and in image deblurring.
Collapse
|
13
|
Xie M, Liu X, Yang X. A Nonlocal Self-Similarity-Based Weighted Tensor Low-Rank Decomposition for Multichannel Image Completion With Mixture Noise. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; PP:73-87. [PMID: 35544496 DOI: 10.1109/tnnls.2022.3172184] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Multichannel image completion with mixture noise is a challenging problem in the fields of machine learning, computer vision, image processing, and data mining. Traditional image completion models are not appropriate to deal with this problem directly since their reconstruction priors may mismatch corruption priors. To address this issue, we propose a novel nonlocal self-similarity-based weighted tensor low-rank decomposition (NSWTLD) model that can achieve global optimization and local enhancement. In the proposed model, based on the corruption priors and the reconstruction priors, a pixel weighting strategy is given to characterize the joint effects of missing data, the Gaussian noise, and the impulse noise. To discover and utilize the accurate nonlocal self-similarity information to enhance the restoration quality of the details, the traditional nonlocal learning framework is optimized by employing improved index determination of patch group and handling strip noise caused by patch overlapping. In addition, an efficient and convergent algorithm is presented to solve the NSWTLD model. Comprehensive experiments are conducted on four types of multichannel images under various corruption scenarios. The results demonstrate the efficiency and effectiveness of the proposed model.
Collapse
|
14
|
Qin W, Wang H, Zhang F, Wang J, Luo X, Huang T. Low-Rank High-Order Tensor Completion With Applications in Visual Data. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2022; 31:2433-2448. [PMID: 35259105 DOI: 10.1109/tip.2022.3155949] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Recently, tensor Singular Value Decomposition (t-SVD)-based low-rank tensor completion (LRTC) has achieved unprecedented success in addressing various pattern analysis issues. However, existing studies mostly focus on third-order tensors while order- d ( d ≥ 4 ) tensors are commonly encountered in real-world applications, like fourth-order color videos, fourth-order hyper-spectral videos, fifth-order light-field images, and sixth-order bidirectional texture functions. Aiming at addressing this critical issue, this paper establishes an order- d tensor recovery framework including the model, algorithm and theories by innovatively developing a novel algebraic foundation for order- d t-SVD, thereby achieving exact completion for any order- d low t-SVD rank tensors with missing values with an overwhelming probability. Emperical studies on synthetic data and real-world visual data illustrate that compared with other state-of-the-art recovery frameworks, the proposed one achieves highly competitive performance in terms of both qualitative and quantitative metrics. In particular, as the observed data density becomes low, i.e., about 10%, the proposed recovery framework is still significantly better than its peers. The code of our algorithm is released at https://github.com/Qinwenjinswu/TIP-Code.
Collapse
|
15
|
Li L, Li W, Qu Y, Zhao C, Tao R, Du Q. Prior-Based Tensor Approximation for Anomaly Detection in Hyperspectral Imagery. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:1037-1050. [PMID: 33296310 DOI: 10.1109/tnnls.2020.3038659] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
The key to hyperspectral anomaly detection is to effectively distinguish anomalies from the background, especially in the case that background is complex and anomalies are weak. Hyperspectral imagery (HSI) as an image-spectrum merging cube data can be intrinsically represented as a third-order tensor that integrates spectral information and spatial information. In this article, a prior-based tensor approximation (PTA) is proposed for hyperspectral anomaly detection, in which HSI is decomposed into a background tensor and an anomaly tensor. In the background tensor, a low-rank prior is incorporated into spectral dimension by truncated nuclear norm regularization, and a piecewise-smooth prior on spatial dimension can be embedded by a linear total variation-norm regularization. For anomaly tensor, it is unfolded along spectral dimension coupled with spatial group sparse prior that can be represented by the l2,1 -norm regularization. In the designed method, all the priors are integrated into a unified convex framework, and the anomalies can be finally determined by the anomaly tensor. Experimental results validated on several real hyperspectral data sets demonstrate that the proposed algorithm outperforms some state-of-the-art anomaly detection methods.
Collapse
|
16
|
Yokota T, Hontani H, Zhao Q, Cichocki A. Manifold Modeling in Embedded Space: An Interpretable Alternative to Deep Image Prior. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:1022-1036. [PMID: 33275587 DOI: 10.1109/tnnls.2020.3037923] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Deep image prior (DIP), which uses a deep convolutional network (ConvNet) structure as an image prior, has attracted wide attention in computer vision and machine learning. DIP empirically shows the effectiveness of the ConvNet structures for various image restoration applications. However, why the DIP works so well is still unknown. In addition, the reason why the convolution operation is useful in image reconstruction, or image enhancement is not very clear. This study tackles this ambiguity of ConvNet/DIP by proposing an interpretable approach that divides the convolution into "delay embedding" and "transformation" (i.e., encoder-decoder). Our approach is a simple, but essential, image/tensor modeling method that is closely related to self-similarity. The proposed method is called manifold modeling in embedded space (MMES) since it is implemented using a denoising autoencoder in combination with a multiway delay-embedding transform. In spite of its simplicity, MMES can obtain quite similar results to DIP on image/tensor completion, super-resolution, deconvolution, and denoising. In addition, MMES is proven to be competitive with DIP, as shown in our experiments. These results can also facilitate interpretation/characterization of DIP from the perspective of a "low-dimensional patch-manifold prior."
Collapse
|
17
|
Zhang H, Chen H, Yang G, Zhang L. LR-Net: Low-Rank Spatial-Spectral Network for Hyperspectral Image Denoising. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2021; 30:8743-8758. [PMID: 34665726 DOI: 10.1109/tip.2021.3120037] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Due to the physical limitations of the imaging devices, hyperspectral images (HSIs) are commonly distorted by a mixture of Gaussian noise, impulse noise, stripes, and dead lines, leading to the decline in the performance of unmixing, classification, and other subsequent applications. In this paper, we propose a novel end-to-end low-rank spatial-spectral network (LR-Net) for the removal of the hybrid noise in HSIs. By integrating the low-rank physical property into a deep convolutional neural network (DCNN), the proposed LR-Net simultaneously enjoys the strong feature representation ability from DCNN and the implicit physical constraint of clean HSIs. Firstly, spatial-spectral atrous blocks (SSABs) are built to exploit spatial-spectral features of HSIs. Secondly, these spatial-spectral features are forwarded to a multi-atrous block (MAB) to aggregate the context in different receptive fields. Thirdly, the contextual features and spatial-spectral features from different levels are concatenated before being fed into a plug-and-play low-rank module (LRM) for feature reconstruction. With the help of the LRM, the workflow of low-rank matrix reconstruction can be streamlined in a differentiable manner. Finally, the low-rank features are utilized to capture the latent semantic relationships of the HSIs to recover clean HSIs. Extensive experiments on both simulated and real-world datasets were conducted. The experimental results show that the LR-Net outperforms other state-of-the-art denoising methods in terms of evaluation metrics and visual assessments. Particularly, through the collaborative integration of DCNNs and the low-rank property, the LR-Net shows strong stability and capacity for generalization.
Collapse
|
18
|
Sarkar S, Sahay RR. A Non-Local Superpatch-Based Algorithm Exploiting Low Rank Prior for Restoration of Hyperspectral Images. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2021; 30:6335-6348. [PMID: 34232876 DOI: 10.1109/tip.2021.3093780] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
We propose a novel algorithm for the restoration of a degraded hyperspectral image. The proposed algorithm exploits the spatial as well as the spectral redundancy of a degraded hyperspectral image in order to restore it without having any prior knowledge about the type of degradation present. Our work uses superpatches to exploit the spatial and spectral redundancies. We formulate a restoration algorithm incorporating structural similarity index measure as the data fidelity term and nuclear norm as the regularization term. The proposed algorithm is able to cope with additive Gaussian noise, signal dependent Poisson noise, mixed Poisson-Gaussian noise and can restore a hyperspectral image corrupted by dead lines and stripes. As we demonstrate with the aid of extensive experiments, our algorithm is capable of recovering the spectra even in the case of severe degradation. A comparison with the state-of-the-art low rank hyperspectral image restoration methods via experiments with real world and simulated data establishes the competitiveness of the proposed algorithm with the existing methods.
Collapse
|
19
|
Xue J, Zhao YQ, Bu Y, Liao W, Chan JCW, Philips W. Spatial-Spectral Structured Sparse Low-Rank Representation for Hyperspectral Image Super-Resolution. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2021; 30:3084-3097. [PMID: 33596175 DOI: 10.1109/tip.2021.3058590] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Hyperspectral image super-resolution by fusing high-resolution multispectral image (HR-MSI) and low-resolution hyperspectral image (LR-HSI) aims at reconstructing high resolution spatial-spectral information of the scene. Existing methods mostly based on spectral unmixing and sparse representation are often developed from a low-level vision task perspective, they cannot sufficiently make use of the spatial and spectral priors available from higher-level analysis. To this issue, this paper proposes a novel HSI super-resolution method that fully considers the spatial/spectral subspace low-rank relationships between available HR-MSI/LR-HSI and latent HSI. Specifically, it relies on a new subspace clustering method named "structured sparse low-rank representation" (SSLRR), to represent the data samples as linear combinations of the bases in a given dictionary, where the sparse structure is induced by low-rank factorization for the affinity matrix. Then we exploit the proposed SSLRR model to learn the SSLRR along spatial/spectral domain from the MSI/HSI inputs. By using the learned spatial and spectral low-rank structures, we formulate the proposed HSI super-resolution model as a variational optimization problem, which can be readily solved by the ADMM algorithm. Compared with state-of-the-art hyperspectral super-resolution methods, the proposed method shows better performance on three benchmark datasets in terms of both visual and quantitative evaluation.
Collapse
|
20
|
|
21
|
Hyperspectral Image Denoising Based on Nonlocal Low-Rank and TV Regularization. REMOTE SENSING 2020. [DOI: 10.3390/rs12121956] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Hyperspectral image (HSI) acquisitions are degraded by various noises, among which additive Gaussian noise may be the worst-case, as suggested by information theory. In this paper, we present a novel tensor-based HSI denoising approach by fully identifying the intrinsic structures of the clean HSI and the noise. Specifically, the HSI is first divided into local overlapping full-band patches (FBPs), then the nonlocal similar patches in each group are unfolded and stacked into a new third order tensor. As this tensor shows a stronger low-rank property than the original degraded HSI, the tensor weighted nuclear norm minimization (TWNNM) on the constructed tensor can effectively separate the low-rank clean HSI patches. In addition, a regularization strategy with spatial–spectral total variation (SSTV) is utilized to ensure the global spatial–spectral smoothness in both spatial and spectral domains. Our method is designed to model the spatial–spectral non-local self-similarity and global spatial–spectral smoothness simultaneously. Experiments conducted on simulated and real datasets show the superiority of the proposed method.
Collapse
|
22
|
Abstract
Sparse subspace clustering (SSC) techniques provide the state-of-the-art in clustering of hyperspectral images (HSIs). However, their computational complexity hinders their applicability to large-scale HSIs. In this paper, we propose a large-scale SSC-based method, which can effectively process large HSIs while also achieving improved clustering accuracy compared to the current SSC methods. We build our approach based on an emerging concept of sketched subspace clustering, which was to our knowledge not explored at all in hyperspectral imaging yet. Moreover, there are only scarce results on any large-scale SSC approaches for HSI. We show that a direct application of sketched SSC does not provide a satisfactory performance on HSIs but it does provide an excellent basis for an effective and elegant method that we build by extending this approach with a spatial prior and deriving the corresponding solver. In particular, a random matrix constructed by the Johnson-Lindenstrauss transform is first used to sketch the self-representation dictionary as a compact dictionary, which significantly reduces the number of sparse coefficients to be solved, thereby reducing the overall complexity. In order to alleviate the effect of noise and within-class spectral variations of HSIs, we employ a total variation constraint on the coefficient matrix, which accounts for the spatial dependencies among the neighbouring pixels. We derive an efficient solver for the resulting optimization problem, and we theoretically prove its convergence property under mild conditions. The experimental results on real HSIs show a notable improvement in comparison with the traditional SSC-based methods and the state-of-the-art methods for clustering of large-scale images.
Collapse
|
23
|
Global and Local Tensor Sparse Approximation Models for Hyperspectral Image Destriping. REMOTE SENSING 2020. [DOI: 10.3390/rs12040704] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
This paper presents a global and local tensor sparse approximation (GLTSA) model for removing the stripes in hyperspectral images (HSIs). HSIs can easily be degraded by unwanted stripes. Two intrinsic characteristics of the stripes are (1) global sparse distribution and (2) local smoothness along the stripe direction. Stripe-free hyperspectral images are smooth in spatial domain, with strong spectral correlation. Existing destriping approaches often do not fully investigate such intrinsic characteristics of the stripes in spatial and spectral domains simultaneously. Those methods may generate new artifacts in extreme areas, causing spectral distortion. The proposed GLTSA model applies two ℓ 0 -norm regularizers to the stripe components and along-stripe gradient to improve the destriping performance. Two ℓ 1 -norm regularizers are applied to the gradients of clean image in spatial and spectral domains. The double non-convex functions in GLTSA are converted to single non-convex function by mathematical program with equilibrium constraints (MPEC). Experiment results demonstrate that GLTSA is effective and outperforms existing competitive matrix-based and tensor-based destriping methods in visual, as well as quantitative, evaluation measures.
Collapse
|