Robust Hyperspectral Unmixing With Correntropy-Based Metric

Stochastic Gradient Descent for Kernel-Based Maximum Correntropy Criterion

Maximum correntropy criterion (MCC) has been an important method in machine learning and signal processing communities since it was successfully applied in various non-Gaussian noise scenarios. In comparison with the classical least squares method (LS), which takes only the second-order moment of models into consideration and belongs to the convex optimization problem, MCC captures the high-order information of models that play crucial roles in robust learning, which is usually accompanied by solving the non-convexity optimization problems. As we know, the theoretical research on convex optimizations has made significant achievements, while theoretical understandings of non-convex optimization are still far from mature. Motivated by the popularity of the stochastic gradient descent (SGD) for solving nonconvex problems, this paper considers SGD applied to the kernel version of MCC, which has been shown to be robust to outliers and non-Gaussian data in nonlinear structure models. As the existing theoretical results for the SGD algorithm applied to the kernel MCC are not well established, we present the rigorous analysis for the convergence behaviors and provide explicit convergence rates under some standard conditions. Our work can fill the gap between optimization process and convergence during the iterations: the iterates need to converge to the global minimizer while the obtained estimator cannot ensure the global optimality in the learning process.

Accurate identification of single-cell types via correntropy-based Sparse PCA combining hypergraph and fusion similarity

The advent of single-cell RNA sequencing (scRNA-seq) technology enables researchers to gain deep insights into cellular heterogeneity. However, the high dimensionality and noise of scRNA-seq data pose significant challenges to clustering. Therefore, we propose a new single-cell type identification method, called CHLSPCA, to address these challenges. In this model, we innovatively combine correntropy with PCA to address the noise and outliers inherent in scRNA-seq data. Meanwhile, we integrate the hypergraph into the model to extract more valuable information from the local structure of the original data. Subsequently, to capture crucial similarity information not considered by the PCA model, we employ the Gaussian kernel function and the Euclidean metric to mine the similarity information between cells, and incorporate this information into the model as the similarity constraint. Furthermore, the principal components (PCs) of PCA are very dense. A new sparse constraint is introduced into the model to gain sparse PCs. Finally, based on the principal direction matrix learned from CHLSPCA, we conduct extensive downstream analyses on real scRNA-seq datasets. The experimental results show that CHLSPCA performs better than many popular clustering methods and is expected to promote the understanding of cellular heterogeneity in scRNA-seq data analysis and support biomedical research.

Fast Rates of Gaussian Empirical Gain Maximization With Heavy-Tailed Noise

In a regression setup, we study in this brief the performance of Gaussian empirical gain maximization (EGM), which includes a broad variety of well-established robust estimation approaches. In particular, we conduct a refined learning theory analysis for Gaussian EGM, investigate its regression calibration properties, and develop improved convergence rates in the presence of heavy-tailed noise. To achieve these purposes, we first introduce a new weak moment condition that could accommodate the cases where the noise distribution may be heavy-tailed. Based on the moment condition, we then develop a novel comparison theorem that can be used to characterize the regression calibration properties of Gaussian EGM. It also plays an essential role in deriving improved convergence rates. Therefore, the present study broadens our theoretical understanding of Gaussian EGM.

Correntropy-Based Hypergraph Regularized NMF for Clustering and Feature Selection on Multi-Cancer Integrated Data

Non-negative matrix factorization (NMF) has become one of the most powerful methods for clustering and feature selection. However, the performance of the traditional NMF method severely degrades when the data contain noises and outliers or the manifold structure of the data is not taken into account. In this article, a novel method called correntropy-based hypergraph regularized NMF (CHNMF) is proposed to solve the above problem. Specifically, we use the correntropy instead of the Euclidean norm in the loss term of CHNMF, which will improve the robustness of the algorithm. And the hypergraph regularization term is also applied to the objective function, which can explore the high-order geometric information in more sample points. Then, the half-quadratic (HQ) optimization technique is adopted to solve the complex optimization problem of CHNMF. Finally, extensive experimental results on multi-cancer integrated data indicate that the proposed CHNMF method is superior to other state-of-the-art methods for clustering and feature selection.

Maximum Likelihood Estimation Based Nonnegative Matrix Factorization for Hyperspectral Unmixing

Hyperspectral unmixing (HU) is a research hotspot of hyperspectral remote sensing technology. As a classical HU method, the nonnegative matrix factorization (NMF) unmixing method can decompose an observed hyperspectral data matrix into the product of two nonnegative matrices, i.e., endmember and abundance matrices. Because the objective function of NMF is the traditional least-squares function, NMF is sensitive to noise. In order to improve the robustness of NMF, this paper proposes a maximum likelihood estimation (MLE) based NMF model (MLENMF) for unmixing of hyperspectral images (HSIs), which substitutes the least-squares objective function in traditional NMF by a robust MLE-based loss function. Experimental results on a simulated and two widely used real hyperspectral data sets demonstrate the superiority of our MLENMF over existing NMF methods.

Estimation of the Number of Endmembers in Hyperspectral Images Using Agglomerative Clustering

Many tasks in hyperspectral imaging, such as spectral unmixing and sub-pixel matching, require knowing how many substances or materials are present in the scene captured by a hyperspectral image. In this paper, we present an algorithm that estimates the number of materials in the scene using agglomerative clustering. The algorithm is based on the assumption that a valid clustering of the image has one cluster for each different material. After reducing the dimensionality of the hyperspectral image, the proposed method obtains an initial clustering using K-means. In this stage, cluster densities are estimated using Independent Component Analysis. Based on the K-means result, a model-based agglomerative clustering is performed, which provides a hierarchy of clusterings. Finally, a validation algorithm selects a clustering of the hierarchy; the number of clusters it contains is the estimated number of materials. Besides estimating the number of endmembers, the proposed method can approximately obtain the endmember (or spectrum) of each material by computing the centroid of its corresponding cluster. We have tested the proposed method using several hyperspectral images. The results show that the proposed method obtains approximately the number of materials that these images contain.

A Graph Regularized Multilinear Mixing Model for Nonlinear Hyperspectral Unmixing

Spectral unmixing of hyperspectral images is an important issue in the fields of remotesensing. Jointly exploring the spectral and spatial information embedded in the data is helpful toenhance the consistency between mixing/unmixing models and real scenarios. This paper proposesa graph regularized nonlinear unmixing method based on the recent multilinear mixing model(MLM). The MLM takes account of all orders of interactions between endmembers, and indicates thepixel-wise nonlinearity with a single probability parameter. By incorporating the Laplacian graphregularizers, the proposed method exploits the underlying manifold structure of the pixels’ spectra,in order to augment the estimations of both abundances and nonlinear probability parameters.Besides the spectrum-based regularizations, the sparsity of abundances is also incorporated for theproposed model. The resulting optimization problem is addressed by using the alternating directionmethod of multipliers (ADMM), yielding the so-called graph regularized MLM (G-MLM) algorithm.To implement the proposed method on large hypersepectral images in real world, we proposeto utilize a superpixel construction approach before unmixing, and then apply G-MLM on eachsuperpixel. The proposed methods achieve superior unmixing performances to state-of-the-artstrategies in terms of both abundances and probability parameters, on both synthetic and real datasets.

A Novel Hyperspectral Endmember Extraction Algorithm Based on Online Robust Dictionary Learning

Due to the sparsity of hyperspectral images, the dictionary learning framework has been applied in hyperspectral endmember extraction. However, current endmember extraction methods based on dictionary learning are not robust enough in noisy environments. To solve this problem, this paper proposes a novel endmember extraction approach based on online robust dictionary learning, termed EEORDL. Because of the large scale of the hyperspectral image (HSI) data, an online scheme is introduced to reduce the computational time of dictionary learning. In the proposed algorithm, a new form of the objective function is introduced into the dictionary learning process to improve the robustness for noisy HSI data. The experimental results, conducted with both synthetic and real-world hyperspectral datasets, illustrate that the proposed EEORDL outperforms the state-of-the-art approaches under different signal-to-noise ratio (SNR) conditions, especially for high-level noise.

Nonconvex-sparsity and Nonlocal-smoothness Based Blind Hyperspectral Unmixing

Blind hyperspectral unmixing (HU), as a crucial technique for hyperspectral data exploitation, aims to decompose mixed pixels into a collection of constituent materials weighted by the corresponding fractional abundances. In recent years, nonnegative matrix factorization (NMF) based methods have become more and more popular for this task and achieved promising performance. Among these methods, two types of properties upon the abundances, namely the sparseness and the structural smoothness, have been explored and shown to be important for blind HU. However, all of previous methods ignores another important insightful property possessed by a natural hyperspectral images (HSI), non-local smoothness, which means that similar patches in a larger region of an HSI are sharing the similar smoothness structure. Based on previous attempts on other tasks, such a prior structure reflects intrinsic configurations underlying a HSI, and is thus expected to largely improve the performance of the investigated HU problem. In this paper, we firstly consider such prior in HSI by encoding it as the nonlocal total variation (NLTV) regularizer. Furthermore, by fully exploring the intrinsic structure of HSI, we generalize NLTV to non-local HSI TV (NLHTV) to make the model more suitable for the bind HU task. By incorporating these two regularizers, together with a non-convex log-sum form regularizer characterizing the sparseness of abundance maps, to the NMF model, we propose novel blind HU models named NLTV/NLHTV and log-sum regularized NMF (NLTV-LSRNMF/NLHTV-LSRNMF), respectively. To solve the proposed models, an efficient algorithm is designed based on alternative optimization strategy (AOS) and alternating direction method of multipliers (ADMM). Extensive experiments conducted on both simulated and real hyperspectral data sets substantiate the superiority of the proposed approach over other competing ones for blind HU task.

Parameterized Nonlinear Least Squares for Unsupervised Nonlinear Spectral Unmixing

This paper presents a new parameterized nonlinear least squares (PNLS) algorithm for unsupervised nonlinear spectral unmixing (UNSU). The PNLS-based algorithms transform the original optimization problem with respect to the endmembers, abundances, and nonlinearity coefficients estimation into separate alternate parameterized nonlinear least squares problems. Owing to the Sigmoid parameterization, the PNLS-based algorithms are able to thoroughly relax the additional nonnegative constraint and the nonnegative constraint in the original optimization problems, which facilitates finding a solution to the optimization problems . Subsequently, we propose to solve the PNLS problems based on the Gauss–Newton method. Compared to the existing nonnegative matrix factorization (NMF)-based algorithms for UNSU, the well-designed PNLS-based algorithms have faster convergence speed and better unmixing accuracy. To verify the performance of the proposed algorithms, the PNLS-based algorithms and other state-of-the-art algorithms are applied to synthetic data generated by the Fan model and the generalized bilinear model (GBM), as well as real hyperspectral data. The results demonstrate the superiority of the PNLS-based algorithms.

An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing

Hyperspectral imagery collected from airborne or satellite sources inevitably suffers from spectral variability, making it difficult for spectral unmixing to accurately estimate abundance maps. The classical unmixing model, the linear mixing model (LMM), generally fails to handle this sticky issue effectively. To this end, we propose a novel spectral mixture model, called the augmented linear mixing model (ALMM), to address spectral variability by applying a data-driven learning strategy in inverse problems of hyperspectral unmixing. The proposed approach models the main spectral variability (i.e., scaling factors) generated by variations in illumination or typography separately by means of the endmember dictionary. It then models other spectral variabilities caused by environmental conditions (e.g., local temperature and humidity, atmospheric effects) and instrumental configurations (e.g., sensor noise), as well as material nonlinear mixing effects, by introducing a spectral variability dictionary. To effectively run the data-driven learning strategy, we also propose a reasonable prior knowledge for the spectral variability dictionary, whose atoms are assumed to be low-coherent with spectral signatures of endmembers, which leads to a well-known low-coherence dictionary learning problem. Thus, a dictionary learning technique is embedded in the framework of spectral unmixing so that the algorithm can learn the spectral variability dictionary and estimate the abundance maps simultaneously. Extensive experiments on synthetic and real datasets are performed to demonstrate the superiority and effectiveness of the proposed method in comparison with previous state-of-the-art methods.