Projection-pursuit-based method for blind separation of nonnegative sources

Sequential Independent Component Analysis Density Estimation

A problem of multivariate probability density function estimation by exploiting linear independent components analysis (ICA) is addressed. Historically, ICA density estimation was initially proposed under the name projection pursuit density estimation (PPDE) and two basic methods, named forward and backward, were published. We derive a modification of the forward PPDE method, which avoids a computationally demanding optimization involving Monte Carlo sampling of the original method. The results of the experiments show that the proposed method presents an attractive choice for density estimation, which is pronounced for a small number of training observations. Under such conditions, our method usually outperforms model-based Gaussian mixture model. We also found that our method obtained better results than the backward PPDE methods in the situation of nonfactorizable underlying density functions. The proposed method has demonstrated a competitive performance compared with the support vector machine and the extreme learning machine in some real classification tasks.

Local Smoothness Constrained Nonnegative Matrix Factorization with Nonlinear Convergence Rate for Spectral Decomposition

With the development of the detection technology using multispectra sensors, spectral decomposition (SD) attracts more and more attention in the biomedical signal processing and image processing. In this paper, a local smoothness constrained nonnegative matrix factorization (NMF) with nonlinear convergence rate (NMF-NCR) is proposed to solve SD problem and our contributions are as follows. First, it proves that the gradients of the cost function with respect to each variable matrix are Lipschitz continuous. Then, a proximal function is constructed for optimizing the cost function. As a result, our method can achieve an NCR much faster than the traditional methods. Simulations show the advantage in solving SD of our algorithm over the compared methods.

Adaptive Method for Nonsmooth Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is an emerging tool for meaningful low-rank matrix representation. In NMF, explicit constraints are usually required, such that NMF generates desired products (or factorizations), especially when the products have significant sparseness features. It is known that the ability of NMF in learning sparse representation can be improved by embedding a smoothness factor between the products. Motivated by this result, we propose an adaptive nonsmooth NMF (Ans-NMF) method in this paper. In our method, the embedded factor is obtained by using a data-related approach, so it matches well with the underlying products, implying a superior faithfulness of the representations. Besides, due to the usage of an adaptive selection scheme to this factor, the sparseness of the products can be separately constrained, leading to wider applicability and interpretability. Furthermore, since the adaptive selection scheme is processed through solving a series of typical linear programming problems, it can be easily implemented. Simulations using computer-generated data and real-world data show the advantages of the proposed Ans-NMF method over the state-of-the-art methods.

Sparse Bayesian Classification of EEG for Brain-Computer Interface

Regularization has been one of the most popular approaches to prevent overfitting in electroencephalogram (EEG) classification of brain-computer interfaces (BCIs). The effectiveness of regularization is often highly dependent on the selection of regularization parameters that are typically determined by cross-validation (CV). However, the CV imposes two main limitations on BCIs: 1) a large amount of training data is required from the user and 2) it takes a relatively long time to calibrate the classifier. These limitations substantially deteriorate the system's practicability and may cause a user to be reluctant to use BCIs. In this paper, we introduce a sparse Bayesian method by exploiting Laplace priors, namely, SBLaplace, for EEG classification. A sparse discriminant vector is learned with a Laplace prior in a hierarchical fashion under a Bayesian evidence framework. All required model parameters are automatically estimated from training data without the need of CV. Extensive comparisons are carried out between the SBLaplace algorithm and several other competing methods based on two EEG data sets. The experimental results demonstrate that the SBLaplace algorithm achieves better overall performance than the competing algorithms for EEG classification.

Group Component Analysis for Multiblock Data: Common and Individual Feature Extraction

Real-world data are often acquired as a collection of matrices rather than as a single matrix. Such multiblock data are naturally linked and typically share some common features while at the same time exhibiting their own individual features, reflecting the underlying data generation mechanisms. To exploit the linked nature of data, we propose a new framework for common and individual feature extraction (CIFE) which identifies and separates the common and individual features from the multiblock data. Two efficient algorithms termed common orthogonal basis extraction (COBE) are proposed to extract common basis is shared by all data, independent on whether the number of common components is known beforehand. Feature extraction is then performed on the common and individual subspaces separately, by incorporating dimensionality reduction and blind source separation techniques. Comprehensive experimental results on both the synthetic and real-world data demonstrate significant advantages of the proposed CIFE method in comparison with the state-of-the-art.

A Convex Geometry-Based Blind Source Separation Method for Separating Nonnegative Sources

This paper presents a convex geometry (CG)-based method for blind separation of nonnegative sources. First, the unaccessible source matrix is normalized to be column-sum-to-one by mapping the available observation matrix. Then, its zero-samples are found by searching the facets of the convex hull spanned by the mapped observations. Considering these zero-samples, a quadratic cost function with respect to each row of the unmixing matrix, together with a linear constraint in relation to the involved variables, is proposed. Upon which, an algorithm is presented to estimate the unmixing matrix by solving a classical convex optimization problem. Unlike the traditional blind source separation (BSS) methods, the CG-based method does not require the independence assumption, nor the uncorrelation assumption. Compared with the BSS methods that are specifically designed to distinguish between nonnegative sources, the proposed method requires a weaker sparsity condition. Provided simulation results illustrate the performance of our method.