A constrained EM algorithm for independent component analysis

Markov and Semi-Markov switching of source appearances for nonstationary independent component analysis

Independent component analysis (ICA) is currently the most popularly used approach to blind source separation (BSS), the problem of recovering unknown source signals when their mixtures are observed but the actual mixing process is unknown. Many ICA algorithms assume that a fixed set of source signals consistently exists in mixtures throughout the time-series to be examined. However, real-world signals often have such difficult nonstationarity that each source signal abruptly appears or disappears, thus the set of active sources dynamically changes with time. In this paper, we propose switching ICA (SwICA), which focuses on such situations. The proposed approach is based on the noisy ICA formulated as a generative model. We employ a special type of hidden Markov model (HMM) to represent such prior knowledge that the source may abruptly appear or disappear with time. The special HMM setting t hen provides an effect ofvariable selection in a dynamic way. We use the variational Bayes (VB) method to derive an effective approximation of Bayesian inference for this model. In simulation experiments using artificial and realistic source signals, the proposed method exhibited performance superior to existing methods, especially in the presence of noise. The compared methods include the natural-gradient ICA with a nonholonomic constraint, and the existing ICA method incorporating an HMM source model, which aims to deal with general nonstationarities that may exist in source signals. In addition, the proposed method could successfully recover the source signals even when the total number of true sources was overestimated or was larger than that of mixtures. We also propose a modification of the basic Markov model into a semi-Markov model, and show that the semi-Markov one is more effective for robust estimation of the source appearance.

An independent component analysis approach for minimizing effects of recirculation in dynamic susceptibility contrast magnetic resonance imaging

In dynamic susceptibility contrast (DSC) perfusion-weighted imaging, effects of recirculation are normally minimized by a gamma-variate fitting procedure of the concentration curves before estimating hemodynamic parameters. The success of this method, however, hinges largely on the extent to which magnetic resonance signal is altered in the presence of a contrast agent and a temporal separation between the first and subsequent passages of the contrast agent. Moreover, important physiologic information might be compromised by imposing an analytic equation to all measured concentration curves. This investigation proposes to exploit independent component analysis to minimize effects of recirculation in DSC. Results obtained from simulation, normal healthy volunteers, and acute stroke patients show that such a technique can greatly minimize the effects of recirculation despite a substantial overlap between the first passage and recirculation. This in turn should improve estimation of cerebral hemodynamics particularly when an overlap between the first passage and recirculation is suspected as in an ischemic lesion.

One-bit-matching theorem for ICA, convex-concave programming on polyhedral set, and distribution approximation for combinatorics

According to the proof by Liu, Chiu, and Xu (2004) on the so-called one-bit-matching conjecture (Xu, Cheung, and Amari, 1998a), all the sources can be separated as long as there is an one-to-one same-sign correspondence between the kurtosis signs of all source probability density functions (pdf's) and the kurtosis signs of all model pdf's, which is widely believed and implicitly supported by many empirical studies. However, this proof is made only in a weak sense that the conjecture is true when the global optimal solution of an independent component analysis criterion is reached. Thus, it cannot support the successes of many existing iterative algorithms that usually converge at one of the local optimal solutions. This article presents a new mathematical proof that is obtained in a strong sense that the conjecture is also true when any one of local optimal solutions is reached in helping to investigating convex-concave programming on a polyhedral set. Theorems are also provided not only on partial separation of sources when there is a partial matching between the kurtosis signs, but also on an interesting duality of maximization and minimization on source separation. Moreover, corollaries are obtained on an interesting duality, with supergaussian sources separated by maximization and subgaussian sources separated by minimization. Also, a corollary is obtained to confirm the symmetric orthogonalization implementation of the kurtosis extreme approach for separating multiple sources in parallel, which works empirically but lacks mathematical proof. Furthermore, a linkage has been set up to combinatorial optimization from a distribution approximation perspective and a Stiefel manifold perspective, with algorithms that guarantee convergence as well as satisfaction of constraints.

Source density-driven independent component analysis approach for fMRI data

Independent component analysis (ICA) has become a popular tool for functional magnetic resonance imaging (fMRI) data analysis. Conventional ICA algorithms including Infomax and FAST-ICA algorithms employ the underlying assumption that data can be decomposed into statistically independent sources and implicitly model the probability density functions of the underlying sources as highly kurtotic or symmetric. When source data violate these assumptions (e.g., are asymmetric), however, conventional ICA methods might not work well. As a result, modeling of the underlying sources becomes an important issue for ICA applications. We propose a source density-driven ICA (SD-ICA) method. The SD-ICA algorithm involves a two-step procedure. It uses a conventional ICA algorithm to obtain initial independent source estimates for the first-step and then, using a kernel estimator technique, the source density is calculated. A refitted nonlinear function is used for each source at the second step. We show that the proposed SD-ICA algorithm provides flexible source adaptivity and improves ICA performance. On SD-ICA application to fMRI signals, the physiologic meaningful components (e.g., activated regions) of fMRI signals are governed typically by a small percentage of the whole-brain map on a task-related activation. Extra prior information (using a skewed-weighted distribution transformation) is thus additionally applied to the algorithm for the regions of interest of data (e.g., visual activated regions) to emphasize the importance of the tail part of the distribution. Our experimental results show that the source density-driven ICA method can improve performance further by incorporating some a priori information into ICA analysis of fMRI signals.

On Convergence Conditions of an Extended Projection Neural Network

The output trajectory convergence of an extended projection neural network was developed under the positive definiteness condition of the Jacobian matrix of nonlinear mapping. This note offers several new convergence results. The state trajectory convergence and the output trajectory convergence of the extended projection neural network are obtained under the positive semidefiniteness condition of the Jacobian matrix. Comparison and illustrative examples demonstrate applied significance of these new results.

A Further Result on the ICA One-Bit-Matching Conjecture

The one-bit-matching conjecture for independent component analysis (ICA) has been widely believed in the ICA community. Theoretically, it has been proved that under the assumption of zero skewness for the model probability density functions, the global maximum of a cost function derived from the typical objective function on the ICA problem with the one-bit-matching condition corresponds to a feasible solution of the ICA problem. In this note, we further prove that all the local maximums of the cost function correspond to the feasible solutions of the ICA problem in the two-source case under the same assumption. That is, as long as the one-bit-matching condition is satisfied, the two-source ICA problem can be successfully solved using any local descent algorithm of the typical objective function with the assumption of zero skewness for all the model probability density functions.

One-bit-matching conjecture for independent component analysis

The one-bit-matching conjecture for independent component analysis (ICA) could be understood from different perspectives but is basically stated as "all the sources can be separated as long as there is a one-to-one same-sign-correspondence between the kurtosis signs of all source probability density functions (pdf's) and the kurtosis signs of all model pdf's" (Xu, Cheung, & Amari, 1998a). This conjecture has been widely believed in the ICA community and implicitly supported by many ICA studies, such as the Extended Infomax (Lee, Girolami, & Sejnowski, 1999) and the soft switching algorithm (Welling & Weber, 2001). However, there is no mathematical proof to confirm the conjecture theoretically. In this article, only skewness and kurtosis are considered, and such a mathematical proof is given under the assumption that the skewness of the model densities vanishes. Moreover, empirical experiments are demonstrated on the robustness of the conjecture as the vanishing skewness assumption breaks. As a by-product, we also show that the kurtosis maximization criterion (Moreau & Macchi, 1996) is actually a special case of the minimum mutual information criterion for ICA.

Overview of independent component analysis technique with an application to synthetic aperture radar (SAR) imagery processing

We present an overview of independent component analysis, an emerging signal processing technique based on neural networks, with the aim to provide an up-to-date survey of the theoretical streams in this discipline and of the current applications in the engineering area. We also focus on a particular application, dealing with a remote sensing technique based on synthetic aperture radar imagery processing: we briefly review the features and main applications of synthetic aperture radar and show how blind signal processing by neural networks may be advantageously employed to enhance the quality of remote sensing data.