Exponential family factors for Bayesian factor analysis

Generalized infinite factorization models

Factorization models express a statistical object of interest in terms of a collection of simpler objects. For example, a matrix or tensor can be expressed as a sum of rank-one components. However, in practice, it can be challenging to infer the relative impact of the different components as well as the number of components. A popular idea is to include infinitely many components having impact decreasing with the component index. This article is motivated by two limitations of existing methods: (i) the lack of careful consideration of the within component sparsity structure; and (ii) no accommodation for grouped variables and other non-exchangeable structures. We propose a general class of infinite factorization models that address these limitations. Theoretical support is provided, practical gains are shown in simulation studies, and an ecology application focusing on modelling bird species occurrence is discussed.

Discriminant projective non-negative matrix factorization

Projective non-negative matrix factorization (PNMF) projects high-dimensional non-negative examples X onto a lower-dimensional subspace spanned by a non-negative basis W and considers W^T X as their coefficients, i.e., X≈WW^T X. Since PNMF learns the natural parts-based representation Wof X, it has been widely used in many fields such as pattern recognition and computer vision. However, PNMF does not perform well in classification tasks because it completely ignores the label information of the dataset. This paper proposes a Discriminant PNMF method (DPNMF) to overcome this deficiency. In particular, DPNMF exploits Fisher's criterion to PNMF for utilizing the label information. Similar to PNMF, DPNMF learns a single non-negative basis matrix and needs less computational burden than NMF. In contrast to PNMF, DPNMF maximizes the distance between centers of any two classes of examples meanwhile minimizes the distance between any two examples of the same class in the lower-dimensional subspace and thus has more discriminant power. We develop a multiplicative update rule to solve DPNMF and prove its convergence. Experimental results on four popular face image datasets confirm its effectiveness comparing with the representative NMF and PNMF algorithms.

A Bayesian hierarchical factorization model for vector fields

Factorization-based techniques explain arrays of observations using a relatively small number of factors and provide an essential arsenal for multi-dimensional data analysis. Most factorization models are, however, developed on general arrays of scalar values. For a class of practical data arising from observing spatial signals including images, it is desirable for a model to consider general observations, e.g., handling a vector field and non-exchangeable factors, e.g., handling spatial connections between the columns and the rows of the data. In this paper, a probabilistic model for factorization is proposed. We adopt Bayesian hierarchical modeling and treat the factors as latent random variables. A Markov structure is imposed on the distribution of factors to account for the spatial connections. The model is designed to represent vector arrays sampled from fields of continuous domains. Therefore, a tailored observation model is developed to represent the link between the factor product and the data. The proposed technique has been shown effective in analyzing optical flow fields computed on both synthetic images and real-life videoclips.