Unsupervised learning of categorical data with competing models

A Fast and Efficient Method for Training Categorical Radial Basis Function Networks

This brief presents a novel learning scheme for categorical data based on radial basis function (RBF) networks. The proposed approach replaces the numerical vectors known as RBF centers with categorical tuple centers, and employs specially designed measures for calculating the distance between the center and the input tuples. Furthermore, a fast noniterative categorical clustering algorithm is proposed to accomplish the first stage of RBF training involving categorical center selection, whereas the weights are calculated through linear regression. The method is applied on 22 categorical data sets and compared with several different learning schemes, including neural networks, support vector machines, naïve Bayes classifier, and decision trees. Results show that the proposed method is very competitive, outperforming its rivals in terms of predictive capabilities in the majority of the tested cases.

Twin support vector machine for clustering

The twin support vector machine (TWSVM) is one of the powerful classification methods. In this brief, a TWSVM-type clustering method, called twin support vector clustering (TWSVC), is proposed. Our TWSVC includes both linear and nonlinear versions. It determines k cluster center planes by solving a series of quadratic programming problems. To make TWSVC more efficient and stable, an initialization algorithm based on the nearest neighbor graph is also suggested. The experimental results on several benchmark data sets have shown a comparable performance of our TWSVC.

A quadratically constrained MAP classifier using the mixture of Gaussians models as a weight function

In this paper, we propose classifiers derived from quadratically constrained maximum a posteriori (QCMAP) estimation. The QCMAP consists of the maximization of the expectation of a cost function, which is derived from the maximum a posteriori probability and a quadratic constraint. This criterion is highly general since its forms include least squares regressions and a support vector machine. Furthermore, the criterion provides a novel classifier, the "Gaussian QCMAP." The QCMAP procedure still has large theoretical interest and its full extensibility has yet to be explored. In this paper, we propose using the mixture of Gaussian distributions as the QCMAP weight function. The mixture of Gaussian distributions has wide-ranging applicability, and encompasses forms, such as a normal distribution model and a kernel density model. We propose four types of mixture of Gaussian functions for QCMAP classifiers, and conduct experiments to demonstrate their advantages.

Sparse coding from a Bayesian perspective

Sparse coding is a promising theme in computer vision. Most of the existing sparse coding methods are based on either l0 or l1 penalty, which often leads to unstable solution or biased estimation. This is because of the nonconvexity and discontinuity of the l0 penalty and the over-penalization on the true large coefficients of the l1 penalty. In this paper, sparse coding is interpreted from a novel Bayesian perspective, which results in a new objective function through maximum a posteriori estimation. The obtained solution of the objective function can generate more stable results than the l0 penalty and smaller reconstruction errors than the l1 penalty. In addition, the convergence property of the proposed algorithm for sparse coding is also established. The experiments on applications in single image super-resolution and visual tracking demonstrate that the proposed method is more effective than other state-of-the-art methods.