Structural Minimax Probability Machine

Distribution-free Bayesian regularized learning framework for semi-supervised learning

In machine learning it is often necessary to assume or know the distribution of the data, however it is difficult to do so in practical applications. Aiming to this problem, this work, we propose a novel distribution-free Bayesian regularized learning framework for semi-supervised learning, which is called Hessian regularized twin minimax probability extreme learning machine (HRTMPELM). In this framework, we attempt to construct two non-parallel hyperplanes by introducing the high separation probability assumption, such that each hyperplane separates samples from one class with maximum probability while moving away from samples from the other class. Subsidiently, the framework can be utilized to construct reasonable semi-supervised classifiers by using the information of the inherent geometric distribution of the samples through the Hessian regularization term. Additionally, the proposed framework controls the misclassification error of samples by minimizing the upper limit of the worst-case misclassification probability, and improves the generalization performance of the model by introducing the idea of regularization to avoid the occurrence of ill-posedness and overfitting problems. More importantly, the framework has no hyperparameters, making the learning process very simplified and efficient. Finally, a simple and reliable algorithm with globally optimal solutions via multivariate Chebyshev inequalities is designed for solving the proposed learning framework. Experiments on multiple datasets demonstrate the reliability and effectiveness of the proposed learning framework compared to other methods. Especially, we applied the framework to Ningxia wolfberry quality detection, which greatly enriches and facilitates the application of machine learning algorithms in the agricultural field.

On stratified single-valued soft topogenous structures

This paper presents novel concepts including stratified single-valued neutrosophic soft topogenous (stratified svns-topogenous), stratified single-valued neutrosophic soft filter (stratified svns-filter), stratified single-valued neutrosophic soft quasi uniformity (stratified svnsq-uniformity) and stratified single-valued neutrosophic soft quasi proximity (stratified svnsq-proximity). Additionally, we present the idea of single-valued neutrosophic soft topogenous structures, formed by integrating svns-topogenous with svns-filter, and discuss their properties. Furthermore, we explore the connections between these single-valued neutrosophic soft topological structures and their corresponding stratifications.

Kernel-Free Quadratic Surface Minimax Probability Machine for a Binary Classification Problem

In this paper, we propose a novel binary classification method called the kernel-free quadratic surface minimax probability machine (QSMPM), that makes use of the kernel-free techniques of the quadratic surface support vector machine (QSSVM) and inherits the advantage of the minimax probability machine (MPM) without any parameters. Specifically, it attempts to find a quadratic hypersurface that separates two classes of samples with maximum probability. However, the optimization problem derived directly was too difficult to solve. Therefore, a nonlinear transformation was introduced to change the quadratic function involved into a linear function. Through such processing, our optimization problem finally became a second-order cone programming problem, which was solved efficiently by an alternate iteration method. It should be pointed out that our method is both kernel-free and parameter-free, making it easy to use. In addition, the quadratic hypersurface obtained by our method was allowed to be any general form of quadratic hypersurface. It has better interpretability than the methods with the kernel function. Finally, in order to demonstrate the geometric interpretation of our QSMPM, five artificial datasets were implemented, including showing the ability to obtain a linear separating hyperplane. Furthermore, numerical experiments on benchmark datasets confirmed that the proposed method had better accuracy and less CPU time than corresponding methods.

Accelerating Sequential Minimal Optimization via Stochastic Subgradient Descent

Sequential minimal optimization (SMO) is one of the most popular methods for solving a variety of support vector machines (SVMs). The shrinking and caching techniques are commonly used to accelerate SMO. An interesting phenomenon of SMO is that most of the computational time is wasted on the first half of iterations for building a good solution closing to the optimal. However, as we all know, the stochastic subgradient descent (SSGD) method is extremely fast for building a good solution. In this paper, we propose a generalized framework of accelerating SMO through SSGD for a variety of SVMs of binary classification, regression, ordinal regression, and so on. We also provide a deep insight about why SSGD can accelerate SMO. Experimental results on a variety of datasets and learning applications confirm that our method can effectively speed up SMO.

Twin minimax probability machine for pattern classification

We propose a new distribution-free Bayes optimal classifier, called the twin minimax probability machine (TWMPM), which combines the benefits of both minimax probability machine(MPM) and twin support vector machine (TWSVM). TWMPM tries to construct two nonparallel hyperplanes such that each hyperplane separates one class samples with maximal probability, and is distant from the other class samples simultaneously. Moreover, the proposed TWMPM can control the misclassification error of samples in a worst-case setting by minimizing the upper bound on misclassification probability. An efficient algorithm for TWMPM is first proposed, which transforms TWMPM into concave fractional programming by applying multivariate Chebyshev inequality. Then the proposed TWMPM is reformulated as a pair of convex quadric programming (QP) by proper mathematical transformations. This guarantees TWMPM to have global optimal solution and be solved simply and effectively. In addition, we develop also an iterative algorithm for the proposed TWMPM. By comparing the two proposed algorithms theoretically, it is easy to know that the convex quadric programming algorithm is with lower computation burden than iterative algorithm for the TWMPM. A linear TWMPM version is first built, and then we show how to exploit mercer kernel to obtain nonlinear TWMPM version. The computation complexity for QP algorithm of TWMPM is in the same order as the traditional twin support vector machine (TWSVM). Experiments are carried out on three databases: UCI benchmark database, a practical application database and an artificial database. With low computation complexity and fewer parameters, experiments show the feasibility and effectiveness of the proposed TWMPM and its QP algorithm.

Multi-Label Active Learning Algorithms for Image Classification: Overview and Future Promise

Image classification is a key task in image understanding, and multi-label image classification has become a popular topic in recent years. However, the success of multi-label image classification is closely related to the way of constructing a training set. As active learning aims to construct an effective training set through iteratively selecting the most informative examples to query labels from annotators, it was introduced into multi-label image classification. Accordingly, multi-label active learning is becoming an important research direction. In this work, we first review existing multi-label active learning algorithms for image classification. These algorithms can be categorized into two top groups from two aspects respectively: sampling and annotation. The most important component of multi-label active learning is to design an effective sampling strategy that actively selects the examples with the highest informativeness from an unlabeled data pool, according to various information measures. Thus, different informativeness measures are emphasized in this survey. Furthermore, this work also makes a deep investigation on existing challenging issues and future promises in multi-label active learning with a focus on four core aspects: example dimension, label dimension, annotation, and application extension.

A Training Data Set Cleaning Method by Classification Ability Ranking for the k -Nearest Neighbor Classifier

The k -nearest neighbor (KNN) rule is a successful technique in pattern classification due to its simplicity and effectiveness. As a supervised classifier, KNN classification performance usually suffers from low-quality samples in the training data set. Thus, training data set cleaning (TDC) methods are needed for enhancing the classification accuracy by cleaning out noisy, or even wrong, samples in the original training data set. In this paper, we propose a classification ability ranking (CAR)-based TDC method to improve the performance of a KNN classifier, namely CAR-based TDC method. The proposed classification ability function ranks a training sample in terms of its contribution to correctly classify other training samples as a KNN through the leave-one-out (LV1) strategy in the cleaning stage. The training sample that likely misclassifies the other samples during the KNN classifications according to the LV1 strategy is considered to have lower classification ability and will be cleaned out from the original training data set. Extensive experiments, based on ten real-world data sets, show that the proposed CAR-based TDC method can significantly reduce the classification error rates of KNN-based classifiers, while reducing computational complexity thanks to a smaller cleaned training data set.

Investigation of High-Efficiency Iterative ILU Preconditioner Algorithm for Partial-Differential Equation Systems

In this paper, we investigate an iterative incomplete lower and upper (ILU) factorization preconditioner for partial-differential equation systems. We discretize the partial-differential equations into linear equation systems. An iterative scheme of linear systems is used. The ILU preconditioners of linear systems are performed on the different computation nodes of multi-central processing unit (CPU) cores. Firstly, the preconditioner of general tridiagonal matrix equations is tested on supercomputers. Then, the effects of partial-differential equation systems on the speedup of parallel multiprocessors are examined. The numerical results estimate that the parallel efficiency is higher than in other algorithms.

Optimization of Distributions Differences for Classification

In this paper, we introduce a new classification algorithm called the optimization of distribution differences (ODD). The algorithm aims to find a transformation from the feature space to a new space where the instances in the same class are as close as possible to one another, whereas the gravity centers of these classes are as far as possible from one another. This aim is formulated as a multiobjective optimization problem that is solved by a hybrid of an evolutionary strategy and the quasi-Newton method. The choice of the transformation function is flexible and could be any continuous space function. We experiment with a linear and a nonlinear transformation in this paper. We show that the algorithm can outperform eight other classification methods, namely naive Bayes, support vector machines, linear discriminant analysis, multilayer perceptrons, decision trees, and k -nearest neighbors, and two recently proposed classification methods, in 12 standard classification data sets. Our results show that the method is less sensitive to the imbalanced number of instances compared with these methods. We also show that ODD maintains its performance better than other classification methods in these data sets and hence offers a better generalization ability.

A Grouping Particle Swarm Optimizer with Personal-Best-Position Guidance for Large Scale Optimization

Particle Swarm Optimization (PSO) is a popular algorithm which is widely investigated and well implemented in many areas. However, the canonical PSO does not perform well in population diversity maintenance so that usually leads to a premature convergence or local optima. To address this issue, we propose a variant of PSO named Grouping PSO with Personal-Best-Position ( P_best) Guidance (GPSO-PG) which maintains the population diversity by preserving the diversity of exemplars. On one hand, we adopt uniform random allocation strategy to assign particles into different groups and in each group the losers will learn from the winner. On the other hand, we employ personal historical best position of each particle in social learning rather than the current global best particle. In this way, the exemplars diversity increases and the effect from the global best particle is eliminated. We test the proposed algorithm to the benchmarks in CEC 2008 and CEC 2010, which concern the large scale optimization problems (LSOPs). By comparing several current peer algorithms, GPSO-PG exhibits a competitive performance to maintain population diversity and obtains a satisfactory performance to the problems.

Multiview Discrete Hashing for Scalable Multimedia Search

Hashing techniques have recently gained increasing research interest in multimedia studies. Most existing hashing methods only employ single features for hash code learning. Multiview data with each view corresponding to a type of feature generally provides more comprehensive information. How to efficiently integrate multiple views for learning compact hash codes still remains challenging. In this article, we propose a novel unsupervised hashing method, dubbed multiview discrete hashing (MvDH), by effectively exploring multiview data. Specifically, MvDH performs matrix factorization to generate the hash codes as the latent representations shared by multiple views, during which spectral clustering is performed simultaneously. The joint learning of hash codes and cluster labels enables that MvDH can generate more discriminative hash codes, which are optimal for classification. An efficient alternating algorithm is developed to solve the proposed optimization problem with guaranteed convergence and low computational complexity. The binary codes are optimized via the discrete cyclic coordinate descent (DCC) method to reduce the quantization errors. Extensive experimental results on three large-scale benchmark datasets demonstrate the superiority of the proposed method over several state-of-the-art methods in terms of both accuracy and scalability.

Multilabel Prediction via Cross-View Search

Embedding methods have shown promising performance in multilabel prediction, as they are able to discover the label dependence. However, most methods ignore the correlations between the input and output, such that their learned embeddings are not well aligned, which leads to degradation in prediction performance. This paper presents a formulation for multilabel learning, from the perspective of cross-view learning, that explores the correlations between the input and the output. The proposed method, called Co-Embedding (CoE), jointly learns a semantic common subspace and view-specific mappings within one framework. The semantic similarity structure among the embeddings is further preserved, ensuring that close embeddings share similar labels. Additionally, CoE conducts multilabel prediction through the cross-view $k$ nearest neighborhood ( $k$ NN) search among the learned embeddings, which significantly reduces computational costs compared with conventional decoding schemes. A hashing-based model, i.e., Co-Hashing (CoH), is further proposed. CoH is based on CoE, and imposes the binary constraint on continuous latent embeddings. CoH aims to generate compact binary representations to improve the prediction efficiency by benefiting from the efficient $k$ NN search of multiple labels in the Hamming space. Extensive experiments on various real-world data sets demonstrate the superiority of the proposed methods over the state of the arts in terms of both prediction accuracy and efficiency.

Word clustering based on POS feature for efficient twitter sentiment analysis

With rapid growth of social networking service on Internet, huge amount of information are continuously generated in real time. As a result, sentiment analysis of online reviews and messages has become a popular research issue [1]. In this paper a novel modified Chi Square-based feature clustering and weighting scheme is proposed for the sentiment analysis of twitter message. Along with the part of speech tagging, the discriminability and dependency of the words in the tagged training dataset are taken into account in the clustering and weighting process. The multinomial Naïve Bayes model is also employed to handle redundant features, and the influence of emotional words is raised for maximizing the accuracy. Computer simulation with Sentiment 140 workload shows that the proposed scheme significantly outperforms four existing representative sentiment analysis schemes in terms of the accuracy regardless of the size of training and test data.

Non-Negative Spectral Learning and Sparse Regression-Based Dual-Graph Regularized Feature Selection

Feature selection is an important approach for reducing the dimension of high-dimensional data. In recent years, many feature selection algorithms have been proposed, but most of them only exploit information from the data space. They often neglect useful information contained in the feature space, and do not make full use of the characteristics of the data. To overcome this problem, this paper proposes a new unsupervised feature selection algorithm, called non-negative spectral learning and sparse regression-based dual-graph regularized feature selection (NSSRD). NSSRD is based on the feature selection framework of joint embedding learning and sparse regression, but extends this framework by introducing the feature graph. By using low dimensional embedding learning in both data space and feature space, NSSRD simultaneously exploits the geometric information of both spaces. Second, the algorithm uses non-negative constraints to constrain the low-dimensional embedding matrix of both feature space and data space, ensuring that the elements in the matrix are non-negative. Third, NSSRD unifies the embedding matrix of the feature space and the sparse transformation matrix. To ensure the sparsity of the feature array, the sparse transformation matrix is constrained using the -norm. Thus feature selection can obtain accurate discriminative information from these matrices. Finally, NSSRD uses an iterative and alternative updating rule to optimize the objective function, enabling it to select the representative features more quickly and efficiently. This paper explains the objective function, the iterative updating rules and a proof of convergence. Experimental results show that NSSRD is significantly more effective than several other feature selection algorithms from the literature, on a variety of test data.

Dynamic Gesture Recognition with a Terahertz Radar Based on Range Profile Sequences and Doppler Signatures

The frequency of terahertz radar ranges from 0.1 THz to 10 THz, which is higher than that of microwaves. Multi-modal signals, including high-resolution range profile (HRRP) and Doppler signatures, can be acquired by the terahertz radar system. These two kinds of information are commonly used in automatic target recognition; however, dynamic gesture recognition is rarely discussed in the terahertz regime. In this paper, a dynamic gesture recognition system using a terahertz radar is proposed, based on multi-modal signals. The HRRP sequences and Doppler signatures were first achieved from the radar echoes. Considering the electromagnetic scattering characteristics, a feature extraction model is designed using location parameter estimation of scattering centers. Dynamic Time Warping (DTW) extended to multi-modal signals is used to accomplish the classifications. Ten types of gesture signals, collected from a terahertz radar, are applied to validate the analysis and the recognition system. The results of the experiment indicate that the recognition rate reaches more than 91%. This research verifies the potential applications of dynamic gesture recognition using a terahertz radar.