Multiplicative update rules for concurrent nonnegative matrix factorization and maximum margin classification

Graph-Based Multicentroid Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is a widely recognized approach for data representation. When it comes to clustering, NMF fails to handle data points located in complex geometries, as each sample cluster is represented by a centroid. In this article, a novel multicentroid-based clustering method called graph-based multicentroid NMF (MCNMF) is proposed. Because the method constructs the neighborhood connection graph between data points and centroids, each data point is represented by adjacent centroids, which preserves the local geometric structure. Second, because the method constructs an undirected connected graph with centroids as nodes, in which the centroids are divided into different centroid clusters, a novel data clustering method based on MCNMF is proposed. In addition, the membership index matrix is reconstructed based on the obtained centroid clusters, which solves the problem of membership identification of the final sample. Extensive experiments conducted on synthetic datasets and real benchmark datasets illustrate the effectiveness of the proposed MCNMF method. Compared with single-centroid-based methods, the MCNMF can obtain the best experimental results.

Construction of a non-negative matrix factorization model of immunogenic cell death-related genes in lung adenocarcinoma and analysis of survival prognosis

Purpose

To explore the effectiveness of the model based on non-negative matrix factorization (NMF), analyze the tumor microenvironment and immune microenvironment for evaluating the prognosis of lung adenocarcinoma, establish a risk model, and screen independent prognostic factors.

Methods

Downloading the transcription data files and clinical information files of lung adenocarcinoma from TCGA database and GO database, the R software was used to establish the NMF cluster model, and then the survival analysis between groups, tumor microenvironment analysis, and immune microenvironment analysis was performed according to the NMF cluster result. R software was used to construct prognostic models and calculate risk scores. Survival analysis was used to compare survival differences between different risk score groups.

Results

Two ICD subgroups were established according to the NMF model. The survival of the ICD low-expression subgroup was better than that of the ICD high-expression subgroup. Univariate COX analysis screened out HSP90AA1, IL1, and NT5E as prognostic genes, and the prognostic model established on this basis has clinical guiding significance.

Conclusion

The model based on NMF has the prognostic ability for lung adenocarcinoma, and the prognostic model of ICD-related genes has a certain guiding significance for survival.

An Efficient Feature Selection Algorithm for Gene Families Using NMF and ReliefF

Gene families, which are parts of a genome's information storage hierarchy, play a significant role in the development and diversity of multicellular organisms. Several studies have focused on the characteristics of gene families, such as function, homology, or phenotype. However, statistical and correlation analyses on the distribution of gene family members in the genome have yet to be conducted. Here, a novel framework incorporating gene family analysis and genome selection based on NMF-ReliefF is reported. Specifically, the proposed method starts by obtaining gene families from the TreeFam database and determining the number of gene families within the feature matrix. Then, NMF-ReliefF is used to select features from the gene feature matrix, which is a new feature selection algorithm that overcomes the inefficiencies of traditional methods. Finally, a support vector machine is utilized to classify the acquired features. The results show that the framework achieved an accuracy of 89.1% and an AUC of 0.919 on the insect genome test set. We also employed four microarray gene data sets to evaluate the performance of the NMF-ReliefF algorithm. The outcomes show that the proposed method may strike a delicate balance between robustness and discrimination. Additionally, the proposed method's categorization is superior to state-of-the-art feature selection approaches.

Simultaneous Dimensionality Reduction and Classification via Dual Embedding Regularized Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is a well-known paradigm for data representation. Traditional NMF-based classification methods first perform NMF or one of its variants on input data samples to obtain their low-dimensional representations, which are successively classified by means of a typical classifier [e.g., k -nearest neighbors (KNN) and support vector machine (SVM)]. Such a stepwise manner may overlook the dependency between the two processes, resulting in the compromise of the classification accuracy. In this paper, we elegantly unify the two processes by formulating a novel constrained optimization model, namely dual embedding regularized NMF (DENMF), which is semi-supervised. Our DENMF solution simultaneously finds the low-dimensional representations and assignment matrix via joint optimization for better classification. Specifically, input data samples are projected onto a couple of low-dimensional spaces (i.e., feature and label spaces), and locally linear embedding is employed to preserve the identical local geometric structure in different spaces. Moreover, we propose an alternating iteration algorithm to solve the resulting DENMF, whose convergence is theoretically proven. Experimental results over five benchmark datasets demonstrate that DENMF can achieve higher classification accuracy than state-of-the-art algorithms.

Concept Factorization With Adaptive Neighbors for Document Clustering

In this paper, a novel concept factorization (CF) method, called CF with adaptive neighbors (CFANs), is proposed. The idea of CFAN is to integrate an ANs regularization constraint into the CF decomposition. The goal of CFAN is to extract the representation space that maintains geometrical neighborhood structure of the data. Similar to the existing graph-regularized CF, CFAN builds a neighbor graph weights matrix. The key difference is that the CFAN performs dimensionality reduction and finds the neighbor graph weights matrix simultaneously. An efficient algorithm is also derived to solve the proposed problem. We apply the proposed method to the problem of document clustering on the 20 Newsgroups, Reuters-21578, and TDT2 document data sets. Our experiments demonstrate the effectiveness of the method.

Graph Regularized Non-Negative Low-Rank Matrix Factorization for Image Clustering

Non-negative matrix factorization (NMF) has been one of the most popular methods for feature learning in the field of machine learning and computer vision. Most existing works directly apply NMF on high-dimensional image datasets for computing the effective representation of the raw images. However, in fact, the common essential information of a given class of images is hidden in their low rank parts. For obtaining an effective low-rank data representation, we in this paper propose a non-negative low-rank matrix factorization (NLMF) method for image clustering. For the purpose of improving its robustness for the data in a manifold structure, we further propose a graph regularized NLMF by incorporating the manifold structure information into our proposed objective function. Finally, we develop an efficient alternating iterative algorithm to learn the low-dimensional representation of low-rank parts of images for clustering. Alternatively, we also incorporate robust principal component analysis into our proposed scheme. Experimental results on four image datasets reveal that our proposed methods outperform four representative methods.

A Collective Neurodynamic Optimization Approach to Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is an advanced method for nonnegative feature extraction, with widespread applications. However, the NMF solution often entails to solve a global optimization problem with a nonconvex objective function and nonnegativity constraints. This paper presents a collective neurodynamic optimization (CNO) approach to this challenging problem. The proposed collective neurodynamic system consists of a population of recurrent neural networks (RNNs) at the lower level and a particle swarm optimization (PSO) algorithm with wavelet mutation at the upper level. The RNNs act as search agents carrying out precise local searches according to their neurodynamics and initial conditions. The PSO algorithm coordinates and guides the RNNs with updated initial states toward global optimal solution(s). A wavelet mutation operator is added to enhance PSO exploration diversity. Through iterative interaction and improvement of the locally best solutions of RNNs and global best positions of the whole population, the population-based neurodynamic systems are almost sure able to achieve the global optimality for the NMF problem. It is proved that the convergence of the group-best state to the global optimal solution with probability one. The experimental results substantiate the efficacy and superiority of the CNO approach to bound-constrained global optimization with several benchmark nonconvex functions and NMF-based clustering with benchmark data sets in comparison with the state-of-the-art algorithms.

Adaptive Method for Nonsmooth Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is an emerging tool for meaningful low-rank matrix representation. In NMF, explicit constraints are usually required, such that NMF generates desired products (or factorizations), especially when the products have significant sparseness features. It is known that the ability of NMF in learning sparse representation can be improved by embedding a smoothness factor between the products. Motivated by this result, we propose an adaptive nonsmooth NMF (Ans-NMF) method in this paper. In our method, the embedded factor is obtained by using a data-related approach, so it matches well with the underlying products, implying a superior faithfulness of the representations. Besides, due to the usage of an adaptive selection scheme to this factor, the sparseness of the products can be separately constrained, leading to wider applicability and interpretability. Furthermore, since the adaptive selection scheme is processed through solving a series of typical linear programming problems, it can be easily implemented. Simulations using computer-generated data and real-world data show the advantages of the proposed Ans-NMF method over the state-of-the-art methods.

Symmetric subspace learning for image analysis

Subspace learning (SL) is one of the most useful tools for image analysis and recognition. A large number of such techniques have been proposed utilizing a priori knowledge about the data. In this paper, new subspace learning techniques are presented that use symmetry constraints in their objective functions. The rational behind this idea is to exploit the a priori knowledge that geometrical symmetry appears in several types of data, such as images, objects, faces, and so on. Experiments on artificial, facial expression recognition, face recognition, and object categorization databases highlight the superiority and the robustness of the proposed techniques, in comparison with standard SL techniques.

Automated graph regularized projective nonnegative matrix factorization for document clustering

In this paper, a novel projective nonnegative matrix factorization (PNMF) method for enhancing the clustering performance is presented, called automated graph regularized projective nonnegative matrix factorization (AGPNMF). The idea of AGPNMF is to extend the original PNMF by incorporating the automated graph regularized constraint into the PNMF decomposition. The key advantage of this approach is that AGPNMF simultaneously finds graph weights matrix and dimensionality reduction of data. AGPNMF seeks to extract the data representation space that preserves the local geometry structure. This character makes AGPNMF more intuitive and more powerful than the original method for clustering tasks. The kernel trick is used to extend AGPNMF model related to the input space by some nonlinear map. The proposed method has been applied to the problem of document clustering using the well-known Reuters-21578, TDT2, and SECTOR data sets. Our experimental evaluations show that the proposed method enhances the performance of PNMF for document clustering.