Multi-view clustering by non-negative matrix factorization with co-orthogonal constraints

Fast Multiview Anchor-Graph Clustering

Due to its high computational complexity, graph-based methods have limited applicability in large-scale multiview clustering tasks. To address this issue, many accelerated algorithms, especially anchor graph-based methods and indicator learning-based methods, have been developed and made a great success. Nevertheless, since the restrictions of the optimization strategy, these accelerated methods still need to approximate the discrete graph-cutting problem to a continuous spectral embedding problem and utilize different discretization strategies to obtain discrete sample categories. To avoid the loss of effectiveness and efficiency caused by the approximation and discretization, we establish a discrete fast multiview anchor graph clustering (FMAGC) model that first constructs an anchor graph of each view and then generates a discrete cluster indicator matrix by solving the discrete multiview graph-cutting problem directly. Since the gradient descent-based method makes it hard to solve this discrete model, we propose a fast coordinate descent-based optimization strategy with linear complexity to solve it without approximating it as a continuous one. Extensive experiments on widely used normal and large-scale multiview datasets show that FMAGC can improve clustering effectiveness and efficiency compared to other state-of-the-art baselines.

One-Step Multi-View Clustering With Diverse Representation

Multi-View clustering has attracted broad attention due to its capacity to utilize consistent and complementary information among views. Although tremendous progress has been made recently, most existing methods undergo high complexity, preventing them from being applied to large-scale tasks. Multi-View clustering via matrix factorization is a representative to address this issue. However, most of them map the data matrices into a fixed dimension, limiting the model's expressiveness. Moreover, a range of methods suffers from a two-step process, i.e., multimodal learning and the subsequent k-means, inevitably causing a suboptimal clustering result. In light of this, we propose a one-step multi-view clustering with diverse representation (OMVCDR) method, which incorporates multi-view learning and k-means into a unified framework. Specifically, we first project original data matrices into various latent spaces to attain comprehensive information and auto-weight them in a self-supervised manner. Then, we directly use the information matrices under diverse dimensions to obtain consensus discrete clustering labels. The unified work of representation learning and clustering boosts the quality of the final results. Furthermore, we develop an efficient optimization algorithm with proven convergence to solve the resultant problem. Comprehensive experiments on various datasets demonstrate the promising clustering performance of our proposed method. The code is publicly available at https://github.com/wanxinhang/OMVCDR.

One-Step Multiview Fuzzy Clustering With Collaborative Learning Between Common and Specific Hidden Space Information

Multiview data are widespread in real-world applications, and multiview clustering is a commonly used technique to effectively mine the data. Most of the existing algorithms perform multiview clustering by mining the commonly hidden space between views. Although this strategy is effective, there are two challenges that still need to be addressed to further improve the performance. First, how to design an efficient hidden space learning method so that the learned hidden spaces contain both shared and specific information of multiview data. Second, how to design an efficient mechanism to make the learned hidden space more suitable for the clustering task. In this study, a novel one-step multiview fuzzy clustering (OMFC-CS) method is proposed to address the two challenges by collaborative learning between the common and specific space information. To tackle the first challenge, we propose a mechanism to extract the common and specific information simultaneously based on matrix factorization. For the second challenge, we design a one-step learning framework to integrate the learning of common and specific spaces and the learning of fuzzy partitions. The integration is achieved in the framework by performing the two learning processes alternately and thereby yielding mutual benefit. Furthermore, the Shannon entropy strategy is introduced to obtain the optimal views weight assignment during clustering. The experimental results based on benchmark multiview datasets demonstrate that the proposed OMFC-CS outperforms many existing methods.

Structural deep multi-view clustering with integrated abstraction and detail

Deep multi-view clustering, which can obtain complementary information from different views, has received considerable attention in recent years. Although some efforts have been made and achieve decent performances, most of them overlook the structural information and are susceptible to poor quality views, which may seriously restrict the capacity for clustering. To this end, we propose Structural deep Multi-View Clustering with integrated abstraction and detail (SMVC). Specifically, multi-layer perceptrons are used to extract features from specific views, which are then concatenated to form the global features. Besides, a global target distribution is constructed and guides the soft cluster assignments of specific views. In addition to the exploitation of the top-level abstraction, we also design the mining of the underlying details. We construct instance-level contrastive learning using high-order adjacency matrices, which has an equivalent effect to graph attention network and reduces feature redundancy. By integrating the top-level abstraction and underlying detail into a unified framework, our model can jointly optimize the cluster assignments and feature embeddings. Extensive experiments on four benchmark datasets have demonstrated that the proposed SMVC consistently outperforms the state-of-the-art methods.

Comprehensive Multiview Representation Learning via Deep Autoencoder-Like Nonnegative Matrix Factorization

Learning a comprehensive representation from multiview data is crucial in many real-world applications. Multiview representation learning (MRL) based on nonnegative matrix factorization (NMF) has been widely adopted by projecting high-dimensional space into a lower order dimensional space with great interpretability. However, most prior NMF-based MRL techniques are shallow models that ignore hierarchical information. Although deep matrix factorization (DMF)-based methods have been proposed recently, most of them only focus on the consistency of multiple views and have cumbersome clustering steps. To address the above issues, in this article, we propose a novel model termed deep autoencoder-like NMF for MRL (DANMF-MRL), which obtains the representation matrix through the deep encoding stage and decodes it back to the original data. In this way, through a DANMF-based framework, we can simultaneously consider the multiview consistency and complementarity, allowing for a more comprehensive representation. We further propose a one-step DANMF-MRL, which learns the latent representation and final clustering labels matrix in a unified framework. In this approach, the two steps can negotiate with each other to fully exploit the latent clustering structure, avoid previous tedious clustering steps, and achieve optimal clustering performance. Furthermore, two efficient iterative optimization algorithms are developed to solve the proposed models both with theoretical convergence analysis. Extensive experiments on five benchmark datasets demonstrate the superiority of our approaches against other state-of-the-art MRL methods.

Fast multi-view clustering via correntropy-based orthogonal concept factorization

Owing to its ability to handle negative data and promising clustering performance, concept factorization (CF), an improved version of non-negative matrix factorization, has been incorporated into multi-view clustering recently. Nevertheless, existing CF-based multi-view clustering methods still have the following issues: (1) they directly conduct factorization in the original data space, which means its efficiency is sensitive to the feature dimension; (2) they ignore the high degree of factorization freedom of standard CF, which may lead to non-uniqueness factorization thereby causing reduced effectiveness; (3) traditional robust norms they used are unable to handle complex noises, significantly challenging their robustness. To address these issues, we establish a fast multi-view clustering via correntropy-based orthogonal concept factorization (FMVCCF). Specifically, FMVCCF executes factorization on a learned consensus anchor graph rather than directly decomposing the original data, lessening the dimensionality sensitivity. Then, a lightweight graph regularization term is incorporated to refine the factorization process with a low computational burden. Moreover, an improved multi-view correntropy-based orthogonal CF model is developed, which can enhance the effectiveness and robustness under the orthogonal constraint and correntropy criterion, respectively. Extensive experiments demonstrate that FMVCCF can achieve promising effectiveness and robustness on various real-world datasets with high efficiency.

Multiview Clustering via Proximity Learning in Latent Representation Space

Most existing multiview clustering methods are based on the original feature space. However, the feature redundancy and noise in the original feature space limit their clustering performance. Aiming at addressing this problem, some multiview clustering methods learn the latent data representation linearly, while performance may decline if the relation between the latent data representation and the original data is nonlinear. The other methods which nonlinearly learn the latent data representation usually conduct the latent representation learning and clustering separately, resulting in that the latent data representation might be not well adapted to clustering. Furthermore, none of them model the intercluster relation and intracluster correlation of data points, which limits the quality of the learned latent data representation and therefore influences the clustering performance. To solve these problems, this article proposes a novel multiview clustering method via proximity learning in latent representation space, named multiview latent proximity learning (MLPL). For one thing, MLPL learns the latent data representation in a nonlinear manner which takes the intercluster relation and intracluster correlation into consideration simultaneously. For another, through conducting the latent representation learning and consensus proximity learning simultaneously, MLPL learns a consensus proximity matrix with k connected components to output the clustering result directly. Extensive experiments are conducted on seven real-world datasets to demonstrate the effectiveness and superiority of the MLPL method compared with the state-of-the-art multiview clustering methods.

CSMVC: A Multiview Method for Multivariate Time-Series Clustering

Multivariate time-series (MTS) clustering is a fundamental technique in data mining with a wide range of real-world applications. To date, though some approaches have been developed, they suffer from various drawbacks, such as high computational cost or loss of information. Most existing approaches are single-view methods without considering the benefits of mutual-support multiple views. Moreover, due to its data structure, MTS data cannot be handled well by most multiview clustering methods. Toward this end, we propose a consistent and specific non-negative matrix factorization-based multiview clustering (CSMVC) method for MTS clustering. The proposed method constructs a multilayer graph to represent the original MTS data and generates multiple views with a subspace technique. The obtained multiview data are processed through a novel non-negative matrix factorization (NMF) method, which can explore the view-consistent and view-specific information simultaneously. Furthermore, an alternating optimization scheme is proposed to solve the corresponding optimization problem. We conduct extensive experiments on 13 benchmark datasets and the results demonstrate the superiority of our proposed method against other state-of-the-art algorithms under a wide range of evaluation metrics.

Efficient correntropy-based multi-view clustering with anchor graph embedding

Although multi-view clustering has received widespread attention due to its far superior performance to single-view clustering, it still faces the following issues: (1) high computational cost, considering the introduction of multi-view information, reduces the clustering efficiency greatly; (2) complex noises and outliers, existed in real-world data, pose a huge challenge to the robustness of clustering algorithms. Currently, how to increase the efficiency and robustness has become two important issues of multi-view clustering. To cope with the above issues, an efficient correntropy-based multi-view clustering algorithm (ECMC) is proposed in this paper, which can not only improve clustering efficiency by constructing embedded anchor graph and utilizing nonnegative matrix factorization (NMF), but also enhance the robustness by exploring correntropy to suppress various noises and outliers. To further improve clustering efficiency, one of the factors of NMF is constrained to be an indicator matrix instead of a traditional non-negative matrix, so that the categories of samples can be obtained directly without any extra operation. Subsequently, a novel half-quadratic-based strategy is proposed to optimize the non-convex objective function of ECMC. Finally, extensive experiments on eight real-world datasets and eighteen noisy datasets show that ECMC can guarantee faster speed and better robustness than other state-of-the-art multi-view clustering algorithms.