A study of graph-based system for multi-view clustering

One-Step Multiview Clustering via Adaptive Graph Learning and Spectral Rotation

In graph based multiview clustering methods, the ultimate partition result is usually achieved by spectral embedding of the consistent graph using some traditional clustering methods, such as -means. However, optimal performance will be reduced by this multistep procedure since it cannot unify graph learning with partition generation closely. In this article, we propose a one-step multiview clustering method through adaptive graph learning and spectral rotation (AGLSR). For every view, AGLSR adaptively learns affinity graphs to capture similar relationships of samples. Then, a spectral embedding is designed to take advantage of the potential feature space shared by different views. In addition, AGLSR utilizes a spectral rotation strategy to obtain the discrete clustering labels from the learned spectral embeddings directly. An effective updating algorithm with proven convergence is derived to optimize the optimization problem. Sufficient experiments on benchmark datasets have clearly demonstrated the effectiveness of the proposed method in six metrics. The code of AGLSR is uploaded at https://github.com/tangchuan2000/AGLSR.

A survey on representation learning for multi-view data

Multi-view clustering has become a rapidly growing field in machine learning and data mining areas by combining useful information from different views for last decades. Although there have been some surveys based on multi-view clustering, most of these works ignore simultaneously taking the self-supervised and non-self supervised multi-view clustering into consideration. We give a novel survey for sorting out the existing algorithms of multi-view clustering in this work, which can be classified into two different categories, i.e., non-self supervised and self-supervised multi-view clustering. We first review the representative approaches based on the non-self supervised multi-view clustering, which consist of methods based on non-representation learning and representation learning. Furthermore, the methods built on non-representation learning contain works based on matrix factorization, kernel and other non-representation learning. Methods based on representation learning consist of multi-view graph clustering, deep representation learning and multi-view subspace clustering. For the methods based on self-supervised multi-view clustering, we divide them into contrastive methods and generative methods. Overall, this survey attempts to give an insightful overview regarding the developments in the multi-view clustering field.

Multiview learning with twin parametric margin SVM

Multiview learning (MVL) seeks to leverage the benefits of diverse perspectives to complement each other, effectively extracting and utilizing the latent information within the dataset. Several twin support vector machine-based MVL (MvTSVM) models have been introduced and demonstrated outstanding performance in various learning tasks. However, MvTSVM-based models face significant challenges in the form of computational complexity due to four matrix inversions, the need to reformulate optimization problems in order to employ kernel-generated surfaces for handling non-linear cases, and the constraint of uniform noise assumption in the training data. Particularly in cases where the data possesses a heteroscedastic error structure, these challenges become even more pronounced. In view of the aforementioned challenges, we propose multiview twin parametric margin support vector machine (MvTPMSVM). MvTPMSVM constructs parametric margin hyperplanes corresponding to both classes, aiming to regulate and manage the impact of the heteroscedastic noise structure existing within the data. The proposed MvTPMSVM model avoids the explicit computation of matrix inversions in the dual formulation, leading to enhanced computational efficiency. We perform an extensive assessment of the MvTPMSVM model using benchmark datasets such as UCI, KEEL, synthetic, and Animals with Attributes (AwA). Our experimental results, coupled with rigorous statistical analyses, confirm the superior generalization capabilities of the proposed MvTPMSVM model compared to the baseline models. The source code of the proposed MvTPMSVM model is available at https://github.com/mtanveer1/MvTPMSVM.

One-Stage Shifted Laplacian Refining for Multiple Kernel Clustering

Graph learning can effectively characterize the similarity structure of sample pairs, hence multiple kernel clustering based on graph learning (MKC-GL) achieves promising results on nonlinear clustering tasks. However, previous methods confine to a "three-stage" scheme, that is, affinity graph learning, Laplacian construction, and clustering indicator extracting, which results in the information distortion in the step alternating. Meanwhile, the energy of Laplacian reconstruction and the necessary cluster information cannot be preserved simultaneously. To address these problems, we propose a one-stage shifted Laplacian refining (OSLR) method for multiple kernel clustering (MKC), where using the "one-stage" scheme focuses on Laplacian learning rather than traditional graph learning. Concretely, our method treats each kernel matrix as an affinity graph rather than ordinary data and constructs its corresponding Laplacian matrix in advance. Compared to the traditional Laplacian methods, we transform each Laplacian to an approximately shifted Laplacian (ASL) for refining a consensus Laplacian. Then, we project the consensus Laplacian onto a Fantope space to ensure that reconstruction information and clustering information concentrate on larger eigenvalues. Theoretically, our OSLR reduces the memory complexity and computation complexity to O(n) and O(n2) , respectively. Moreover, experimental results have shown that it outperforms state-of-the-art MKC methods on multiple benchmark datasets.

Low-Rank Graph Completion-Based Incomplete Multiview Clustering

In order to reduce the negative effect of missing data on clustering, incomplete multiview clustering (IMVC) has become an important research content in machine learning. At present, graph-based methods are widely used in IMVC, but these methods still have some defects. First, some of the methods overlook potential relationships across views. Second, most of the methods depend on local structure information and ignore the global structure information. Third, most of the methods cannot use both global structure information and potential information across views to adaptively recover the incomplete relationship structure. To address the above issues, we propose a unified optimization framework to learn reasonable affinity relationships, called low-rank graph completion-based IMVC (LRGR_IMVC). 1) Our method introduces adaptive graph embedding to effectively explore the potential relationship among views; 2) we append a low-rank constraint to adequately exploit the global structure information among views; and 3) this method unites related information within views, potential information across views, and global structure information to adaptively recover the incomplete graph structure and obtain complete affinity relationships. Experimental results on several commonly used datasets show that the proposed method achieves better clustering performance significantly than some of the most advanced methods.

Deep graph reconstruction for multi-view clustering

Graph-based multi-view clustering methods have achieved impressive success by exploring a complemental or independent graph embedding with low-dimension among multiple views. The majority of them, however, are shallow models with limited ability to learn the nonlinear information in multi-view data. To this end, we propose a novel deep graph reconstruction (DGR) framework for multi-view clustering, which contains three modules. Specifically, a Multi-graph Fusion Module (MFM) is employed to obtain the consensus graph. Then node representation is learned by the Graph Embedding Network (GEN). To assign clusters directly, the Clustering Assignment Module (CAM) is devised to obtain the final low-dimensional graph embedding, which can serve as the indicator matrix. In addition, a simple and powerful loss function is designed in the proposed DGR. Extensive experiments on seven real-world datasets have been conducted to verify the superior clustering performance and efficiency of DGR compared with the state-of-the-art methods.

Balance guided incomplete multi-view spectral clustering

There is a large volume of incomplete multi-view data in the real-world. How to partition these incomplete multi-view data is an urgent realistic problem since almost all of the conventional multi-view clustering methods are inapplicable to cases with missing views. In this paper, a novel graph learning-based incomplete multi-view clustering (IMVC) method is proposed to address this issue. Different from existing works, our method aims at learning a common consensus graph from all incomplete views and obtaining a clustering indicator matrix in a unified framework. To achieve a stable clustering result, a relaxed spectral clustering model is introduced to obtain a probability consensus representation with all positive elements that reflect the data clustering result. Considering the different contributions of views to the clustering task, a weighted multi-view learning mechanism is introduced to automatically balance the effects of different views in model optimization. In this way, the intrinsic information of the incomplete multi-view data can be fully exploited. The experiments on several incomplete multi-view datasets show that our method outperforms the compared state-of-the-art clustering methods, which demonstrates the effectiveness of our method for IMVC.

Multi-view subspace clustering via adaptive graph learning and late fusion alignment

Multi-view subspace clustering has attracted great attention due to its ability to explore data structure by utilizing complementary information from different views. Most of existing methods learn a sample representation coefficient matrix or an affinity graph for each single view, then the final clustering result is obtained from the spectral embedding of a consensus graph using certain traditional clustering techniques, such as k-means. However, clustering performance will be degenerated if the early fusion of partitions cannot fully exploit relationships between all samples. Different from existing methods, we propose a multi-view subspace clustering method via adaptive graph learning and late fusion alignment (AGLLFA). For each view, AGLLFA learns an affinity graph adaptively to capture the similarity relationship among samples. Moreover, a spectral embedding learning term is designed to exploit the latent feature space of different views. Furthermore, we design a late fusion alignment mechanism to generate an optimal clustering partition by fusing view-specific partitions obtained from multiple views. An alternate updating algorithm with validated convergence is developed to solve the resultant optimization problem. Extensive experiments on several benchmark datasets are conducted to illustrate the effectiveness of the proposed method when compared with other state-of-the-art methods. The demo code of this work is publicly available at https://github.com/tangchuan2000/AGLLFA.

Incomplete multi-view clustering network via nonlinear manifold embedding and probability-induced loss

Incomplete multi-view clustering, which included missing data in different views, is more challenging than multi-view clustering. For the purpose of eliminating the negative influence of incomplete data, researchers have proposed a series of solutions. However, the present incomplete multi-view clustering methods still confront three major issues: (1) The interference of redundant features hinders these methods to learn the most discriminative features. (2) The importance role of local structure is not considered during clustering. (3) These methods fail to utilize data distribution information to guide models update to decrease the effects of outliers and noise. To address above issues, a novel deep clustering network which exerted on incomplete multi-view data was proposed in this paper. We combine multi-view autoencoders with nonlinear manifold embedding method UMAP to extract latent consistent features of incomplete multi-view data. In the clustering method, we introduce Gaussian Mixture Model (GMM) to fit the complex distribution of data and deal with the interference of outliers. In addition, we reasonably utilize the probability distribution information generated by GMM, using probability-induced loss function to integrate feature learning and clustering as a joint framework. In experiments conducted on multiple benchmark datasets, our method captures incomplete multi-view data features effectively and perform excellent.

Multi-View Clustering via Nonnegative and Orthogonal Graph Reconstruction

The goal of multi-view clustering is to partition samples into different subsets according to their diverse features. Previous multi-view clustering methods mainly exist two forms: multi-view spectral clustering and multi-view matrix factorization. Although they have shown excellent performance in many occasions, there are still many disadvantages. For example, multi-view spectral clustering usually needs to perform postprocessing. Multi-view matrix factorization directly decomposes the original data features. When the size of features is large, it encounters the expensive time consumption to decompose these data features thoroughly. Therefore, we proposed a novel multi-view clustering approach. The main advantages include the following three aspects: 1) it searches for a common joint graph across multiple views, which fully explores the hidden structure information by utilizing the compatibility among views; 2) the introduced nonnegative constraint manipulates that the final clustering results can be directly obtained; and 3) straightforwardly decomposing the similarity matrix can transform the eigenvalue factorization in spectral clustering with computational complexity O(n³) into the singular value decomposition (SVD) with O(nc²) time cost, where n and c , respectively, denote the numbers of samples and classes. Thus, the computational efficiency can be improved. Moreover, in order to learn a better clustering model, we set that the constructed similarity graph approximates each view affinity graph as close as possible by adding the constraint as the initial affinity matrices own. Furthermore, substantial experiments are conducted, which verifies the superiority of the proposed two clustering methods comparing with single-view clustering approaches and state-of-the-art multi-view clustering methods.

A novel clustering approach to bipartite investor-startup networks

We propose a novel similarity-based clustering approach to venture capital investors that takes as input the bipartite graph of funding interactions between investors and startups and returns clusterings of investors built upon 5 characteristic dimensions. We first validate that investors are clustered in a meaningful manner and present methods of visualizing cluster characteristics. We further analyze the temporal dynamics at the cluster level and observe a meaningful second-order evolution of the sectoral investment trends. Finally, and surprisingly, we report that clusters appear stable even when running the clustering algorithm with all but one of the 5 characteristic dimensions, for instance observing geography-focused clusters without taking into account the geographical dimension or sector-focused clusters without taking into account the sectoral dimension, suggesting the presence of significant underlying complex investment patterns.

An improved multi-view spectral clustering based on tissue-like P systems

Multi-view spectral clustering is one of the multi-view clustering methods widely studied by numerous scholars. The first step of multi-view spectral clustering is to construct the similarity matrix of each view. Consequently, the clustering performance will be greatly affected by the quality of the similarity matrix of each view. To solve this problem well, an improved multi-view spectral clustering based on tissue-like P systems is proposed in this paper. The optimal per-view similarity matrix is generated in an iterative manner. In addition, spectral clustering is combined with the symmetric nonnegative matrix factorization method to directly output the clustering results to avoid the secondary operation, such as k-means or spectral rotation. Furthermore, improved multi-view spectral clustering is integrated with the tissue-like P system to enhance the computational efficiency of the multi-view clustering algorithm. Extensive experiments verify the effectiveness of this algorithm over other state-of-the-art algorithms.

Fast Component Density Clustering in Spatial Databases: A Novel Algorithm

Clustering analysis is a significant technique in various fields, including unsupervised machine learning, data mining, pattern recognition, and image analysis. Many clustering algorithms are currently used, but almost all of them encounter various challenges, such as low accuracy, required number of clusters, slow processing, inability to produce non-spherical shaped clusters, and unstable performance with respect to data characteristics and size. In this research, a novel clustering algorithm called the fast component density clustering in spatial databases (FCDCSD) is proposed by utilizing a density-based clustering technique to address the aforementioned existing challenges. First, from the smallest to the largest point in the spatial field, each point is labeled with a temporary value, and the adjacent values in one component are stored in a set. Then, all sets with shared values are merged and resolved to obtain a single value that is representative of the merged sets. These values represent final cluster values; that is, the temporary equivalents in the dataset are replaced to generate the final clusters. If some noise appears, then a post-process is performed, and values are assigned to the nearest cluster based on a set of rules. Various synthetic datasets were used in the experiments to evaluate the efficiency of the proposed method. Results indicate that FCDCSD is generally superior to affinity propagation, agglomerative hierarchical, k-means, mean-shift, spectral, and density-based spatial clustering of applications with noise, ordering points for identifying clustering structures, and Gaussian mixture clustering methods.

Block Diagonal Least Squares Regression for Subspace Clustering

Least squares regression (LSR) is an effective method that has been widely used for subspace clustering. Under the conditions of independent subspaces and noise-free data, coefficient matrices can satisfy enforced block diagonal (EBD) structures and achieve good clustering results. More importantly, LSR produces closed solutions that are easier to solve. However, solutions with block diagonal properties that have been solved using LSR are sensitive to noise or corruption as they are fragile and easily destroyed. Moreover, when using actual datasets, these structures cannot always guarantee satisfactory clustering results. Considering that block diagonal representation has excellent clustering performance, the idea of block diagonal constraints has been introduced into LSR and a new subspace clustering method, which is named block diagonal least squares regression (BDLSR), has been proposed. By using a block diagonal regularizer, BDLSR can effectively reinforce the fragile block diagonal structures of the obtained matrices and improve the clustering performance. Our experiments using several real datasets illustrated that BDLSR produced a higher clustering performance compared to other algorithms.

Agglomerative Neural Networks for Multiview Clustering

Conventional multiview clustering methods seek a view consensus through minimizing the pairwise discrepancy between the consensus and subviews. However, pairwise comparison cannot portray the interview relationship precisely if some of the subviews can be further agglomerated. To address the above challenge, we propose the agglomerative analysis to approximate the optimal consensus view, thereby describing the subview relationship within a view structure. We present an agglomerative neural network (ANN) based on constrained Laplacian rank to cluster multiview data directly without a dedicated postprocessing step (e.g., using K -means). We further extend ANN with a learnable data space to handle data of complex scenarios. Our evaluations against several state-of-the-art multiview clustering approaches on four popular data sets show the promising view-consensus analysis ability of ANN. We further demonstrate ANN's capability in analyzing complex view structures, extensibility through our case study and robustness and effectiveness of data-driven modifications.

Fusing Local and Global Information for One-Step Multi-View Subspace Clustering

Multi-view subspace clustering has drawn significant attention in the pattern recognition and machine learning research community. However, most of the existing multi-view subspace clustering methods are still limited in two aspects. (1) The subspace representation yielded by the self-expression reconstruction model ignores the local structure information of the data. (2) The construction of subspace representation and clustering are used as two individual procedures, which ignores their interactions. To address these problems, we propose a novel multi-view subspace clustering method fusing local and global information for one-step multi-view clustering. Our contribution lies in three aspects. First, we merge the graph learning into the self-expression model to explore the local structure information for constructing the specific subspace representations of different views. Second, we consider the multi-view information fusion by integrating these specific subspace representations into one common subspace representation. Third, we combine the subspace representation learning, multi-view information fusion, and clustering into a joint optimization model to realize the one-step clustering. We also develop an effective optimization algorithm to solve the proposed method. Comprehensive experimental results on nine popular multi-view data sets confirm the effectiveness and superiority of the proposed method by comparing it with many state-of-the-art multi-view clustering methods.