Multi-view graph embedding clustering network: Joint self-supervision and block diagonal representation

Attributed Multi-Order Graph Convolutional Network for Heterogeneous Graphs

Heterogeneous graph neural networks play a crucial role in discovering discriminative node embeddings and relations from multi-relational networks. One of the key challenges in heterogeneous graph learning lies in designing learnable meta-paths, which significantly impact the quality of learned embeddings. In this paper, we propose an Attributed Multi-Order Graph Convolutional Network (AMOGCN), which automatically explores meta-paths that involve multi-hop neighbors by aggregating multi-order adjacency matrices. The proposed model first constructs different orders of adjacency matrices from manually designed node connections. Next, AMOGCN fuses these various orders of adjacency matrices to create an intact multi-order adjacency matrix. This process is supervised by the node semantic information, which is extracted from the node homophily evaluated by attributes. Eventually, we employ a one-layer simplifying graph convolutional network with the learned multi-order adjacency matrix, which is equivalent to the cross-hop node information propagation with multi-layer graph neural networks. Substantial experiments reveal that AMOGCN achieves superior semi-supervised classification performance compared with state-of-the-art competitors.

Adaptive-weighted deep multi-view clustering with uniform scale representation

Multi-view clustering has attracted growing attention owing to its powerful capacity of multi-source information integration. Although numerous advanced methods have been proposed in past decades, most of them generally fail to distinguish the unequal importance of multiple views to the clustering task and overlook the scale uniformity of learned latent representation among different views, resulting in blurry physical meaning and suboptimal model performance. To address these issues, in this paper, we propose a joint learning framework, termed Adaptive-weighted deep Multi-view Clustering with Uniform scale representation (AMCU). Specifically, to achieve more reasonable multi-view fusion, we introduce an adaptive weighting strategy, which imposes simplex constraints on heterogeneous views for measuring their varying degrees of contribution to consensus prediction. Such a simple yet effective strategy shows its clear physical meaning for the multi-view clustering task. Furthermore, a novel regularizer is incorporated to learn multiple latent representations sharing approximately the same scale, so that the objective for calculating clustering loss cannot be sensitive to the views and thus the entire model training process can be guaranteed to be more stable as well. Through comprehensive experiments on eight popular real-world datasets, we demonstrate that our proposal performs better than several state-of-the-art single-view and multi-view competitors.

Collaborative Decision-Reinforced Self-Supervision for Attributed Graph Clustering

Attributed graph clustering aims to partition nodes of a graph structure into different groups. Recent works usually use variational graph autoencoder (VGAE) to make the node representations obey a specific distribution. Although they have shown promising results, how to introduce supervised information to guide the representation learning of graph nodes and improve clustering performance is still an open problem. In this article, we propose a Collaborative Decision-Reinforced Self-Supervision (CDRS) method to solve the problem, in which a pseudo node classification task collaborates with the clustering task to enhance the representation learning of graph nodes. First, a transformation module is used to enable end-to-end training of existing methods based on VGAE. Second, the pseudo node classification task is introduced into the network through multitask learning to make classification decisions for graph nodes. The graph nodes that have consistent decisions on clustering and pseudo node classification are added to a pseudo-label set, which can provide fruitful self-supervision for subsequent training. This pseudo-label set is gradually augmented during training, thus reinforcing the generalization capability of the network. Finally, we investigate different sorting strategies to further improve the quality of the pseudo-label set. Extensive experiments on multiple datasets show that the proposed method achieves outstanding performance compared with state-of-the-art methods. Our code is available at https://github.com/Jillian555/TNNLS_CDRS.

Contrastive self-representation learning for data clustering

This paper is concerned with self-representation subspace learning. It is one of the most representative subspace techniques, which has attracted considerable attention for clustering due to its good performance. Among these methods, low-rank representation (LRR) has achieved impressive results for subspace clustering. However, it only considers the similarity between the data itself, while neglecting the differences with other samples. Besides, it cannot well deal with noise and portray cluster-to-cluster relationships well. To solve these problems, we propose a Contrastive Self-representation model for Clustering (CSC). CSC simultaneously takes into account the similarity/dissimilarity between positive/negative pairs when learning the self-representation coefficient matrix of data while the form of the loss function can reduce the effect of noise on the results. Moreover, We use the ℓ1,2-norm regularizer on the coefficient matrix to achieve its sparsity to better characterize the cluster structure. Thus, the learned self-representation coefficient matrix well encodes both the discriminative information and cluster structure. Extensive experiments on seven benchmark databases indicate the superiority of our proposed method.

LoyalDE: Improving the performance of Graph Neural Networks with loyal node discovery and emphasis

Recent years have witnessed an increasing focus on graph-based semi-supervised learning with Graph Neural Networks (GNNs). Despite existing GNNs having achieved remarkable accuracy, research on the quality of graph supervision information has inadvertently been ignored. In fact, there are significant differences in the quality of supervision information provided by different labeled nodes, and treating supervision information with different qualities equally may lead to sub-optimal performance of GNNs. We refer to this as the graph supervision loyalty problem, which is a new perspective for improving the performance of GNNs. In this paper, we devise FT-Score to quantify node loyalty by considering both the local feature similarity and the local topology similarity, and nodes with higher loyalty are more likely to provide higher-quality supervision. Based on this, we propose LoyalDE (Loyal Node Discovery and Emphasis), a model-agnostic hot-plugging training strategy, which can discover potential nodes with high loyalty to expand the training set, and then emphasize nodes with high loyalty during model training to improve performance. Experiments demonstrate that the graph supervision loyalty problem will fail most existing GNNs. In contrast, LoyalDE brings about at most 9.1% performance improvement to vanilla GNNs and consistently outperforms several state-of-the-art training strategies for semi-supervised node classification.

Deep learning on multi-view sequential data: a survey

With the progress of human daily interaction activities and the development of industrial society, a large amount of media data and sensor data become accessible. Humans collect these multi-source data in chronological order, called multi-view sequential data (MvSD). MvSD has numerous potential application domains, including intelligent transportation, climate science, health care, public safety and multimedia, etc. However, as the volume and scale of MvSD increases, the traditional machine learning methods become difficult to withstand such large-scale data, and it is no longer appropriate to use hand-craft features to represent these complex data. In addition, there is no general framework in the process of mining multi-view relationships and integrating multi-view information. In this paper, We first introduce four common data types that constitute MvSD, including point data, sequence data, graph data, and raster data. Then, we summarize the technical challenges of MvSD. Subsequently, we review the recent progress in deep learning technology applied to MvSD. Meanwhile, we discuss how the network represents and learns features of MvSD. Finally, we summarize the applications of MvSD in different domains and give potential research directions.

Sequential multi-view subspace clustering

Self-representation based subspace learning has shown its effectiveness in many applications, but most existing methods do not consider the difference between different views. As a result, the learned self-representation matrix cannot well characterize the clustering structure. Moreover, some methods involve an undesired weighted vector of the tensor nuclear norm, which reduces the flexibility of the algorithm in practical applications. To handle these problems, we present a tensorized multi-view subspace clustering. Specifically, our method employs matrix factorization and decomposes the self-representation matrix to orthogonal projection matrix and affinity matrix. We also add ℓ_1,2-norm regularization on affinity representation to characterize the cluster structure. Moreover, the proposed method uses weighted tensor Schatten p-norm to explore higher-order structure and complementary information embedded in multi-view data, which can allocate the ideal weight for each view automatically without additional weight and penalty parameters. We apply the adaptive loss function to the model to maintain the robustness to outliers and efficiently learn the data distribution. Extensive experimental results on different datasets reveal that our method is superior to other state-of-the-art multi-view subspace clustering methods.