Federated Multi-View K-Means Clustering

The increasing effect of Internet of Things (IoT) unlocks the massive volume of the availability of Big Data in many fields. Generally, these Big Data may be in a non-independently and identically distributed fashion (non-IID). In this paper, we have contributions in such a way enable multi-view k-means (MVKM) clustering to maintain the privacy of each database by allowing MVKM to be operated on the local principle of clients' multi-view data. This work integrates the exponential distance to transform the weighted Euclidean distance on MVKM so that it can make full use of development in federated learning via the MVKM clustering algorithm. The proposed algorithm, called a federated MVKM (Fed-MVKM), can provide a whole new level adding a lot of new ideas to produce a much better output. The proposed Fed-MVKM is highly suitable for clustering large data sets. To demonstrate its efficient and applicable, we implement a synthetic and six real multi-view data sets and then perform Federated Peter-Clark in Huang et al. 2023 for causal inference setting to split the data instances over multiple clients, efficiently. The results show that shared-models based local cluster centers with data-driven in the federated environment can generate a satisfying final pattern of one multi-view data that simultaneously improve the clustering performance of (non-federated) MVKM clustering algorithms.

One-Step Multi-View Clustering With Diverse Representation

Multi-View clustering has attracted broad attention due to its capacity to utilize consistent and complementary information among views. Although tremendous progress has been made recently, most existing methods undergo high complexity, preventing them from being applied to large-scale tasks. Multi-View clustering via matrix factorization is a representative to address this issue. However, most of them map the data matrices into a fixed dimension, limiting the model's expressiveness. Moreover, a range of methods suffers from a two-step process, i.e., multimodal learning and the subsequent k-means, inevitably causing a suboptimal clustering result. In light of this, we propose a one-step multi-view clustering with diverse representation (OMVCDR) method, which incorporates multi-view learning and k-means into a unified framework. Specifically, we first project original data matrices into various latent spaces to attain comprehensive information and auto-weight them in a self-supervised manner. Then, we directly use the information matrices under diverse dimensions to obtain consensus discrete clustering labels. The unified work of representation learning and clustering boosts the quality of the final results. Furthermore, we develop an efficient optimization algorithm with proven convergence to solve the resultant problem. Comprehensive experiments on various datasets demonstrate the promising clustering performance of our proposed method. The code is publicly available at https://github.com/wanxinhang/OMVCDR.

A Survey on Multi-View Clustering

Clustering is a machine learning paradigm of dividing sample subjects into a number of groups such that subjects in the same groups are more similar to those in other groups. With advances in information acquisition technologies, samples can frequently be viewed from different angles or in different modalities, generating multi-view data. Multi-view clustering, that clusters subjects into subgroups using multi-view data, has attracted more and more attentions. Although MVC methods have been developed rapidly, there has not been enough survey to summarize and analyze the current progress. Therefore, we propose a novel taxonomy of the MVC approaches. Similar to other machine learning methods, we categorize them into generative and discriminative classes. In discriminative class, based on the way of view integration, we split it further into five groups: Common Eigenvector Matrix, Common Coefficient Matrix, Common Indicator Matrix, Direct Combination and Combination After Projection. Furthermore, we relate MVC to other topics: multi-view representation, ensemble clustering, multi-task clustering, multi-view supervised and semi-supervised learning. Several representative real-world applications are elaborated for practitioners. Some benchmark multi-view datasets are introduced and representative MVC algorithms from each group are empirically evaluated to analyze how they perform on benchmark datasets. To promote future development of MVC approaches, we point out several open problems that may require further investigation and thorough examination.

Prediction of Short-Term Stock Price Trend Based on Multiview RBF Neural Network

Stock price prediction is important in both financial and commercial domains, and using neural networks to forecast stock prices has been a topic of ongoing research and development. Traditional prediction models are often based on a single type of data and do not account for the interplay of many variables. This study covers a radial basis neural network modeling technique with multiview collaborative learning capabilities for incorporating the impacts of numerous elements into the prediction model. This research offers a multiview RBF neural network prediction model based on the classic RBF network by integrating a collaborative learning item with multiview learning capabilities (MV-RBF). MV-RBF can make full use of both the internal information provided by the correlation between each view and the distinct characteristics of each view to form independent sample information. By using two separate stock qualities as input feature information for trials, this study proves the viability of the multiview RBF neural network prediction model on a real data set.

Multiple clustering for identifying subject clusters and brain sub-networks using functional connectivity matrices without vectorization

In neuroscience, the functional magnetic resonance imaging (fMRI) is a vital tool to non-invasively access brain activity. Using fMRI, the functional connectivity (FC) between brain regions can be inferred, which has contributed to a number of findings of the fundamental properties of the brain. As an important clinical application of FC, clustering of subjects based on FC recently draws much attention, which can potentially reveal important heterogeneity in subjects such as subtypes of psychiatric disorders. In particular, a multiple clustering method is a powerful analytical tool, which identifies clustering patterns of subjects depending on their FC in specific brain areas. However, when one applies an existing multiple clustering method to fMRI data, there is a need to simplify the data structure, independently dealing with elements in a FC matrix, i.e., vectorizing a correlation matrix. Such a simplification may distort the clustering results. To overcome this problem, we propose a novel multiple clustering method based on Wishart mixture models, which preserves the correlation matrix structure without vectorization. The uniqueness of this method is that the multiple clustering of subjects is based on particular networks of nodes (or regions of interest, ROIs), optimized in a data-driven manner. Hence, it can identify multiple underlying pairs of associations between a subject cluster solution and a ROI sub-network. The key assumption of the method is independence among sub-networks, which is effectively addressed by whitening correlation matrices. We applied the proposed method to synthetic and fMRI data, demonstrating the usefulness and power of the proposed method.

Discrete Multi-Graph Clustering

Spectral clustering plays a significant role in applications that rely on multi-view data due to its well-defined mathematical framework and excellent performance on arbitrarily-shaped clusters. Unfortunately, directly optimizing the spectral clustering inevitably results in an NP-hard problem due to the discrete constraints on the clustering labels. Hence, conventional approaches intuitively include a relax-and-discretize strategy to approximate the original solution. However, there are no principles in this strategy that prevent the possibility of information loss between each stage of the process. This uncertainty is aggravated when a procedure of heterogeneous features fusion has to be included in multi-view spectral clustering. In this paper, we avoid an NP-hard optimization problem and develop a general framework for multi-view discrete graph clustering by directly learning a consensus partition across multiple views, instead of using the relax-and-discretize strategy. An effective re-weighting optimization algorithm is exploited to solve the proposed challenging problem. Further, we provide a theoretical analysis of the model's convergence properties and computational complexity for the proposed algorithm. Extensive experiments on several benchmark datasets verify the effectiveness and superiority of the proposed algorithm on clustering and image segmentation tasks.

Dynamic Affinity Graph Construction for Spectral Clustering Using Multiple Features

Spectral clustering (SC) has been widely applied to various computer vision tasks, where the key is to construct a robust affinity matrix for data partitioning. With the increase in visual features, conventional SC methods are facing two challenges: 1) how to effectively generate an affinity matrix based on multiple features? and 2) how to deal with high-dimensional visual features which could be redundant? To address these issues mentioned earlier, we present a new approach to: 1) learn a robust affinity matrix using multiple features, allowing us to simultaneously determine optimal weights for each feature; and 2) decide a set of optimal projection matrixes, one for each feature, that decide the lower dimensional space, as well as the optimal affinity weight of each data pair in the lower dimensional space. There are two major advantages of our new approach over the existing clustering techniques. First, our approach assigns affinity weights for data points on a per-data-pair basis. The learning procedure avoids the explicit specification of the size of the neighborhood in the affinity matrix, and the bandwidth parameter required to compute the Gaussian kernel, both of which are sensitive and yet difficult to determine beforehand. Second, the affinity weights are based on the distances in a lower dimensional space, while the low-dimensional space is inferred according to the optimized affinity weights. Both variables are jointly optimized so as to leverage mutual benefits. The experimental results outperform the compared alternatives, which indicate that the proposed method is effective in simultaneously learning the affinity graph and feature fusion, resulting in better clustering results.

Alternative Multiview Maximum Entropy Discrimination

Maximum entropy discrimination (MED) is a general framework for discriminative estimation based on maximum entropy and maximum margin principles, and can produce hard-margin support vector machines under some assumptions. Recently, the multiview version of MED multiview MED (MVMED) was proposed. In this paper, we try to explore a more natural MVMED framework by assuming two separate distributions p1( Θ1) over the first-view classifier parameter Θ1 and p2( Θ2) over the second-view classifier parameter Θ2 . We name the new MVMED framework as alternative MVMED (AMVMED), which enforces the posteriors of two view margins to be equal. The proposed AMVMED is more flexible than the existing MVMED, because compared with MVMED, which optimizes one relative entropy, AMVMED assigns one relative entropy term to each of the two views, thus incorporating a tradeoff between the two views. We give the detailed solving procedure, which can be divided into two steps. The first step is solving our optimization problem without considering the equal margin posteriors from two views, and then, in the second step, we consider the equal posteriors. Experimental results on multiple real-world data sets verify the effectiveness of the AMVMED, and comparisons with MVMED are also reported.