Incomplete-Data Oriented Multiview Dimension Reduction via Sparse Low-Rank Representation

Selective Contrastive Learning for Unpaired Multi-View Clustering

In this article, we investigate a novel but insufficiently studied issue, unpaired multi-view clustering (UMC), where no paired observed samples exist in multi-view data, and the goal is to leverage the unpaired observed samples in all views for effective joint clustering. Existing methods in incomplete multi-view clustering usually utilize the sample pairing relationship between views to connect the views for joint clustering, but unfortunately, it is invalid for the UMC case. Therefore, we strive to mine a consistent cluster structure between views and propose an effective method, namely selective contrastive learning for UMC (scl-UMC), which needs to solve the following two challenging issues: 1) uncertain clustering structure under no supervision information and 2) uncertain pairing relationship between the clusters of views. Specifically, for the first one, we design an inner-view (IV) selective contrastive learning module to enhance the clustering structures and alleviate the uncertainty, which selects confident samples near the cluster centroids to perform contrastive learning in each view. For the second one, we design a cross-view (CV) selective contrastive learning module to first iteratively match the clusters between views and then tighten the matched clusters. Also, we utilize mutual information to further enhance the correlation of the matched clusters between views. Extensive experiments show the efficiency of our methods for UMC, compared with the state-of-the-art methods.

Iterative Multiview Subspace Learning for Unpaired Multiview Clustering

In real applications, several unpredictable or uncertain factors could result in unpaired multiview data, i.e., the observed samples between views cannot be matched. Since joint clustering among views is more effective than individual clustering in each view, we investigate unpaired multiview clustering (UMC), which is a valuable but insufficiently studied problem. Due to lack of matched samples between views, we could fail to build the connection between views. Therefore, we aim to learn the latent subspace shared by views. However, existing multiview subspace learning methods usually rely on the matched samples between views. To address this issue, we propose an iterative multiview subspace learning strategy [iterative unpaired multiview clustering (IUMC)], aiming to learn a complete and consistent subspace representation among views for UMC. Moreover, based on IUMC, we design two effective UMC methods: 1) Iterative unpaired multiview clustering via covariance matrix alignment (IUMC-CA) that further aligns the covariance matrix of subspace representations and then performs clustering on the subspace and 2) iterative unpaired multiview clustering via one-stage clustering assignments (IUMC-CY) that performs one-stage multiview clustering (MVC) by replacing the subspace representations with clustering assignments. Extensive experiments show the excellent performance of our methods for UMC, compared with the state-of-the-art methods. Also, the clustering performance of observed samples in each view can be considerably improved by those observed samples from the other views. In addition, our methods have good applicability in incomplete MVC.

Partial Tubal Nuclear Norm-Regularized Multiview Subspace Learning

In this article, a unified multiview subspace learning model, called partial tubal nuclear norm-regularized multiview subspace learning (PTN2MSL), was proposed for unsupervised multiview subspace clustering (MVSC), semisupervised MVSC, and multiview dimension reduction. Unlike most of the existing methods which treat the above three related tasks independently, PTN2MSL integrates the projection learning and the low-rank tensor representation to promote each other and mine their underlying correlations. Moreover, instead of minimizing the tensor nuclear norm which treats all singular values equally and neglects their differences, PTN2MSL develops the partial tubal nuclear norm (PTNN) as a better alternative solution by minimizing the partial sum of tubal singular values. The PTN2MSL method was applied to the above three multiview subspace learning tasks. It demonstrated that these tasks organically benefited from each other and PTN2MSL has achieved better performance in comparison to state-of-the-art methods.

Diversified Regularization Enhanced Training for Effective Manipulator Calibration

Recently, robot arms have become an irreplaceable production tool, which play an important role in the industrial production. It is necessary to ensure the absolute positioning accuracy of the robot to realize automatic production. Due to the influence of machining tolerance, assembly tolerance, the robot positioning accuracy is poor. Therefore, in order to enable the precise operation of the robot, it is necessary to calibrate the robotic kinematic parameters. The least square method and Levenberg-Marquardt (LM) algorithm are commonly used to identify the positioning error of robot. However, it generally has the overfitting caused by improper regularization schemes. To solve this problem, this article discusses six regularization schemes based on its error models, i.e., L1 , L2 , dropout, elastic, log, and swish. Moreover, this article proposes a scheme with six regularization to obtain a reliable ensemble, which can effectively avoid overfitting. The positioning accuracy of the robot is improved significantly after calibration by enough experiments, which verifies the feasibility of the proposed method.

Feature selection based on self-information and entropy measures for incomplete neighborhood decision systems

For incomplete datasets with mixed numerical and symbolic features, feature selection based on neighborhood multi-granulation rough sets (NMRS) is developing rapidly. However, its evaluation function only considers the information contained in the lower approximation of the neighborhood decision, which easily leads to the loss of some information. To solve this problem, we construct a novel NMRS-based uncertain measure for feature selection, named neighborhood multi-granulation self-information-based pessimistic neighborhood multi-granulation tolerance joint entropy (PTSIJE), which can be used to incomplete neighborhood decision systems. First, from the algebra view, four kinds of neighborhood multi-granulation self-information measures of decision variables are proposed by using the upper and lower approximations of NMRS. We discuss the related properties, and find the fourth measure-lenient neighborhood multi-granulation self-information measure (NMSI) has better classification performance. Then, inspired by the algebra and information views simultaneously, a feature selection method based on PTSIJE is proposed. Finally, the Fisher score method is used to delete uncorrelated features to reduce the computational complexity for high-dimensional gene datasets, and a heuristic feature selection algorithm is raised to improve classification performance for mixed and incomplete datasets. Experimental results on 11 datasets show that our method selects fewer features and has higher classification accuracy than related methods.

A Novel Formulation of Trace Ratio Linear Discriminant Analysis

The linear discriminant analysis (LDA) method needs to be transformed into another form to acquire an approximate closed-form solution, which could lead to the error between the approximate solution and the true value. Furthermore, the sensitivity of dimensionality reduction (DR) methods to subspace dimensionality cannot be eliminated. In this article, a new formulation of trace ratio LDA (TRLDA) is proposed, which has an optimal solution of LDA. When solving the projection matrix, the TRLDA method given by us is transformed into a quadratic problem with regard to the Stiefel manifold. In addition, we propose a new trace difference problem named optimal dimensionality linear discriminant analysis (ODLDA) to determine the optimal subspace dimension. The nonmonotonicity of ODLDA guarantees the existence of optimal subspace dimensionality. Both the two approaches have achieved efficient DR on several data sets.

Convergence Analysis of Single Latent Factor-Dependent, Nonnegative, and Multiplicative Update-Based Nonnegative Latent Factor Models

A single latent factor (LF)-dependent, nonnegative, and multiplicative update (SLF-NMU) learning algorithm is highly efficient in building a nonnegative LF (NLF) model defined on a high-dimensional and sparse (HiDS) matrix. However, convergence characteristics of such NLF models are never justified in theory. To address this issue, this study conducts rigorous convergence analysis for an SLF-NMU-based NLF model. The main idea is twofold: 1) proving that its learning objective keeps nonincreasing with its SLF-NMU-based learning rules via constructing specific auxiliary functions; and 2) proving that it converges to a stable equilibrium point with its SLF-NMU-based learning rules via analyzing the Karush-Kuhn-Tucker (KKT) conditions of its learning objective. Experimental results on ten HiDS matrices from real applications provide numerical evidence that indicates the correctness of the achieved proof.

Multiview Concept Learning Via Deep Matrix Factorization

Multiview representation learning (MVRL) leverages information from multiple views to obtain a common representation summarizing the consistency and complementarity in multiview data. Most previous matrix factorization-based MVRL methods are shallow models that neglect the complex hierarchical information. The recently proposed deep multiview factorization models cannot explicitly capture consistency and complementarity in multiview data. We present the deep multiview concept learning (DMCL) method, which hierarchically factorizes the multiview data, and tries to explicitly model consistent and complementary information and capture semantic structures at the highest abstraction level. We explore two variants of the DMCL framework, DMCL-L and DMCL-N, with respectively linear/nonlinear transformations between adjacent layers. We propose two block coordinate descent-based optimization methods for DMCL-L and DMCL-N. We verify the effectiveness of DMCL on three real-world data sets for both clustering and classification tasks.

Multiview Uncorrelated Locality Preserving Projection

Canonical Correlation Analysis (CCA) is a popular multiview dimension reduction method, which aims to maximize the correlation between two views to find the common subspace shared by these two views. However, it can only deal with two-view data, while the number of views frequently exceeds two in many real applications. To handle data with more than two views, in the previous studies, either the pairwise correlation or the high-order correlation was employed. These two types of correlation define the relation of multiview data from different viewpoints, and both have special effects for view consistency. To obtain flexible view consistency, in this article, we propose multiview uncorrelated locality preserving projection (MULPP), which considers two types of correlation simultaneously. The MULPP also considers the complementary property of different views by preserving the local structures of all the views. To obtain multiple projections and minimize the redundancy of low-dimensional features, for each view, the MULPP makes the features extracted by different projections uncorrelated. The MULPP is solved by an iteration algorithm, and the convergence of the algorithm is proven. The experiments on Multiple Feature, Coil-100, 3Sources, and NUS-WIDE data sets demonstrate the effectiveness of MULPP.

Tensor Networks for Latent Variable Analysis: Higher Order Canonical Polyadic Decomposition

The canonical polyadic decomposition (CPD) is a convenient and intuitive tool for tensor factorization; however, for higher order tensors, it often exhibits high computational cost and permutation of tensor entries, and these undesirable effects grow exponentially with the tensor order. Prior compression of tensor in-hand can reduce the computational cost of CPD, but this is only applicable when the rank R of the decomposition does not exceed the tensor dimensions. To resolve these issues, we present a novel method for CPD of higher order tensors, which rests upon a simple tensor network of representative inter-connected core tensors of orders not higher than 3. For rigor, we develop an exact conversion scheme from the core tensors to the factor matrices in CPD and an iterative algorithm of low complexity to estimate these factor matrices for the inexact case. Comprehensive simulations over a variety of scenarios support the proposed approach.

Leveraging Coupled Interaction for Multimodal Alzheimer's Disease Diagnosis

As the population becomes older worldwide, accurate computer-aided diagnosis for Alzheimer's disease (AD) in the early stage has been regarded as a crucial step for neurodegeneration care in recent years. Since it extracts the low-level features from the neuroimaging data, previous methods regarded this computer-aided diagnosis as a classification problem that ignored latent featurewise relation. However, it is known that multiple brain regions in the human brain are anatomically and functionally interlinked according to the current neuroscience perspective. Thus, it is reasonable to assume that the extracted features from different brain regions are related to each other to some extent. Also, the complementary information between different neuroimaging modalities could benefit multimodal fusion. To this end, we consider leveraging the coupled interactions in the feature level and modality level for diagnosis in this paper. First, we propose capturing the feature-level coupled interaction using a coupled feature representation. Then, to model the modality-level coupled interaction, we present two novel methods: 1) the coupled boosting (CB) that models the correlation of pairwise coupled-diversity on both inconsistently and incorrectly classified samples between different modalities and 2) the coupled metric ensemble (CME) that learns an informative feature projection from different modalities by integrating the intrarelation and interrelation of training samples. We systematically evaluated our methods with the AD neuroimaging initiative data set. By comparison with the baseline learning-based methods and the state-of-the-art methods that are specially developed for AD/MCI (mild cognitive impairment) diagnosis, our methods achieved the best performance with accuracy of 95.0% and 80.7% (CB), 94.9% and 79.9% (CME) for AD/NC (normal control), and MCI/NC identification, respectively.

Adaptive Sample-level Graph Combination for Partial Multiview Clustering

Multiview clustering explores complementary information among distinct views to enhance clustering performance under the assumption that all samples have complete information in all available views. However, this assumption does not hold in many real applications, where the information of some samples in one or more views may be missing, leading to partial multiview clustering problems. In this case, significant performance degeneration is usually observed. A collection of partial multiview clustering algorithms has been proposed to address this issue and most treat all different views equally during clustering. In fact, because different views provide features collected from different angles/feature spaces, they might play different roles in the clustering process. With the diversity of different views considered, in this study, a novel adaptive method is proposed for partial multiview clustering by automatically adjusting the contributions of different views. The samples are divided into complete and incomplete sets, while a joint learning mechanism is established to facilitate the connection between them and thereby improve clustering performance. More specifically, the method is characterized by a joint optimization model comprising two terms. The first term mines the underlying cluster structure from both complete and incomplete samples by adaptively updating their importance in all available views. The second term is designed to group all data with the aid of the cluster structure modeled in the first term. These two terms seamlessly integrate the complementary information among multiple views and enhance the performance of partial multiview clustering. Experimental results on real-world datasets illustrate the effectiveness and efficiency of our proposed method.

A Fast and Accurate Matrix Completion Method Based on QR Decomposition and L2,1 -Norm Minimization

Low-rank matrix completion aims to recover matrices with missing entries and has attracted considerable attention from machine learning researchers. Most of the existing methods, such as weighted nuclear-norm-minimization-based methods and Qatar Riyal (QR)-decomposition-based methods, cannot provide both convergence accuracy and convergence speed. To investigate a fast and accurate completion method, an iterative QR-decomposition-based method is proposed for computing an approximate singular value decomposition. This method can compute the largest singular values of a matrix by iterative QR decomposition. Then, under the framework of matrix trifactorization, a method for computing an approximate SVD based on QR decomposition (CSVD-QR)-based L_2,1 -norm minimization method (LNM-QR) is proposed for fast matrix completion. Theoretical analysis shows that this QR-decomposition-based method can obtain the same optimal solution as a nuclear norm minimization method, i.e., the L_2,1 -norm of a submatrix can converge to its nuclear norm. Consequently, an LNM-QR-based iteratively reweighted L_2,1 -norm minimization method (IRLNM-QR) is proposed to improve the accuracy of LNM-QR. Theoretical analysis shows that IRLNM-QR is as accurate as an iteratively reweighted nuclear norm minimization method, which is much more accurate than the traditional QR-decomposition-based matrix completion methods. Experimental results obtained on both synthetic and real-world visual data sets show that our methods are much faster and more accurate than the state-of-the-art methods.