Adaptive Weighted Sparse Principal Component Analysis for Robust Unsupervised Feature Selection

Robust generalized PCA for enhancing discriminability and recoverability

The dependency of low-dimensional embedding to principal component space seriously limits the effectiveness of existing robust principal component analysis (PCA) algorithms. Simply projecting the original sample coordinates onto orthogonal principal component directions may not effectively address various noise-corrupted scenarios, impairing both discriminability and recoverability. Our method addresses this issue through a generalized PCA (GPCA), which optimizes regression bias rather than sample mean, leading to more adaptable properties. And, we propose a robust GPCA model with joint loss and regularization based on the ℓ2,μ norm and ℓ2,ν norms, respectively. This approach not only mitigates sensitivity to outliers but also enhances feature extraction and selection flexibility. Additionally, we introduce a truncated and reweighted loss strategy, where truncation eliminates severely deviated outliers, and reweighting prioritizes the remaining samples. These innovations collectively improve the GPCA model's performance. To solve the proposed model, we propose a non-greedy iterative algorithm and theoretically guarantee the convergence. Experimental results demonstrate that the proposed GPCA model outperforms the previous robust PCA models in both recoverability and discrimination.

Discriminative and Robust Autoencoders for Unsupervised Feature Selection

Many recent research works on unsupervised feature selection (UFS) have focused on how to exploit autoencoders (AEs) to seek informative features. However, existing methods typically employ the squared error to estimate the data reconstruction, which amplifies the negative effect of outliers and can lead to performance degradation. Moreover, traditional AEs aim to extract latent features that capture intrinsic information of the data for accurate data recovery. Without incorporating explicit cluster structure-detecting objectives into the training criterion, AEs fail to capture the latent cluster structure of the data which is essential for identifying discriminative features. Thus, the selected features lack strong discriminative power. To address the issues, we propose to jointly perform robust feature selection and -means clustering in a unified framework. Concretely, we exploit an AE with a -norm as a basic model to seek informative features. To improve robustness against outliers, we introduce an adaptive weight vector for the data reconstruction terms of AE, which assigns smaller weights to the data with larger errors to automatically reduce the influence of the outliers, and larger weights to the data with smaller errors to strengthen the influence of clean data. To enhance the discriminative power of the selected features, we incorporate -means clustering into the representation learning of the AE. This allows the AE to continually explore cluster structure information, which can be used to discover more discriminative features. Then, we also present an efficient approach to solve the objective of the corresponding problem. Extensive experiments on various benchmark datasets are provided, which clearly demonstrate that the proposed method outperforms state-of-the-art methods.

Informative Data Selection With Uncertainty for Multimodal Object Detection

Noise has always been nonnegligible trouble in object detection by creating confusion in model reasoning, thereby reducing the informativeness of the data. It can lead to inaccurate recognition due to the shift in the observed pattern, that requires a robust generalization of the models. To implement a general vision model, we need to develop deep learning models that can adaptively select valid information from multimodal data. This is mainly based on two reasons. Multimodal learning can break through the inherent defects of single-modal data, and adaptive information selection can reduce chaos in multimodal data. To tackle this problem, we propose a universal uncertainty-aware multimodal fusion model. It adopts a multipipeline loosely coupled architecture to combine the features and results from point clouds and images. To quantify the correlation in multimodal information, we model the uncertainty, as the inverse of data information, in different modalities and embed it in the bounding box generation. In this way, our model reduces the randomness in fusion and generates reliable output. Moreover, we conducted a completed investigation on the KITTI 2-D object detection dataset and its derived dirty data. Our fusion model is proven to resist severe noise interference like Gaussian, motion blur, and frost, with only slight degradation. The experiment results demonstrate the benefits of our adaptive fusion. Our analysis on the robustness of multimodal fusion will provide further insights for future research.

Robust Principal Component Analysis via Joint Reconstruction and Projection

Principal component analysis (PCA) is one of the most widely used unsupervised dimensionality reduction algorithms, but it is very sensitive to outliers because the squared l2 -norm is used as distance metric. Recently, many scholars have devoted themselves to solving this difficulty. They learn the projection matrix from minimum reconstruction error or maximum projection variance as the starting point, which leads them to ignore a serious problem, that is, the original PCA learns the projection matrix by minimizing the reconstruction error and maximizing the projection variance simultaneously, but they only consider one of them, which imposes various limitations on the performance of model. To solve this problem, we propose a novel robust principal component analysis via joint reconstruction and projection, namely, RPCA-RP, which combines reconstruction error and projection variance to fully mine the potential information of data. Furthermore, we carefully design a discrete weight for model to implicitly distinguish between normal data and outliers, so as to easily remove outliers and improve the robustness of method. In addition, we also unexpectedly discovered that our method has anomaly detection capabilities. Subsequently, an effective iterative algorithm is explored to solve this problem and perform related theoretical analysis. Extensive experimental results on several real-world datasets and RGB large-scale dataset demonstrate the superiority of our method.

Generalized and Robust Least Squares Regression

As a simple yet effective method, least squares regression (LSR) is extensively applied for data regression and classification. Combined with sparse representation, LSR can be extended to feature selection (FS) as well, in which l1 regularization is often applied in embedded FS algorithms. However, because the loss function is in the form of squared error, LSR and its variants are sensitive to noises, which significantly degrades the effectiveness and performance of classification and FS. To cope with the problem, we propose a generalized and robust LSR (GRLSR) for classification and FS, which is made up of arbitrary concave loss function and the l2,p -norm regularization term. Meanwhile, an iterative algorithm is applied to efficiently deal with the nonconvex minimization problem, in which an additional weight to suppress the effect of noises is added to each data point. The weights can be automatically assigned according to the error of the samples. When the error is large, the value of the corresponding weight is small. It is this mechanism that allows GRLSR to reduce the impact of noises and outliers. According to the different formulations of the concave loss function, four specific methods are proposed to clarify the essence of the framework. Comprehensive experiments on corrupted datasets have proven the advantage of the proposed method.

Exposure of Helicobacter pylori to clarithromycin in vitro resulting in the development of resistance and triggers metabolic reprogramming associated with virulence and pathogenicity

In H. pylori infection, antibiotic-resistance is one of the most common causes of treatment failure. Bacterial metabolic activities, such as energy production, bacterial growth, cell wall construction, and cell-cell communication, all play important roles in antimicrobial resistance mechanisms. Identification of microbial metabolites may result in the discovery of novel antimicrobial therapeutic targets and treatments. The purpose of this work is to assess H. pylori metabolomic reprogramming in order to reveal the underlying mechanisms associated with the development of clarithromycin resistance. Previously, four H. pylori isolates were induced to become resistant to clarithromycin in vitro by incrementally increasing the concentrations of clarithromycin. Bacterial metabolites were extracted using the Bligh and Dyer technique and analyzed using metabolomic fingerprinting based on Liquid Chromatography Quadrupole Time-of-Flight Mass Spectrometry (LC-Q-ToF-MS). The data was processed and analyzed using the MassHunter Qualitative Analysis and Mass Profiler Professional software. In parental sensitivity (S), breakpoint isolates (B), and induced resistance isolates (R) H. pylori isolates, 982 metabolites were found. Furthermore, based on accurate mass, isotope ratios, abundances, and spacing, 292 metabolites matched the metabolites in the Agilent METLIN precise Mass-Personal Metabolite Database and Library (AM-PCDL). Several metabolites associated with bacterial virulence, pathogenicity, survival, and proliferation (L-leucine, Pyridoxone [Vitamine B6], D-Mannitol, Sphingolipids, Indoleacrylic acid, Dulcitol, and D-Proline) were found to be elevated in generated resistant H. pylori isolates when compared to parental sensitive isolates. The elevated metabolites could be part of antibiotics resistance mechanisms. Understanding the fundamental metabolome changes in the course of progressing from clarithromycin-sensitive to breakpoint to resistant in H. pylori clinical isolates may be a promising strategy for discovering novel alternatives therapeutic targets.

Discrete Robust Principal Component Analysis via Binary Weights Self-Learning

Principal component analysis (PCA) is a typical unsupervised dimensionality reduction algorithm, and one of its important weaknesses is that the squared l2 -norm cannot overcome the influence of outliers. Existing robust PCA methods based on paradigm have the following two drawbacks. First, the objective function of PCA based on the l1 -norm has no rotational invariance and limited robustness to outliers, and its solution mostly uses a greedy search strategy, which is expensive. Second, the robust PCA based on the l2,1 -norm and the l2,p -norm is essential to learn probability weights for data, which only weakens the influence of outliers on the learning projection matrix and cannot be completely eliminated. Moreover, the ability to detect anomalies is also very poor. To solve these problems, we propose a novel discrete robust principal component analysis (DRPCA). Through self-learning binary weights, the influence of outliers on the projection matrix and data center estimation can be completely eliminated, and anomaly detection can be directly performed. In addition, an alternating iterative optimization algorithm is designed to solve the proposed problem and realize the automatic update of binary weights. Finally, our proposed model is successfully applied to anomaly detection applications, and experimental results demonstrate that the superiority of our proposed method compared with the state-of-the-art methods.

Sparse discriminant PCA based on contrastive learning and class-specificity distribution

Much mathematical effort has been devoted to developing Principal Component Analysis (PCA), which is the most popular feature extraction method. To suppress the negative effect of noise on PCA performance, there have been extensive studies and applications of a large number of robust PCAs achieving outstanding results. However, existing methods suffer from at least two shortcomings: (1) They expressed PCA as a reconstruction model measured by Euclidean distance, which only considers the relationship between the data and its reconstruction and ignores the differences between different data points; (2) They did not consider the class-specificity distribution information contained in the data itself, thus lacking discriminative properties. To overcome the above problems, we propose a Sparse Discriminant Principal Components Analysis (SDPCA) model based on contrastive learning and class-specificity distribution. Specifically, we use contrastive learning to measure the relationship between samples and their reconstructions, which fully takes the discriminative information between data into account in PCA. In order to make the extracted low-dimensional features profoundly reflect the class-specificity distribution of the data, we minimize the squared ℓ1,2-norm of the low-dimensional embedding. In addition, to reduce the effects of redundant features and noise and to improve the interpretability of PCA at the same time, we impose sparsity constraints on the projection matrix using the squared ℓ1,2-norm. Our experimental results on different types of benchmark databases demonstrate that our model has state-of-the-art performance.

Soft Subspace Based Ensemble Clustering for Multivariate Time Series Data

Recently, multivariate time series (MTS) clustering has gained lots of attention. However, state-of-the-art algorithms suffer from two major issues. First, few existing studies consider correlations and redundancies between variables of MTS data. Second, since different clusters usually exist in different intrinsic variables, how to efficiently enhance the performance by mining the intrinsic variables of a cluster is challenging work. To deal with these issues, we first propose a variable-weighted K-medoids clustering algorithm (VWKM) based on the importance of a variable for a cluster. In VWKM, the proposed variable weighting scheme could identify the important variables for a cluster, which can also provide knowledge and experience to related experts. Then, a Reverse nearest neighborhood-based density Peaks approach (RP) is proposed to handle the problem of initialization sensitivity of VWKM. Next, based on VWKM and the density peaks approach, an ensemble Clustering framework (SSEC) is advanced to further enhance the clustering performance. Experimental results on ten MTS datasets show that our method works well on MTS datasets and outperforms the state-of-the-art clustering ensemble approaches.

Two-Dimensional Unsupervised Feature Selection via Sparse Feature Filter

Unsupervised feature selection is a vital yet challenging topic for effective data learning. Recently, 2-D feature selection methods show good performance on image analysis by utilizing the structure information of image. Current 2-D methods usually adopt a sparse regularization to spotlight the key features. However, such scheme introduces additional hyperparameter needed for pruning, limiting the applicability of unsupervised algorithms. To overcome these challenges, we design a feature filter to estimate the weight of image features for unsupervised feature selection. Theoretical analysis shows that a sparse regularization can be derived from the feature filter by transformation, indicating that the filter plays the same role as the popular sparse regularization does. We deploy two distinct strategies in terms of feature selection, called multiple feature filters and single common feature filter. The former divides the optimization problem into multiple independent subproblems and selects features that meet the respective interests of each subproblem. The latter selects features that are in the interest of the overall optimization problem. Extensive experiments on seven benchmark datasets show that our unsupervised 2-D weight-based feature selection methods achieve superior performance over the state-of-the-art methods.

Identifying Cancer Driver Pathways Based on the Mouth Brooding Fish Algorithm

Identifying the driver genes of cancer progression is of great significance in improving our understanding of the causes of cancer and promoting the development of personalized treatment. In this paper, we identify the driver genes at the pathway level via an existing intelligent optimization algorithm, named the Mouth Brooding Fish (MBF) algorithm. Many methods based on the maximum weight submatrix model to identify driver pathways attach equal importance to coverage and exclusivity and assign them equal weight, but those methods ignore the impact of mutational heterogeneity. Here, we use principal component analysis (PCA) to incorporate covariate data to reduce the complexity of the algorithm and construct a maximum weight submatrix model considering different weights of coverage and exclusivity. Using this strategy, the unfavorable effect of mutational heterogeneity is overcome to some extent. Data involving lung adenocarcinoma and glioblastoma multiforme were tested with this method and the results compared with the MDPFinder, Dendrix, and Mutex methods. When the driver pathway size was 10, the recognition accuracy of the MBF method reached 80% in both datasets, and the weight values of the submatrix were 1.7 and 1.89, respectively, which are better than those of the compared methods. At the same time, in the signal pathway enrichment analysis, the important role of the driver genes identified by our MBF method in the cancer signaling pathway is revealed, and the validity of these driver genes is demonstrated from the perspective of their biological effects.

Sparse PCA via l2,p-Norm Regularization for Unsupervised Feature Selection

In the field of data mining, how to deal with high-dimensional data is an inevitable topic. Since it does not rely on labels, unsupervised feature selection has attracted a lot of attention. The performance of spectral-based unsupervised methods depends on the quality of the constructed similarity matrix, which is used to depict the intrinsic structure of data. However, real-world data often contain plenty of noise features, making the similarity matrix constructed by original data cannot be completely reliable. Worse still, the size of a similarity matrix expands rapidly as the number of samples rises, making the computational cost increase significantly. To solve this problem, a simple and efficient unsupervised model is proposed to perform feature selection. We formulate PCA as a reconstruction error minimization problem, and incorporate a l_2,p-norm regularization term to make the projection matrix sparse. The learned row-sparse and orthogonal projection matrix is used to select discriminative features. Then, we present an efficient optimization algorithm to solve the proposed unsupervised model, and analyse the convergence and computational complexity of the algorithm theoretically. Finally, experiments on both synthetic and real-world data sets demonstrate the effectiveness of our proposed method.

Face Recognition Method under Adaptive Image Matching and Dictionary Learning Algorithm

In this research, a robust face recognition method based on adaptive image matching and a dictionary learning algorithm was proposed. A Fisher discriminant constraint was introduced into the dictionary learning algorithm program so that the dictionary had certain category discrimination ability. The purpose was to use this technology to reduce the influence of pollution, absence, and other factors on face recognition and improve the recognition rate. The optimization method was used to solve the loop iteration to obtain the expected specific dictionary, and the selected specific dictionary was used as the representation dictionary in adaptive sparse representation. In addition, if a specific dictionary was placed in a seed space of the original training data, the mapping matrix can be used to represent the mapping relationship between the specific dictionary and the original training sample, and the test sample could be corrected according to the mapping matrix to remove the contamination in the test sample. Moreover, the feature face method and dimension reduction method were used to process the specific dictionary and the corrected test sample, and the dimensions were reduced to 25, 50, 75, 100, 125, and 150, respectively. In this research, the recognition rate of the algorithm in 50 dimensions was lower than that of the discriminatory low-rank representation method (DLRR), and the recognition rate in other dimensions was the highest. The adaptive image matching classifier was used for classification and recognition. The experimental results showed that the proposed algorithm had a good recognition rate and good robustness against noise, pollution, and occlusion. Health condition prediction based on face recognition technology has the advantages of being noninvasive and convenient operation.

Classifier Ensemble Based on Multiview Optimization for High-Dimensional Imbalanced Data Classification

High-dimensional class imbalanced data have plagued the performance of classification algorithms seriously. Because of a large number of redundant/invalid features and the class imbalanced issue, it is difficult to construct an optimal classifier for high-dimensional imbalanced data. Classifier ensemble has attracted intensive attention since it can achieve better performance than an individual classifier. In this work, we propose a multiview optimization (MVO) to learn more effective and robust features from high-dimensional imbalanced data, based on which an accurate and robust ensemble system is designed. Specifically, an optimized subview generation (OSG) in MVO is first proposed to generate multiple optimized subviews from different scenarios, which can strengthen the classification ability of features and increase the diversity of ensemble members simultaneously. Second, a new evaluation criterion that considers the distribution of data in each optimized subview is developed based on which a selective ensemble of optimized subviews (SEOS) is designed to perform the subview selective ensemble. Finally, an oversampling approach is executed on the optimized view to obtain a new class rebalanced subset for the classifier. Experimental results on 25 high-dimensional class imbalanced datasets indicate that the proposed method outperforms other mainstream classifier ensemble methods.

Factor analysis, sparse PCA, and Sum of Ranking Differences-based improvements of the Promethee-GAIA multicriteria decision support technique

The Promethee-GAIA method is a multicriteria decision support technique that defines the aggregated ranks of multiple criteria and visualizes them based on Principal Component Analysis (PCA). In the case of numerous criteria, the PCA biplot-based visualization do not perceive how a criterion influences the decision problem. The central question is how the Promethee-GAIA-based decision-making process can be improved to gain more interpretable results that reveal more characteristic inner relationships between the criteria. To improve the Promethee-GAIA method, we suggest three techniques that eliminate redundant criteria as well as clearly outline, which criterion belongs to which factor and explore the similarities between criteria. These methods are the following: A) Principal factoring with rotation and communality analysis (P-PFA), B) the integration of Sparse PCA into the Promethee II method (P-sPCA), and C) the Sum of Ranking Differences method (P-SRD). The suggested methods are presented through an I4.0+ dataset that measures the Industry 4.0 readiness of NUTS 2-classified regions. The proposed methods are useful tools for handling multicriteria ranking problems, if the number of criteria is numerous.