Robust and Sparse Linear Discriminant Analysis via an Alternating Direction Method of Multipliers

Robust Supervised Spline Embedding

High-dimensional data present significant challenges such as inadequate sample size, abundance of noise, and the curse of dimensionality, which make many traditional classification algorithms inapplicable. To provide valid inference for such data, it requires finding a noise-free low-dimensional representation that preserves both the underlying manifold structure and discriminative information. However, the existing methods often fail to take full consideration of these requirements. In this article, we introduce a robust supervised spline embedding (RS2E) algorithm for high-dimensional classification. The proposed approach is highlighted in four aspects: 1) it preserves the class-aware submanifold structure in the thin plate spline embedding space; 2) it eliminates noise and outliers to recover the clean manifold by exploiting its intrinsic low complexity; 3) it separates the class-aware submanifolds by maximizing the distance between each data point and the marginal data points of other class-aware submanifolds; and 4) it applies the alternating direction method of multipliers with generalized power iteration to solve the objective function. Promising experimental results on the real-world, generative adversarial network (GAN)-generated and artificially corrupted datasets demonstrate that RS2E outperforms other supervised dimensionality reduction algorithms in terms of classification accuracy.

Ensemble learning based on matrix completion improves microbe-disease association prediction

Microbes have a profound impact on human health. Identifying disease-associated microbes would provide helpful guidance for drug development and disease treatment. Through an enormous experimental effort, limited disease-associated microbes have been determined. Accurate computational approaches are needed to predict potential microbe-disease associations for biomedical screening. In this study, we present an ensemble learning framework entitled SABMDA to improve microbe-disease association inference. We first integrate multi-source of information from both microbes and diseases, and develop two matrix completion algorithms to predict microbe-disease associations successively. Ablation tests show combining the two matrix completion algorithms can receive better prediction performance. Moreover, comprehensive experiments, including cross-validations and independent test, demonstrate that SABMDA outperforms seven recent baseline methods significantly. Finally, we apply SABMDA to three diseases to predict their associated microbes, and results show SABMDA's remarkable prediction ability in real situations.

Toward Robust Discriminative Projections Learning Against Adversarial Patch Attacks

As one of the most popular supervised dimensionality reduction methods, linear discriminant analysis (LDA) has been widely studied in machine learning community and applied to many scientific applications. Traditional LDA minimizes the ratio of squared norms, which is vulnerable to the adversarial examples. In recent studies, many -norm-based robust dimensionality reduction methods are proposed to improve the robustness of model. However, due to the difficulty of -norm ratio optimization and weakness on defending a large number of adversarial examples, so far, scarce works have been proposed to utilize sparsity-inducing norms for LDA objective. In this article, we propose a novel robust discriminative projections learning (rDPL) method based on the -norm trace-ratio minimization optimization algorithm. Minimizing the -norm ratio problem directly is a much more challenging problem than the traditional methods, and there is no existing optimization algorithm to solve such nonsmooth terms ratio problem. We derive a new efficient algorithm to solve this challenging problem and provide a theoretical analysis on the convergence of our algorithm. The proposed algorithm is easy to implement and converges fast in practice. Extensive experiments on both synthetic data and several real benchmark datasets show the effectiveness of the proposed method on defending the adversarial patch attack by comparison with many state-of-the-art robust dimensionality reduction methods.

On the Equivalence of Linear Discriminant Analysis and Least Squares Regression

Studying the relationship between linear discriminant analysis (LDA) and least squares regression (LSR) is of great theoretical and practical significance. It is well-known that the two-class LDA is equivalent to an LSR problem, and directly casting multiclass LDA as an LSR problem, however, becomes more challenging. Recent study reveals that the equivalence between multiclass LDA and LSR can be established based on a special class indicator matrix, but under a mild condition which may not hold under the scenarios with low-dimensional or oversampled data. In this article, we show that the equivalence between multiclass LDA and LSR can be established based on arbitrary linearly independent class indicator vectors and without any condition. In addition, we show that LDA is also equivalent to a constrained LSR based on the data-dependent indicator vectors. It can be concluded that under exactly the same mild condition, such two regressions are both equivalent to the null space LDA method. Illuminated by the equivalence of LDA and LSR, we propose a direct LDA classifier to replace the conventional framework of LDA plus extra classifier. Extensive experiments well validate the above theoretic analysis.

Spatially aware self-representation learning for tissue structure characterization and spatial functional genes identification

Spatially resolved transcriptomics (SRT) enable the comprehensive characterization of transcriptomic profiles in the context of tissue microenvironments. Unveiling spatial transcriptional heterogeneity needs to effectively incorporate spatial information accounting for the substantial spatial correlation of expression measurements. Here, we develop a computational method, SpaSRL (spatially aware self-representation learning), which flexibly enhances and decodes spatial transcriptional signals to simultaneously achieve spatial domain detection and spatial functional genes identification. This novel tunable spatially aware strategy of SpaSRL not only balances spatial and transcriptional coherence for the two tasks, but also can transfer spatial correlation constraint between them based on a unified model. In addition, this joint analysis by SpaSRL deciphers accurate and fine-grained tissue structures and ensures the effective extraction of biologically informative genes underlying spatial architecture. We verified the superiority of SpaSRL on spatial domain detection, spatial functional genes identification and data denoising using multiple SRT datasets obtained by different platforms and tissue sections. Our results illustrate SpaSRL's utility in flexible integration of spatial information and novel discovery of biological insights from spatial transcriptomic datasets.

BMPMDA: Prediction of MiRNA-Disease Associations Using a Space Projection Model Based on Block Matrix

With the high-quality development of bioinformatics technology, miRNA-disease associations (MDAs) are gradually being uncovered. At present, convenient and efficient prediction methods, which solve the problem of resource-consuming in traditional wet experiments, need to be further put forward. In this study, a space projection model based on block matrix is presented for predicting MDAs (BMPMDA). Specifically, two block matrices are first composed of the known association matrix and similarity to increase comprehensiveness. For the integrity of information in the heterogeneous network, matrix completion (MC) is utilized to mine potential MDAs. Considering the neighborhood information of data points, linear neighborhood similarity (LNS) is regarded as a measure of similarity. Next, LNS is projected onto the corresponding completed association matrix to derive the projection score. Finally, the AUC and AUPR values for BMPMDA reach 0.9691 and 0.6231, respectively. Additionally, the majority of novel MDAs in three disease cases are identified in existing databases and literature. It suggests that BMPMDA can serve as a reliable prediction model for biological research.

Ratio Sum Versus Sum Ratio for Linear Discriminant Analysis

Dimension reduction is a critical technology for high-dimensional data processing, where Linear Discriminant Analysis (LDA) and its variants are effective supervised methods. However, LDA prefers to feature with smaller variance, which causes feature with weak discriminative ability retained. In this paper, we propose a novel Ratio Sum for Linear Discriminant Analysis (RSLDA), which aims at maximizing discriminative ability of each feature in subspace. To be specific, it maximizes the sum of ratio of the between-class distance to the within-class distance in each dimension of subspace. Since the original RSLDA problem is difficult to obtain the closed solution, an equivalent problem is developed which can be solved by an alternative optimization algorithm. For solving the equivalent problem, it is transformed into two sub-problems, one of which can be solved directly, the other is changed into a convex optimization problem, where singular value decomposition is employed instead of matrix inversion. Consequently, performance of algorithm cannot be affected by the non-singularity of covariance matrix. Furthermore, Kernel RSLDA (KRSLDA) is presented to improve the robustness of RSLDA. Additionally, time complexity of RSLDA and KRSLDA are analyzed. Extensive experiments show that RSLDA and KRSLDA outperforms other comparison methods on toy datasets and multiple public datasets.

Joint Sparse Locality-Aware Regression for Robust Discriminative Learning

With the dramatic increase of dimensions in the data representation, extracting latent low-dimensional features becomes of the utmost importance for efficient classification. Aiming at the problems of weakly discriminating marginal representation and difficulty in revealing the data manifold structure in most of the existing linear discriminant methods, we propose a more powerful discriminant feature extraction framework, namely, joint sparse locality-aware regression (JSLAR). In our model, we formulate a new strategy induced by the nonsquared L₂ norm for enhancing the local intraclass compactness of the data manifold, which can achieve the joint learning of the locality-aware graph structure and the desirable projection matrix. Besides, we formulate a weighted retargeted regression to perform the marginal representation learning adaptively instead of using the general average interclass margin. To alleviate the disturbance of outliers and prevent overfitting, we measure the regression term and locality-aware term together with the regularization term by forcing the row sparsity with the joint L_2,1 norms. Then, we derive an effective iterative algorithm for solving the proposed model. The experimental results over a range of benchmark databases demonstrate that the proposed JSLAR outperforms some state-of-the-art approaches.

Fast Unsupervised Projection for Large-Scale Data

Dimensionality reduction (DR) technique has been frequently used to alleviate information redundancy and reduce computational complexity. Traditional DR methods generally are inability to deal with nonlinear data and have high computational complexity. To cope with the problems, we propose a fast unsupervised projection (FUP) method. The simplified graph of FUP is constructed by samples and representative points, where the number of the representative points selected through iterative optimization is less than that of samples. By generating the presented graph, it is proved that large-scale data can be projected faster in numerous scenarios. Thereafter, the orthogonality FUP (OFUP) method is proposed to ensure the orthogonality of projection matrix. Specifically, the OFUP method is proved to be equivalent to PCA upon certain parameter setting. Experimental results on benchmark data sets show the effectiveness in retaining the essential information.

Multiperspective Progressive Structure Adaptation for JPEG Steganography Detection Across Domains

The aim of steganography detection is to identify whether the multimedia data contain hidden information. Although many detection algorithms have been presented to solve tasks with inconsistent distributions between the source and target domains, effectively exploiting transferable correlation information across domains remains challenging. As a solution, we present a novel multiperspective progressive structure adaptation (MPSA) scheme based on active progressive learning (APL) for JPEG steganography detection across domains. First, the source and target data originating from unprocessed steganalysis features are clustered together to explore the structures in different domains, where the intradomain and interdomain structures can be captured to provide adequate information for cross-domain steganography detection. Second, the structure vectors containing the global and local modalities are exploited to reduce nonlinear distribution discrepancy based on APL in the latent representation space. In this way, the signal-to-noise ratio (SNR) of a weak stego signal can be improved by selecting suitable objects and adjusting the learning sequence. Third, the structure adaptation across multiple domains is achieved by the constraints for iterative optimization to promote the discrimination and transferability of structure knowledge. In addition, a unified framework for single-source domain adaptation (SSDA) and multiple-source domain adaptation (MSDA) in mismatched steganalysis can enhance the model's capability to avoid a potential negative transfer. Extensive experiments on various benchmark cross-domain steganography detection tasks show the superiority of the proposed approach over the state-of-the-art methods.

An Integrated Counterfactual Sample Generation and Filtering Approach for SAR Automatic Target Recognition with a Small Sample Set

Although automatic target recognition (ATR) models based on data-driven algorithms have achieved excellent performance in recent years, the synthetic aperture radar (SAR) ATR model often suffered from performance degradation when it encountered a small sample set. In this paper, an integrated counterfactual sample generation and filtering approach is proposed to alleviate the negative influence of a small sample set. The proposed method consists of a generation component and a filtering component. First, the proposed generation component utilizes the overfitting characteristics of generative adversarial networks (GANs), which ensures the generation of counterfactual target samples. Second, the proposed filtering component is built by learning different recognition functions. In the proposed filtering component, multiple SVMs trained by different SAR target sample sets provide pseudo-labels to the other SVMs to improve the recognition rate. Then, the proposed approach improves the performance of the recognition model dynamically while it continuously generates counterfactual target samples. At the same time, counterfactual target samples that are beneficial to the ATR model are also filtered. Moreover, ablation experiments demonstrate the effectiveness of the various components of the proposed method. Experimental results based on the Moving and Stationary Target Acquisition and Recognition (MSTAR) and OpenSARship dataset also show the advantages of the proposed approach. Even though the size of the constructed training set was 14.5% of the original training set, the recognition performance of the ATR model reached 91.27% with the proposed approach.

General Plane-Based Clustering With Distribution Loss

In this article, we propose a general model for plane-based clustering. The general model reveals the relationship between cluster assignment and cluster updating during clustering implementation, and it contains many existing plane-based clustering methods, e.g., k-plane clustering, proximal plane clustering, twin support vector clustering, and their extensions. Under this general model, one may obtain an appropriate clustering method for a specific purpose. The general model is a procedure corresponding to an optimization problem, which minimizes the total loss of the samples. Thereinto, the loss of a sample derives from both within-cluster and between-cluster information. We discuss the theoretical termination conditions and prove that the general model terminates in a finite number of steps at a local or weak local solution. Furthermore, we propose a distribution loss function that fluctuates with the input data and introduce it into the general model to obtain a plane-based clustering method (DPC). DPC can capture the data distribution precisely because of its statistical characteristics, and its termination that finitely terminates at a weak local solution is given immediately based on the general model. The experimental results show that our DPC outperforms the state-of-the-art plane-based clustering methods on many synthetic and benchmark data sets.

Imbalance Fault Classification Based on VMD Denoising and S-LDA for Variable-Speed Marine Current Turbine

Marine current energy as a kind of renewable energy has gradually attracted more and more attention from many countries. However, the blade imbalance fault of marine current turbines (MCTs) will have an effect on the power production efficiency and cause damage to the MCT system. It is hard to classify the severity of an MCT blade imbalance fault under the condition of the current instability and seafloor noise. This paper proposes a fault classification method based on the combination of variational mode decomposition denoising (VMD denoising) and screening linear discriminant analysis (S-LDA). The proposed method consists of three parts. Firstly, phase demodulation of the collected stator current signal is performed by the Hilbert transform (HT) method. Then, the obtained demodulation signal is denoised by variational mode decomposition denoising (VMD denoising), and the denoised signal is analyzed by power spectral density (PSD). Finally, S-LDA is employed on the power signal to determine the severities of fault classification. The effectiveness of the proposed method is verified by experimental results under different severities of blade imbalance fault. The stator current signatures of experiments with different severities of blade imbalance fault are used to validate the effectiveness of the proposed method. The fault classification accuracy is 92.04% based on the proposed method. Moreover, the experimental results verify that the influence of velocity fluctuation on fault classification can be eliminated.

Knowledge-Driven Deep Unrolling for Robust Image Layer Separation

Single-image layer separation targets to decompose the observed image into two independent components in terms of different application demands. It is known that many vision and multimedia applications can be (re)formulated as a separation problem. Due to the fundamentally ill-posed natural of these separations, existing methods are inclined to investigate model priors on the separated components elaborately. Nevertheless, it is knotty to optimize the cost function with complicated model regularizations. Effectiveness is greatly conceded by the settled iteration mechanism, and the adaption cannot be guaranteed due to the poor data fitting. What is more, for a universal framework, the most taxing point is that one type of visual cue cannot be shared with different tasks. To partly overcome the weaknesses mentioned earlier, we delve into a generic optimization unrolling technique to incorporate deep architectures into iterations for adaptive image layer separation. First, we propose a general energy model with implicit priors, which is based on maximum a posterior, and employ the extensively accepted alternating direction method of multiplier to determine our elementary iteration mechanism. By unrolling with one general residual architecture prior and one task-specific prior, we attain a straightforward, flexible, and data-dependent image separation framework successfully. We apply our method to four different tasks, including single-image-rain streak removal, high-dynamic-range tone mapping, low-light image enhancement, and single-image reflection removal. Extensive experiments demonstrate that the proposed method is applicable to multiple tasks and outperforms the state of the arts by a large margin qualitatively and quantitatively.