Semi-Supervised Graph Regularized Deep NMF With Bi-Orthogonal Constraints for Data Representation

Semi-supervised non-negative matrix factorization with structure preserving for image clustering

Semi-supervised learning methods have wide applications thanks to the reasonable utilization for a part of label information of data. In recent years, non-negative matrix factorization (NMF) has received considerable attention because of its interpretability and practicality. Based on the advantages of semi-supervised learning and NMF, many semi-supervised NMF methods have been presented. However, these existing semi-supervised NMF methods construct a label matrix only containing elements 1 and 0 to represent the labeled data and further construct a label regularization, which neglects an intrinsic structure of NMF. To address the deficiency, in this paper, we propose a novel semi-supervised NMF method with structure preserving. Specifically, we first construct a new label matrix with weights and further construct a label constraint regularizer to both utilize the label information and maintain the intrinsic structure of NMF. Then, based on the label constraint regularizer, the basis images of labeled data are extracted for monitoring and modifying the basis images learning of all data by establishing a basis regularizer. Finally, incorporating the label constraint regularizer and the basis regularizer into NMF, we propose a new semi-supervised NMF method. To solve the optimization problem, a multiplicative updating algorithm is developed. The proposed method is applied to image clustering to test its performance. Experimental results on eight data sets demonstrate the effectiveness of the proposed method in contrast with state-of-the-art unsupervised and semi-supervised algorithms.

DeepOmicsSurv: a deep learning-based model for survival prediction of oral cancer

OBJECTIVE

Oral cancer is an important health challenge worldwide and accurate survival time prediction of this disease can guide treatment decisions. This study aims to propose a deep learning-based model, DeepOmicsSurv, to predict survival in oral cancer patients using clinical and multi-omics data.

METHODS

DeepOmicsSurv builds on the DeepSurv model, incorporating multi-head attention convolutional layers, dropout, pooling, and batch normalization to boost its strength and precision. Various dimensionality reduction techniques, including Principal Component Analysis (PCA), Kernel PCA, Non-Negative Matrix Factorization (NMF), Singular Value Decomposition (SVD), Partial Least Squares (PLS), Multidimensional Scaling (MDS), and Autoencoders, were employed to manage the high-dimensional omics data. The model's performance was evaluated against DeepSurv, DeepHit, Cox Proportional Hazards (CoxPH), Convolutional Neural Networks (CNN), and Recurrent Neural Networks (RNN). Additionally, SHapley Additive Explanations (SHAP) was used to analyze the impact of clinical features on survival predictions.

RESULTS

DeepOmicsSurv achieved a C-index of 0.966, MSE of 0.0138, RMSE of 0.1174, MAE of 0.0795, and MedAE of 0.0515, outperforming other deep learning models. Among various dimensionality reduction techniques, autoencoder performed the best with DeepOmicsSurv. SHAP analysis showed that Age, AJCC N Stage, alcohol history and patient smoking history are prevalent clinical features for survival time.

CONCLUSION

In conclusion, DeepOmicsSurv has the potential to predict survival time in oral cancer patients. This model achieved high accuracy with various data types including Clinical, DNAmethylation + clinical, mRNA + clinical, Copy number alteration + clinical, or multi-omics data. Additionally, SHAP analysis reveals clinical factors that influence survival time.

Learning deep representation and discriminative features for clustering of multi-layer networks

The multi-layer network consists of the interactions between different layers, where each layer of the network is depicted as a graph, providing a comprehensive way to model the underlying complex systems. The layer-specific modules of multi-layer networks are critical to understanding the structure and function of the system. However, existing methods fail to characterize and balance the connectivity and specificity of layer-specific modules in networks because of the complicated inter- and intra-coupling of various layers. To address the above issues, a joint learning graph clustering algorithm (DRDF) for detecting layer-specific modules in multi-layer networks is proposed, which simultaneously learns the deep representation and discriminative features. Specifically, DRDF learns the deep representation with deep nonnegative matrix factorization, where the high-order topology of the multi-layer network is gradually and precisely characterized. Moreover, it addresses the specificity of modules with discriminative feature learning, where the intra-class compactness and inter-class separation of pseudo-labels of clusters are explored as self-supervised information, thereby providing a more accurate method to explicitly model the specificity of the multi-layer network. Finally, DRDF balances the connectivity and specificity of layer-specific modules with joint learning, where the overall objective of the graph clustering algorithm and optimization rules are derived. The experiments on ten multi-layer networks showed that DRDF not only outperforms eight baselines on graph clustering but also enhances the robustness of algorithms.

Learning Fair Representations via Distance Correlation Minimization

As machine learning algorithms are increasingly deployed for high-impact automated decision-making, the presence of bias (in datasets or tasks) gradually becomes one of the most critical challenges in machine learning applications. Such challenges range from the bias of race in face recognition to the bias of gender in hiring systems, where race and gender can be denoted as sensitive attributes. In recent years, much progress has been made in ensuring fairness and reducing bias in standard machine learning settings. Among them, learning fair representations with respect to the sensitive attributes has attracted increasing attention due to its flexibility in learning the rich representations based on advances in deep learning. In this article, we propose graph-fair, an algorithmic approach to learning fair representations under the graph Laplacian regularization, which reduces the separation between groups and the clustering within a group by encoding the sensitive attribute information into the graph. We have theoretically proved the underlying connection between graph regularization and distance correlation and show that the latter can be regarded as a standardized version of the former, with an additional advantage of being scale-invariant. Therefore, we naturally adopt the distance correlation as the fairness constraint to decrease the dependence between sensitive attributes and latent representations, called dist-fair. In contrast to existing approaches using measures of dependency and adversarial generators, both graph-fair and dist-fair provide simple fairness constraints, which eliminate the need for parameter tuning (e.g., choosing kernels) and introducing adversarial networks. Experiments conducted on real-world corpora indicate that our proposed fairness constraints applied for representation learning can provide better tradeoffs between fairness and utility results than existing approaches.

Enhanced Dynamic Surface EMG Decomposition Using the Non-Negative Matrix Factorization and Three-Dimensional Motor Unit Localization

OBJECTIVE

Surface electromyography (sEMG) signal decomposition is of great importance in examining neuromuscular diseases and neuromuscular research, especially dynamic sEMG decomposition is even more technically challenging.

METHODS

A novel two-step sEMG decomposition approach was developed. The linear minimum mean square error estimation was first employed to extract estimated firing trains (EFTs) from the eigenvector matrices constructed using the non-negative matrix factorization (NMF). The firing instants of each EFT were then classified into motor units (MUs) according to their specific three-dimensional (3D) space position. The performance of the proposed approach was evaluated using simulated and experimentally recorded sEMG.

RESULTS

The simulation results demonstrated that the proposed approach can reconstruct MUAPTs with true positive rates of 89.12 ± 2.71%, 94.34 ± 1.85% and 95.45 ± 2.11% at signal-to-noise ratios of 10, 20, and 30 dB, respectively. The experimental results also demonstrated a high decomposition accuracy of 90.13 ± 1.31% in the two-source evaluation, and a high accuracy of 91.86 ± 1.14% in decompose-synthesize-decompose- compare evaluation.

CONCLUSIONS

The adoption of NMF reduces the dimension of random pattern under the restriction of non-negativity, as well as keeps the information unchanged as much as possible. The 3D space information of MUs enhances the classification accuracy by tackling the issue of relative movements between MUs and electrodes during dynamic contractions. The accuracy achieved in this study demonstrates the good performance and reliability of the proposed decomposition algorithm in dynamic surface EMG decomposition.

SIGNIFICANCE

The spatiotemporal information is applied to the dynamic surface EMG decomposition.

Global and Local Similarity Learning in Multi-Kernel Space for Nonnegative Matrix Factorization

Most of existing nonnegative matrix factorization (NMF) methods do not fully exploit global and local similarity information from data. In this paper, we propose a novel local similarity learning approach in the convex NMF framework, which encourages inter-class separability that is desired for clustering. Thus, the new model is capable of enhancing intra-class similarity and inter-class separability with simultaneous global and local learning. Moreover, the model learns the factor matrices in an augmented kernel space, which is a convex combination of pre-defined kernels with auto-learned weights. Thus, the learnings of cluster structure, representation factor matrix, and the optimal kernel mutually enhance each other in a seamlessly integrated model, which leads to informative representation. Multiplicative updating rules are developed with theoretical convergence guarantee. Extensive experimental results have confirmed the effectiveness of the proposed model.

Feature Weighted Non-Negative Matrix Factorization

Non-negative matrix factorization (NMF) is one of the most popular techniques for data representation and clustering and has been widely used in machine learning and data analysis. NMF concentrates the features of each sample into a vector and approximates it by the linear combination of basis vectors, such that the low-dimensional representations are achieved. However, in real-world applications, the features usually have different importance. To exploit the discriminative features, some methods project the samples into the subspace with a transformation matrix, which disturbs the original feature attributes and neglects the diversity of samples. To alleviate the above problems, we propose the feature weighted NMF (FNMF) in this article. The salient properties of FNMF can be summarized as three-fold: 1) it learns the weights of features adaptively according to their importance; 2) it utilizes multiple feature weighting components to preserve the diversity; and 3) it can be solved efficiently with the suggested optimization algorithm. The performance on synthetic and real-world datasets demonstrates that the proposed method obtains the state-of-the-art performance.

A Privacy-Preserving Semisupervised Algorithm Under Maximum Correntropy Criterion

Existing semisupervised learning approaches generally focus on the single-agent (centralized) setting, and hence, there is the risk of privacy leakage during joint data processing. At the same time, using the mean square error criterion in such approaches does not allow one to efficiently deal with problems involving non-Gaussian distribution. Thus, in this article, we present a novel privacy-preserving semisupervised algorithm under the maximum correntropy criterion (MCC). The proposed algorithm allows us to share data among different entities while effectively mitigating the risk of privacy leaks. In addition, under MCC, our proposed approach works well for data with non-Gaussian distribution noise. Our experiments on three different learning tasks demonstrate that our method distinctively outperforms the related algorithms in common regression learning scenarios.

Visual-Tactile Fused Graph Learning for Object Clustering

Object clustering has received considerable research attention most recently. However, 1) most existing object clustering methods utilize visual information while ignoring important tactile modality, which would inevitably lead to model performance degradation and 2) simply concatenating visual and tactile information via multiview clustering method can make complementary information to not be fully explored, since there are many differences between vision and touch. To address these issues, we put forward a graph-based visual-tactile fused object clustering framework with two modules: 1) a modality-specific representation learning module M_R and 2) a unified affinity graph learning module M_U . Specifically, M_R focuses on learning modality-specific representations for visual-tactile data, where deep non-negative matrix factorization (NMF) is adopted to extract the hidden information behind each modality. Meanwhile, we employ an autoencoder-like structure to enhance the robustness of the learned representations, and two graphs to improve its compactness. Furthermore, M_U highlights how to mitigate the differences between vision and touch, and further maximize the mutual information, which adopts a minimizing disagreement scheme to guide the modality-specific representations toward a unified affinity graph. To achieve ideal clustering performance, a Laplacian rank constraint is imposed to regularize the learned graph with ideal connected components, where noises that caused wrong connections are removed and clustering labels can be obtained directly. Finally, we propose an efficient alternating iterative minimization updating strategy, followed by a theoretical proof to prove framework convergence. Comprehensive experiments on five public datasets demonstrate the superiority of the proposed framework.

Scalable Gamma-Driven Multilayer Network for Brain Workload Detection Through Functional Near-Infrared Spectroscopy

This work proposes a scalable gamma non-negative matrix network (SGNMN), which uses a Poisson randomized Gamma factor analysis to obtain the neurons of the first layer of a network. These neurons obey Gamma distribution whose shape parameter infers the neurons of the next layer of the network and their related weights. Upsampling the connection weights follows a Dirichlet distribution. Downsampling hidden units obey Gamma distribution. This work performs up-down sampling on each layer to learn the parameters of SGNMN. Experimental results indicate that the width and depth of SGNMN are closely related, and a reasonable network structure for accurately detecting brain fatigue through functional near-infrared spectroscopy can be obtained by considering network width, depth, and parameters.

A Novel Sparse Graph-Regularized Singular Value Decomposition Model and Its Application to Genomic Data Analysis

Learning the gene coexpression pattern is a central challenge for high-dimensional gene expression analysis. Recently, sparse singular value decomposition (SVD) has been used to achieve this goal. However, this model ignores the structural information between variables (e.g., a gene network). The typical graph-regularized penalty can be used to incorporate such prior graph information to achieve more accurate discovery and better interpretability. However, the existing approach fails to consider the opposite effect of variables with negative correlations. In this article, we propose a novel sparse graph-regularized SVD model with absolute operator (AGSVD) for high-dimensional gene expression pattern discovery. The key of AGSVD is to impose a novel graph-regularized penalty ( | u|^T L| u| ). However, such a penalty is a nonconvex and nonsmooth function, so it brings new challenges to model solving. We show that the nonconvex problem can be efficiently handled in a convex fashion by adopting an alternating optimization strategy. The simulation results on synthetic data show that our method is more effective than the existing SVD-based ones. In addition, the results on several real gene expression data sets show that the proposed methods can discover more biologically interpretable expression patterns by incorporating the prior gene network.

Dual-Graph Global and Local Concept Factorization for Data Clustering

Considering a wide range of applications of nonnegative matrix factorization (NMF), many NMF and their variants have been developed. Since previous NMF methods cannot fully describe complex inner global and local manifold structures of the data space and extract complex structural information, we propose a novel NMF method called dual-graph global and local concept factorization (DGLCF). To properly describe the inner manifold structure, DGLCF introduces the global and local structures of the data manifold and the geometric structure of the feature manifold into CF. The global manifold structure makes the model more discriminative, while the two local regularization terms simultaneously preserve the inherent geometry of data and features. Finally, we analyze convergence and the iterative update rules of DGLCF. We illustrate clustering performance by comparing it with latest algorithms on four real-world datasets.

Symmetric Nonnegative Matrix Factorization-Based Community Detection Models and Their Convergence Analysis

Community detection is a popular yet thorny issue in social network analysis. A symmetric and nonnegative matrix factorization (SNMF) model based on a nonnegative multiplicative update (NMU) scheme is frequently adopted to address it. Current research mainly focuses on integrating additional information into it without considering the effects of a learning scheme. This study aims to implement highly accurate community detectors via the connections between an SNMF-based community detector's detection accuracy and an NMU scheme's scaling factor. The main idea is to adjust such scaling factor via a linear or nonlinear strategy, thereby innovatively implementing several scaling-factor-adjusted NMU schemes. They are applied to SNMF and graph-regularized SNMF models to achieve four novel SNMF-based community detectors. Theoretical studies indicate that with the proposed schemes and proper hyperparameter settings, each model can: 1) keep its loss function nonincreasing during its training process and 2) converge to a stationary point. Empirical studies on eight social networks show that they achieve significant accuracy gain in community detection over the state-of-the-art community detectors.

Deep Nonnegative Dictionary Factorization for Hyperspectral Unmixing

As a powerful blind source separation tool, Nonnegative Matrix Factorization (NMF) with effective regularizations has shown significant superiority in spectral unmixing of hyperspectral remote sensing images (HSIs) owing to its good physical interpretability and data adaptability. However, the majority of existing NMF-based spectral unmixing methods only adopt the single layer factorization, which is not favorable for exploiting the complex and structured representation relationship of endmembers implied in HSIs. In order to overcome such an issue, we propose a novel two-stage Deep Nonnegative Dictionary Factorization (DNDF) approach with a sparseness constraint and self-supervised regularization for HSI unmixing. Beyond simply extending one-layer factorization to multi-layer, DNDF conducts fuzzy clustering to tackle the mixed endmembers of HSIs. Moreover, self-supervised regularization is integrated into our DNDF model to impose an effective constraint on the endmember matrix. Experimental results on three real HSIs demonstrate the superiority of DNDF over several state-of-the-art methods.