SymNet: A Simple Symmetric Positive Definite Manifold Deep Learning Method for Image Set Classification

Deep Geometric Learning With Monotonicity Constraints for Alzheimer's Disease Progression

Alzheimer's disease (AD) is a devastating neurodegenerative condition that precedes progressive and irreversible dementia; thus, predicting its progression over time is vital for clinical diagnosis and treatment. For this, numerous studies have implemented structural magnetic resonance imaging (MRI) to model AD progression, focusing on three integral aspects: 1) temporal variability; 2) incomplete observations; and 3) temporal geometric characteristics. However, many pioneer deep learning-based approaches addressing data variability and sparsity have yet to consider inherent geometrical properties sufficiently. These properties are integral to modeling as they correlate with brain region size, thickness, volume, and shape in AD progression. The ordinary differential equation-based geometric modeling method (ODE-RGRU) has recently emerged as a promising strategy for modeling time-series data by intertwining a recurrent neural network (RNN) and an ODE in Riemannian space. Despite its achievements, ODE-RGRU encounters limitations when extrapolating positive definite symmetric matrices from incomplete samples, leading to feature reverse occurrences that are particularly problematic, especially within the clinical facet. Therefore, this study proposes a novel geometric learning approach that models longitudinal MRI biomarkers and cognitive scores by combining three modules: topological space shift, ODE-RGRU, and trajectory estimation. We have also developed a training algorithm that integrates the manifold mapping with monotonicity constraints to reflect measurement transition irreversibility. We verify our proposed method's efficacy by predicting clinical labels and cognitive scores over time in regular and irregular settings. Furthermore, we thoroughly analyze our proposed framework through an ablation study.

Multiple Riemannian Kernel Hashing for Large-Scale Image Set Classification and Retrieval

Conventional image set methods typically learn from small to medium-sized image set datasets. However, when applied to large-scale image set applications such as classification and retrieval, they face two primary challenges: 1) effectively modeling complex image sets; and 2) efficiently performing tasks. To address the above issues, we propose a novel Multiple Riemannian Kernel Hashing (MRKH) method that leverages the powerful capabilities of Riemannian manifold and Hashing on effective and efficient image set representation. MRKH considers multiple heterogeneous Riemannian manifolds to represent each image set. It introduces a multiple kernel learning framework designed to effectively combine statistics from multiple manifolds, and constructs kernels by selecting a small set of anchor points, enabling efficient scalability for large-scale applications. In addition, MRKH further exploits inter- and intra-modal semantic structure to enhance discrimination. Instead of employing continuous feature to represent each image set, MRKH suggests learning hash code for each image set, thereby achieving efficient computation and storage. We present an iterative algorithm with theoretical convergence guarantee to optimize MRKH, and the computational complexity is linear with the size of dataset. Extensive experiments on five image set benchmark datasets including three large-scale ones demonstrate the proposed method outperforms state-of-the-arts in accuracy and efficiency particularly in large-scale image set classification and retrieval.

Joint Metric Learning-Based Class-Specific Representation for Image Set Classification

With the rapid advances in digital imaging and communication technologies, recently image set classification has attracted significant attention and has been widely used in many real-world scenarios. As an effective technology, the class-specific representation theory-based methods have demonstrated their superior performances. However, this type of methods either only uses one gallery set to measure the gallery-to-probe set distance or ignores the inner connection between different metrics, leading to the learned distance metric lacking robustness, and is sensitive to the size of image sets. In this article, we propose a novel joint metric learning-based class-specific representation framework (JMLC), which can jointly learn the related and unrelated metrics. By iteratively modeling probe set and related or unrelated gallery sets as affine hull, we reconstruct this hull sparsely or collaboratively over another image set. With the obtained representation coefficients, the combined metric between the query set and the gallery set can then be calculated. In addition, we also derive the kernel extension of JMLC and propose two new unrelated set constituting strategies. Specifically, kernelized JMLC (KJMLC) embeds the gallery sets and probe sets into the high-dimensional Hilbert space, and in the kernel space, the data become approximately linear separable. Extensive experiments on seven benchmark databases show the superiority of the proposed methods to the state-of-the-art image set classifiers.

U-SPDNet: An SPD manifold learning-based neural network for visual classification

With the development of neural networking techniques, several architectures for symmetric positive definite (SPD) matrix learning have recently been put forward in the computer vision and pattern recognition (CV&PR) community for mining fine-grained geometric features. However, the degradation of structural information during multi-stage feature transformation limits their capacity. To cope with this issue, this paper develops a U-shaped neural network on the SPD manifolds (U-SPDNet) for visual classification. The designed U-SPDNet contains two subsystems, one of which is a shrinking path (encoder) making up of a prevailing SPD manifold neural network (SPDNet (Huang and Van Gool, 2017)) for capturing compact representations from the input data. Another is a constructed symmetric expanding path (decoder) to upsample the encoded features, trained by a reconstruction error term. With this design, the degradation problem will be gradually alleviated during training. To enhance the representational capacity of U-SPDNet, we also append skip connections from encoder to decoder, realized by manifold-valued geometric operations, namely Riemannian barycenter and Riemannian optimization. On the MDSD, Virus, FPHA, and UAV-Human datasets, the accuracy achieved by our method is respectively 6.92%, 8.67%, 1.57%, and 1.08% higher than SPDNet, certifying its effectiveness.

Hierarchical Hashing Learning for Image Set Classification

With the development of video network, image set classification (ISC) has received a lot of attention and can be used for various practical applications, such as video based recognition, action recognition, and so on. Although the existing ISC methods have obtained promising performance, they often have extreme high complexity. Due to the superiority in storage space and complexity cost, learning to hash becomes a powerful solution scheme. However, existing hashing methods often ignore complex structural information and hierarchical semantics of the original features. They usually adopt a single-layer hashing strategy to transform high-dimensional data into short-length binary codes in one step. This sudden drop of dimension could result in the loss of advantageous discriminative information. In addition, they do not take full advantage of intrinsic semantic knowledge from whole gallery sets. To tackle these problems, in this paper, we propose a novel Hierarchical Hashing Learning (HHL) for ISC. Specifically, a coarse-to-fine hierarchical hashing scheme is proposed that utilizes a two-layer hash function to gradually refine the beneficial discriminative information in a layer-wise fashion. Besides, to alleviate the effects of redundant and corrupted features, we impose the $\ell _{2,1}$ norm on the layer-wise hash function. Moreover, we adopt a bidirectional semantic representation with the orthogonal constraint to keep intrinsic semantic information of all samples in whole image sets adequately. Comprehensive experiments demonstrate HHL acquires significant improvements in accuracy and running time. We will release the demo code on https://github.com/sunyuan-cs.

Learning Non-Euclidean Representations With SPD Manifold for Myoelectric Pattern Recognition

How to learn informative representations from Electromyography (EMG) signals is of vital importance for myoelectric control systems. Traditionally, hand-crafted features are extracted from individual EMG channels and combined together for pattern recognition. The spatial topological information between different channels can also be informative, which is seldom considered. This paper presents a radically novel approach to extract spatial structural information within diverse EMG channels based on the symmetric positive definite (SPD) manifold. The object is to learn non-Euclidean representations inside EMG signals for myoelectric pattern recognition. The performance is compared with two classical feature sets using accuracy and F1-score. The algorithm is tested on eleven gestures collected from ten subjects, and the best accuracy reaches 84.85%±5.15% with an improvement of 4.04%~20.25%, which outperforms the contrast method, and reaches a significant improvement with the Wilcoxon signed-rank test. Eleven gestures from three public databases involving Ninapro DB2, DB4, and DB5 are also evaluated, and better performance is observed. Furthermore, the computational cost is less than the contrast method, making it more suitable for low-cost systems. It shows the effectiveness of the presented approach and contributes a new way for myoelectric pattern recognition.