Learnable Subspace Clustering

Ensemble Prototype Network For Weakly Supervised Temporal Action Localization

Weakly supervised temporal action localization (TAL) aims to localize the action instances in untrimmed videos using only video-level action labels. Without snippet-level labels, this task should be hard to distinguish all snippets with accurate action/background categories. The main difficulties are the large variations brought by the unconstraint background snippets and multiple subactions in action snippets. The existing prototype model focuses on describing snippets by covering them with clusters (defined as prototypes). In this work, we argue that the clustered prototype covering snippets with simple variations still suffers from the misclassification of the snippets with large variations. We propose an ensemble prototype network (EPNet), which ensembles prototypes learned with consensus-aware clustering. The network stacks a consensus prototype learning (CPL) module and an ensemble snippet weight learning (ESWL) module as one stage and extends one stage to multiple stages in an ensemble learning way. The CPL module learns the consensus matrix by estimating the similarity of clustering labels between two successive clustering generations. The consensus matrix optimizes the clustering to learn consensus prototypes, which can predict the snippets with consensus labels. The ESWL module estimates the weights of the misclassified snippets using the snippet-level loss. The weights update the posterior probabilities of the snippets in the clustering to learn prototypes in the next stage. We use multiple stages to learn multiple prototypes, which can cover the snippets with large variations for accurate snippet classification. Extensive experiments show that our method achieves the state-of-the-art weakly supervised TAL methods on two benchmark datasets, that is, THUMOS'14, ActivityNet v1.2, and ActivityNet v1.3 datasets.

Knowledge-Induced Multiple Kernel Fuzzy Clustering

The introduction of domain knowledge opens new horizons to fuzzy clustering. Then knowledge-driven and data-driven fuzzy clustering methods come into being. To address the challenges of inadequate extraction mechanism and imperfect fusion mode in such class of methods, we propose the Knowledge-induced Multiple Kernel Fuzzy Clustering (KMKFC) algorithm. First, to extract knowledge points better, the Relative Density-based Knowledge Extraction (RDKE) method is proposed to extract high-density knowledge points close to cluster centers of real data structure, and provide initialized cluster centers. Moreover, the multiple kernel mechanism is introduced to improve the adaptability of clustering algorithm and map data to high-dimensional space, so as to better discover the differences between the data and obtain superior clustering results. Second, knowledge points generated by RDKE are integrated into KMKFC through a knowledge-influence matrix to guide the iterative process of KMKFC. Third, we also provide a strategy of automatically obtaining knowledge points, and thus propose the RDKE with Automatic knowledge acquisition (RDKE-A) method and the corresponding KMKFC-A algorithm. Then we prove the convergence of KMKFC and KMKFC-A. Finally, experimental studies demonstrate that the KMKFC and KMKFC-A algorithms perform better than thirteen comparison algorithms with regard to four evaluation indexes and the convergence speed.

What and How: Generalized Lifelong Spectral Clustering via Dual Memory

Spectral clustering has become one of the most effective clustering algorithms. We in this work explore the problem of spectral clustering in a lifelong learning framework termed as Generalized Lifelong Spectral Clustering (GL 2SC). Different from most current studies, which concentrate on a fixed spectral clustering task set and cannot efficiently incorporate a new clustering task, the goal of our work is to establish a generalized model for new spectral clustering task by What and How to lifelong learn from past tasks. For what to lifelong learn, our GL 2SC framework contains a dual memory mechanism with a deep orthogonal factorization manner: an orthogonal basis memory stores hidden and hierarchical clustering centers among learned tasks, and a feature embedding memory captures deep manifold representation common across multiple related tasks. When a new clustering task arrives, the intuition here for how to lifelong learn is that GL 2SC can transfer intrinsic knowledge from dual memory mechanism to obtain task-specific encoding matrix. Then the encoding matrix can redefine the dual memory over time to provide maximal benefits when learning future tasks. To the end, empirical comparisons on several benchmark datasets show the effectiveness of our GL 2SC, in comparison with several state-of-the-art spectral clustering models.