Efficient multi-modal hypergraph learning for social image classification with complex label correlations

Hypergraph and Uncertain Hypergraph Representation Learning Theory and Methods

With the advent of big data and the information age, the data magnitude of various complex networks is growing rapidly. Many real-life situations cannot be portrayed by ordinary networks, while hypergraphs have the ability to describe and characterize higher order relationships, which have attracted extensive attention from academia and industry in recent years. Firstly, this paper described the development process, the application areas, and the existing review research of hypergraphs; secondly, introduced the theory of hypergraphs briefly; then, compared the learning methods of ordinary graphs and hypergraphs from three aspects: matrix decomposition, random walk, and deep learning; next, introduced the structural optimization of hypergraphs from three perspectives: dynamic hypergraphs, hyperedge weight optimization, and multimodal hypergraph generation; after that, the applicability of three uncertain hypergraph models were analyzed based on three uncertainty theories: probability theory, fuzzy set, and rough set; finally, the future research directions of hypergraphs and uncertain hypergraphs were prospected.

Hypergraph-Induced Convolutional Networks for Visual Classification

At present, convolutional neural networks (CNNs) have become popular in visual classification tasks because of their superior performance. However, CNN-based methods do not consider the correlation of visual data to be classified. Recently, graph convolutional networks (GCNs) have mitigated this problem by modeling the pairwise relationship in visual data. Real-world tasks of visual classification typically must address numerous complex relationships in the data, which are not fit for the modeling of the graph structure using GCNs. Therefore, it is vital to explore the underlying correlation of visual data. Regarding this issue, we propose a framework called the hypergraph-induced convolutional network to explore the high-order correlation in visual data during deep neural networks. First, a hypergraph structure is constructed to formulate the relationship in visual data. Then, the high-order correlation is optimized by a learning process based on the constructed hypergraph. The classification tasks are performed by considering the high-order correlation in the data. Thus, the convolution of the hypergraph-induced convolutional network is based on the corresponding high-order relationship, and the optimization on the network uses each data and considers the high-order correlation of the data. To evaluate the proposed hypergraph-induced convolutional network framework, we have conducted experiments on three visual data sets: the National Taiwan University 3-D model data set, Princeton Shape Benchmark, and multiview RGB-depth object data set. The experimental results and comparison in all data sets demonstrate the effectiveness of our proposed hypergraph-induced convolutional network compared with the state-of-the-art methods.

Joint Hypergraph Learning for Tag-Based Image Retrieval

As the image sharing websites like Flickr become more and more popular, extensive scholars concentrate on tag-based image retrieval. It is one of the important ways to find images contributed by social users. In this research field, tag information and diverse visual features have been investigated. However, most existing methods use these visual features separately or sequentially. In this paper, we propose a global and local visual features fusion approach to learn the relevance of images by hypergraph approach. A hypergraph is constructed first by utilizing global, local visual features, and tag information. Then, we propose a pseudo-relevance feedback mechanism to obtain the pseudo-positive images. Finally, with the hypergraph and pseudo relevance feedback, we adopt the hypergraph learning algorithm to calculate the relevance score of each image to the query. Experimental results demonstrate the effectiveness of the proposed approach.