Multimodal graph-based reranking for web image search

Image search reranking with query-dependent click-based relevance feedback

Our goal is to boost text-based image search results via image reranking. There are diverse modalities (features) of images that we can leverage for reranking, however, the effects of different modalities are query-dependent. The primary challenge we face is how to fuse multiple modalities adaptively for different queries, which has often been overlooked in previous reranking research. Moreover, multimodality fusion without an understanding of the query is risky, and may lead to incorrect judgment in reranking. Therefore, to obtain the best fusion weights for the query, in this paper, we leverage click-through data, which can be viewed as an "implicit" user feedback and an effective means of understanding the query. A novel reranking algorithm, called click-based relevance feedback, is proposed. This algorithm emphasizes the successful use of click-through data for identifying user search intention, while leveraging multiple kernel learning algorithm to adaptively learn the query-dependent fusion weights for multiple modalities. We conduct experiments on a real-world data set collected from a commercial search engine with click-through data. Encouraging experimental results demonstrate that our proposed reranking approach can significantly improve the NDCG@10 of the initial search results by 11.62%, and can outperform several existing approaches for most kinds of queries, such as tail, middle, and top queries.

Pairwise sparsity preserving embedding for unsupervised subspace learning and classification

Two novel unsupervised dimensionality reduction techniques, termed sparse distance preserving embedding (SDPE) and sparse proximity preserving embedding (SPPE), are proposed for feature extraction and classification. SDPE and SPPE perform in the clean data space recovered by sparse representation and enhanced Euclidean distances over noise removed data are employed to measure pairwise similarities of points. In extracting informative features, SDPE and SPPE aim at preserving pairwise similarities between data points in addition to preserving the sparse characteristics. This paper calculates the sparsest representation of all vectors jointly by a convex optimization. The sparsest codes enable certain local information of data to be preserved, and can endow SDPE and SPPE a natural discriminating power, adaptive neighborhood and robust characteristic against noise and errors in delivering low-dimensional embeddings. We also mathematically show SDPE and SPPE can be effectively extended for discriminant learning in a supervised manner. The validity of SDPE and SPPE is examined by extensive simulations. Comparison with other related state-of-the-art unsupervised algorithms show that promising results are delivered by our techniques.

View-based discriminative probabilistic modeling for 3D object retrieval and recognition

In view-based 3D object retrieval and recognition, each object is described by multiple views. A central problem is how to estimate the distance between two objects. Most conventional methods integrate the distances of view pairs across two objects as an estimation of their distance. In this paper, we propose a discriminative probabilistic object modeling approach. It builds probabilistic models for each object based on the distribution of its views, and the distance between two objects is defined as the upper bound of the Kullback-Leibler divergence of the corresponding probabilistic models. 3D object retrieval and recognition is accomplished based on the distance measures. We first learn models for each object by the adaptation from a set of global models with a maximum likelihood principle. A further adaption step is then performed to enhance the discriminative ability of the models. We conduct experiments on the ETH 3D object dataset, the National Taiwan University 3D model dataset, and the Princeton Shape Benchmark. We compare our approach with different methods, and experimental results demonstrate the superiority of our approach.