Average Approximate Hashing-Based Double Projections Learning for Cross-Modal Retrieval

Large-Scale Cross-Modal Hashing with Unified Learning and Multi-Object Regional Correlation Reasoning

To explore the rich information contained in multi-modal data and take into account efficiency, deep cross-modal hash retrieval (DCMHR) is a wise solution. But currently, most DCMHR methods have two key limitations, one is that the recommended classification of DCMHR models is conditioned only on the objects in different regions, respectively. Another flaw is that these methods either do not learn the unified hash codes in training or cannot design an efficient training process. To solve these two problems, this paper designs Large-Scale Cross-Modal Hashing with Unified Learning and Multi-Object Regional Correlation Reasoning (HUMOR). For the proposed related labels classified by ImgNet, HUMOR uses Multiple Instance Learning (MIL) to reason the correlation of these labels. When regional correlation reasoning is low, these labels will be through "reduce-add" to rectification from max-to-min (global precedence) or min-to-max (regional precedence). Then, HUMOR conducts unified learning on hash loss and classification loss, adopts the four-step iterative algorithm to optimize the unified hash codes, and reduces bias in the model. Experiments on two baseline datasets show that the average performance of this method is higher than most of the DCMHR methods. The results demonstrate the effectiveness and innovation of our method.

Cross-modal hashing with missing labels

Hashing-based cross-modal retrieval methods have become increasingly popular due to their advantages in storage and speed. While current methods have demonstrated impressive results, there are still several issues that have not been addressed. Specifically, many of these approaches assume that labels are perfectly assigned, despite the fact that in real-world scenarios, labels are often incomplete or partially missing. There are two reasons for this, as manual labeling can be a complex and time-consuming task, and annotators may only be interested in certain objects. As such, cross-modal retrieval with missing labels is a significant challenge that requires further attention. Moreover, the similarity between labels is frequently ignored, which is important for exploring the high-level semantics of labels. To address these limitations, we propose a novel method called Cross-Modal Hashing with Missing Labels (CMHML). Our method consists of several key components. First, we introduce Reliable Label Learning to preserve reliable information from the observed labels. Next, to infer the uncertain part of the predicted labels, we decompose the predicted labels into latent representations of labels and samples. The representation of samples is extracted from different modalities, which assists in inferring missing labels. We also propose Label Correlation Preservation to enhance the similarity between latent representations of labels. Hash codes are then learned from the representation of samples through Global Approximation Learning. We also construct a similarity matrix according to predicted labels and embed it into hash codes learning to explore the value of labels. Finally, we train linear classifiers to map original samples to a low-dimensional Hamming space. To evaluate the efficacy of CMHML, we conduct extensive experiments on four publicly available datasets. Our method is compared to other state-of-the-art methods, and the results demonstrate that our model performs competitively even when most labels are missing.