Local Semantic-aware Deep Hashing with Hamming-isometric Quantization

Revisiting Multi-Codebook Quantization

Multi-Codebook Quantization (MCQ) is a generalized version of existing codebook-based quantizations for Approximate Nearest Neighbor (ANN) search. Specifically, MCQ picks one codeword for each sub-codebook independently and takes the sum of picked codewords to approximate the original vector. The objective function involves no constraints, therefore, MCQ theoretically has the potential to achieve the best performance because solutions of other codebook-based quantization methods are all covered by MCQ's solution space under the same codebook size setting. However, finding the optimal solution to MCQ is proved to be NP-hard due to its encoding process, i.e., converting an input vector to a binary code. To tackle this, researchers apply constraints to it to find near-optimal solutions or employ heuristic algorithms that are still time-consuming for encoding. Different from previous approaches, this paper takes the first attempt to find a deep solution to MCQ. The encoding network is designed to be as simple as possible, so the very complex encoding problem becomes simply a feed-forward. Compared with other methods on three datasets, our method shows state-of-the-art performance. Notably, our method is 11× - 38× faster than heuristic algorithms for encoding, which makes it more practical for the real scenery of large-scale retrieval. Our code is publicly available: https://github.com/DeepMCQ/DeepQ.

Recovering Surface Normal and Arbitrary Images: A Dual Regression Network for Photometric Stereo

Photometric stereo recovers three-dimensional (3D) object surface normal from multiple images under different illumination directions. Traditional photometric stereo methods suffer from the problem of non-Lambertian surfaces with general reflectance. By leveraging deep neural networks, learning-based methods are capable of improving the surface normal estimation under general non-Lambertian surfaces. These state-of-the-art learning-based methods however do not associate surface normal with reconstructed images and, therefore, they cannot explore the beneficial effect of such association on the estimation of the surface normal. In this paper, we specifically exploit the positive impact of this association and propose a novel dual regression network for both fine surface normals and arbitrary reconstructed images in calibrated photometric stereo. Our work unifies the 3D reconstruction and rendering tasks in a deep learning framework, with the explorations including: 1. generating specified reconstructed images under arbitrary illumination directions, which provides more intuitive perception of the reflectance and is extremely useful for visual applications, such as virtual reality, and 2. our dual regression scheme introduces an additional constraint on observed images and reconstructed images, which forms a closed-loop to provide additional supervision. Experiments show that our proposed method achieves accurate reconstructed images under arbitrarily specified illumination directions and it significantly outperforms the state-of-the-art learning-based single regression methods in calibrated photometric stereo.

Application of semantic location awareness computing based on data mining in COVID-19 prevention and control system

Because of the global spread of COVID-19 in 2020, the analysis of activities and travel behavior of urban residents is the key for the prevention and control of epidemic situation. Based on this, the research on track data mining and semantic location perception is conducted. The analysis of travel behavior characteristics of urban residents is helpful to carry out epidemic prevention activities scientifically. However, the traditional manual survey and statistical analysis cannot meet the needs of the rapid development of urbanization. On the other hand, with the application and development of information technology such as communication, location and storage, a large number of mobile trajectory data of urban residents can be collected and stored. These trajectory data contain rich spatiotemporal semantic information. Through mining and analysis, a lot of valuable travel information can be get and then the daily behavior of individual users and the spatial distribution characteristics of group users’ movement can be found. The results can effectively serve the current epidemic prevention work and can be applied to the infection tracking in the process of epidemic prevention.