A CNN-SIFT Hybrid Pedestrian Navigation Method Based on First-Person Vision

SIFT-CNN: When Convolutional Neural Networks Meet Dense SIFT Descriptors for Image and Sequence Classification

Despite the success of hand-crafted features in computer visioning for many years, nowadays, this has been replaced by end-to-end learnable features that are extracted from deep convolutional neural networks (CNNs). Whilst CNNs can learn robust features directly from image pixels, they require large amounts of samples and extreme augmentations. On the contrary, hand-crafted features, like SIFT, exhibit several interesting properties as they can provide local rotation invariance. In this work, a novel scheme combining the strengths of SIFT descriptors with CNNs, namely SIFT-CNN, is presented. Given a single-channel image, one SIFT descriptor is computed for every pixel, and thus, every pixel is represented as an M-dimensional histogram, which ultimately results in an M-channel image. Thus, the SIFT image is generated from the SIFT descriptors for all the pixels in a single-channel image, while at the same time, the original spatial size is preserved. Next, a CNN is trained to utilize these M-channel images as inputs by operating directly on the multiscale SIFT images with the regular convolution processes. Since these images incorporate spatial relations between the histograms of the SIFT descriptors, the CNN is guided to learn features from local gradient information of images that otherwise can be neglected. In this manner, the SIFT-CNN implicitly acquires a local rotation invariance property, which is desired for problems where local areas within the image can be rotated without affecting the overall classification result of the respective image. Some of these problems refer to indirect immunofluorescence (IIF) cell image classification, ground-based all-sky image-cloud classification and human lip-reading classification. The results for the popular datasets related to the three different aforementioned problems indicate that the proposed SIFT-CNN can improve the performance and surpasses the corresponding CNNs trained directly on pixel values in various challenging tasks due to its robustness in local rotations. Our findings highlight the importance of the input image representation in the overall efficiency of a data-driven system.

A Method for Autonomous Multi-Motion Modes Recognition and Navigation Optimization for Indoor Pedestrian

The indoor navigation method shows great application prospects that is based on a wearable foot-mounted inertial measurement unit and a zero-velocity update principle. Traditional navigation methods mainly support two-dimensional stable motion modes such as walking; special tasks such as rescue and disaster relief, medical search and rescue, in addition to normal walking, are usually accompanied by running, going upstairs, going downstairs and other motion modes, which will greatly affect the dynamic performance of the traditional zero-velocity update algorithm. Based on a wearable multi-node inertial sensor network, this paper presents a method of multi-motion modes recognition for indoor pedestrians based on gait segmentation and a long short-term memory artificial neural network, which improves the accuracy of multi-motion modes recognition. In view of the short effective interval of zero-velocity updates in motion modes with fast speeds such as running, different zero-velocity update detection algorithms and integrated navigation methods based on change of waist/foot headings are designed. The experimental results show that the overall recognition rate of the proposed method is 96.77%, and the navigation error is 1.26% of the total distance of the proposed method, which has good application prospects.

A Coarse-to-Fine Framework for Multiple Pedestrian Crossing Detection

When providing route guidance to pedestrians, one of the major safety considerations is to ensure that streets are crossed at places with pedestrian crossings. As a result, map service providers are keen to gather the location information about pedestrian crossings in the road network. Most, if not all, literature in this field focuses on detecting the pedestrian crossing immediately in front of the camera, while leaving the other pedestrian crossings in the same image undetected. This causes an under-utilization of the information in the video images, because not all pedestrian crossings captured by the camera are detected. In this research, we propose a coarse-to-fine framework to detect pedestrian crossings from probe vehicle videos, which can then be combined with the GPS traces of the corresponding vehicles to determine the exact locations of pedestrian crossings. At the coarse stage of our approach, we identify vanishing points and straight lines associated with the stripes of pedestrian crossings, and partition the edges to obtain rough candidate regions of interest (ROIs). At the fine stage, we determine whether these candidate ROIs are indeed pedestrian crossings by exploring their prior constraint information. Field experiments in Beijing and Shanghai cities show that the proposed approach can produce satisfactory results under a wide variety of situations.

A Novel Pedestrian Dead Reckoning Algorithm for Multi-Mode Recognition Based on Smartphones

With the rapid development of smartphone technology, pedestrian navigation based on built-in inertial sensors in smartphones shows great application prospects. Currently, most smartphone-based pedestrian dead reckoning (PDR) algorithms normally require a user to hold the phone in a fixed mode and, thus, need to correct the gyroscope heading with inputs from other sensors, which restricts the viability of pedestrian navigation significantly. In this paper, in order to improve the accuracy of the traditional step detection and step length estimation method for different users, a state transition-based step detection method and a step length estimation method using a neural network are proposed. In order to decrease the heading errors and inertial sensor errors in multi-mode system, a multi-mode intelligent recognition method based on a neural network was constructed. On this basis, we propose a heading correction method based on zero angular velocity and an overall correction method based on lateral velocity limitation (LV). Experimental results show that the maximum positioning errors obtained by the proposed algorithm are about 0.9% of the total path length. The proposed novel PDR algorithm dramatically enhances the user experience and, thus, has high value in real applications.