Navigating to objects in the real world

Semantic navigation is necessary to deploy mobile robots in uncontrolled environments such as homes or hospitals. Many learning-based approaches have been proposed in response to the lack of semantic understanding of the classical pipeline for spatial navigation, which builds a geometric map using depth sensors and plans to reach point goals. Broadly, end-to-end learning approaches reactively map sensor inputs to actions with deep neural networks, whereas modular learning approaches enrich the classical pipeline with learning-based semantic sensing and exploration. However, learned visual navigation policies have predominantly been evaluated in sim, with little known about what works on a robot. We present a large-scale empirical study of semantic visual navigation methods comparing representative methods with classical, modular, and end-to-end learning approaches across six homes with no prior experience, maps, or instrumentation. We found that modular learning works well in the real world, attaining a 90% success rate. In contrast, end-to-end learning does not, dropping from 77% sim to a 23% real-world success rate because of a large image domain gap between sim and reality. For practitioners, we show that modular learning is a reliable approach to navigate to objects: Modularity and abstraction in policy design enable sim-to-real transfer. For researchers, we identify two key issues that prevent today's simulators from being reliable evaluation benchmarks-a large sim-to-real gap in images and a disconnect between sim and real-world error modes-and propose concrete steps forward.

Non-Uniform Input-Based Adaptive Growing Neural Gas for Unstructured Environment Map Construction

The research and development of special robots such as excavation robots is an important way to achieve safe and efficient production in coal mines. Affected by the unstructured environment such as complex working conditions and unsteady factor disturbances, the real-time construction of section environment maps that can accurately describe the environment and facilitate trajectory planning and decision making has become a key scientific problem to be solved as soon as possible. Therefore, non-uniform input based adaptive growing neural gas for unstructured environment map construction has been proposed. Considering complex load identification, real-time location identification, and the types of unsteady disturbance factors and working conditions, a set of environment identification models has been established based on a large amount of underground measured data training. These models can express whether the section environment has changed, as well as the type and magnitude of the change, to realize the overall knowledge extraction and parametric representation of the unstructured environment. Then, in order to solve the problems of inaccurate topology, excessive aging of connecting edges, and excessive deletion of nodes in non-uniform input environment, an adaptive growing neural gas algorithm based on non-uniform input environment (AGNG-NU) is proposed. Featured by a dynamic response deletion mechanism and adaptive adjustment mechanism of neuron parameters, the generated nodes and their topology can be dynamically adjusted according to the density of regional sample points. Several sets of non-uniform input environments are set to test the algorithm. The experimental results show that the topological maps established by AGNG-NU express clearer environmental information and, at the same time, the accuracy and distribution are improved by 8% and 15%, respectively, compared with the basic GNG algorithm. The accuracy and the distribution have also been significantly improved compared with other common SOM and GCS algorithms.

A Data Association Algorithm for SLAM Based on Central Difference Joint Compatibility Criterion and Clustering

A data association algorithm for simultaneous localization and mapping (SLAM) based on central difference joint compatibility (CDJC) criterion and clustering is proposed to obtain the data association results. Firstly, CDJC criterion is designed to calculate joint Mahalanobis distance. Secondly, ordering points to identify the clustering structure is used to divide all observed features into several groups. Thirdly, CDJC branch and bound method is designed to be performed in each group. The results based on simulation data and benchmark dataset show that the proposed algorithm has low computational complexity and provide accurate association results for SLAM of mobile robot.

Mobile Robot Path Planning Using a Laser Range Finder for Environments with Transparent Obstacles

Environment maps must first be generated to drive mobile robots automatically. Path planning is performed based on the information given in an environment map. Various types of sensors, such as ultrasonic and laser sensors, are used by mobile robots to acquire data on its surrounding environment. Among these, the laser sensor, which has the property of being able to go straight and high accuracy, is used most often. However, the beams from laser sensors are refracted and reflected when it meets a transparent obstacle, thus generating noise. Therefore, in this paper, a state-of-the-art algorithm was proposed to detect transparent obstacles by analyzing the pattern of the reflected noise generated when a laser meets a transparent obstacle. The experiment was carried out using the environment map generated by the aforementioned method and gave results demonstrating that the robot could avoid transparent obstacles while it was moving towards the destination.

Survey of Datafusion Techniques for Laser and Vision Based Sensor Integration for Autonomous Navigation

This paper focuses on data fusion, which is fundamental to one of the most important modules in any autonomous system: perception. Over the past decade, there has been a surge in the usage of smart/autonomous mobility systems. Such systems can be used in various areas of life like safe mobility for the disabled, senior citizens, and so on and are dependent on accurate sensor information in order to function optimally. This information may be from a single sensor or a suite of sensors with the same or different modalities. We review various types of sensors, their data, and the need for fusion of the data with each other to output the best data for the task at hand, which in this case is autonomous navigation. In order to obtain such accurate data, we need to have optimal technology to read the sensor data, process the data, eliminate or at least reduce the noise and then use the data for the required tasks. We present a survey of the current data processing techniques that implement data fusion using different sensors like LiDAR that use light scan technology, stereo/depth cameras, Red Green Blue monocular (RGB) and Time-of-flight (TOF) cameras that use optical technology and review the efficiency of using fused data from multiple sensors rather than a single sensor in autonomous navigation tasks like mapping, obstacle detection, and avoidance or localization. This survey will provide sensor information to researchers who intend to accomplish the task of motion control of a robot and detail the use of LiDAR and cameras to accomplish robot navigation.

Mapping Partially Observable Features from Multiple Uncertain Vantage Points

In this paper we present a technique for mapping partially observable features from multiple uncertain vantage points. The problem of concurrent mapping and localization (CML) is stated as follows. Starting from an initial known position, a mobile robot travels through a sequence of positions, obtaining a set of sensor measurements at each position. The goal is to process the sensor data to produce an estimate of the trajectory of the robot while concurrently building a map of the environment. In this paper, we describe a generalized framework for CML that incorporates temporal as well as spatial correlations. The representation is expanded to incorporate past vehicle positions in the state vector. Estimates of the correlations between current and previous vehicle states are explicitly maintained. This enables the consistent initialization of map features using data from multiple time steps. Updates to the map and the vehicle trajectory can also be performed in batches of data acquired from multiple vantage points. The method is illustrated with sonar data from a testing tank and via experiments with a B21 land mobile robot, demonstrating the ability to perform CML with sparse and ambiguous data.

Square Root SAM: Simultaneous Localization and Mapping via Square Root Information Smoothing

Solving the SLAM (simultaneous localization and mapping) problem is one way to enable a robot to explore, map, and navigate in a previously unknown environment. Smoothing approaches have been investigated as a viable alternative to extended Kalman filter (EKF)-based solutions to the problem. In particular, approaches have been looked at that factorize either the associated information matrix or the measurement Jacobian into square root form. Such techniques have several significant advantages over the EKF: they are faster yet exact; they can be used in either batch or incremental mode; are better equipped to deal with non-linear process and measurement models; and yield the entire robot trajectory, at lower cost for a large class of SLAM problems. In addition, in an indirect but dramatic way, column ordering heuristics automatically exploit the locality inherent in the geographic nature of the SLAM problem. This paper presents the theory underlying these methods, along with an interpretation of factorization in terms of the graphical model associated with the SLAM problem. Both simulation results and actual SLAM experiments in large-scale environments are presented that underscore the potential of these methods as an alternative to EKF-based approaches.

Surface Profile Determination from Multiple Sonar Data Using Morphological Processing

This paper presents a novel method for surface profile determination using multiple sensors. Our approach is based on morphological processing techniques to fuse the range data from multiple sensor returns in a manner that directly reveals the target surface profile. The method has the intrinsic ability of suppressing spurious readings due to noise, crosstalk, and higher-order reflections, as well as processing multiple reflections informatively. The approach taken is extremely flexible and robust, in addition to being simple and straightforward. It can deal with arbitrary numbers and configurations of sensors as well as synthetic arrays. The algorithm is verified both by simulations and experiments in the laboratory by processing real sonar data obtained from a mobile robot. The results are compared to those obtained from a more accurate structured-light system, which is, however, more complex and expensive.

Adaptive Mobile Robot Navigation and Mapping

The task of building a map of an unknown environment and concurrently using that map to navigate is a central problem in mobile robotics research. This paper addresses the problem of how to perform concurrent mapping and localization (CML) adaptively using sonar. Stochastic mapping is a feature-based approach to CML that generalizes the extended Kalman filter to incorporate vehicle localization and environmental mapping. The authors describe an implementation of stochastic mapping that uses a delayed nearest neighbor data association strategy to initialize new features into the map, match measurements to map features, and delete out-of-date features. The authors introduce a metric for adaptive sensing that is defined in terms of Fisher information and represents the sum of the areas of the error ellipses of the vehicle and feature estimates in the map. Predicted sensor readings and expected dead-reckoning errors are used to estimate the metric for each potential action of the robot, and the action that yields the lowest cost (i.e., the maximum information) is selected. This technique is demonstrated via simulations, in-air sonar experiments, and underwater sonar experiments. Results are shown for (1) adaptive control of motion and (2) adaptive control of motion and scanning. The vehicle tends to explore selectively different objects in the environment. The performance of this adaptive algorithm is shown to be superior to straight-line motion and random motion.

Statistical Evaluation Method for Comparing Grid Map Based Sensor Fusion Algorithms

In this paper we present a method for evaluating sensor fusion algorithms based on a quantitative comparison, which is independent of the data acquired and the sensors used. The sensor fusion performance measures and performance analysis procedure provide a basis for modeling, analyzing, experimenting, and comparing different sensor fusion algorithms. The capability to compare different algorithms creates a ranking basis, making it possible to select the best algorithm. The statistical evaluation method defines the experimental design and statistical analysis. The numbers of experiments and repetitions required are derived from the statistical characteristics and the desired confidence level. Since procedures are defined to ensure that the experiments are indeed conducted differently, the results are not specific for either the evaluated test cases or the sensor characteristics. The statistical analysis provides a systematic method for comparing sensor fusion algorithms. Although this method requires experimentation, it offers the ability to compare actual performances in the real world. Quantitative procedures are developed to ensure that specific environmental conditions evaluated do not influence the evaluation. To demonstrate the statistical evaluation method it is applied to a case study that compared five different sensor fusion algorithms in a mobile robot experiment.

A Probabilistic On-Line Mapping Algorithm for Teams of Mobile Robots

An efficient probabilistic algorithm for the concurrent mapping and localization problem that arises in mobile robotics is presented. The algorithm addresses the problem in which a team of robots builds a map on-line while simultaneously accommodating errors in the robots’ odometry. At the core of the algorithm is a technique that combines fast maximum likelihood map growing with a Monte Carlo localizer that uses particle representations. The combination of both yields an on-line algorithm that can cope with large odometric errors typically found when mapping environments with cycles. The algorithm can be implemented in a distributed manner on multiple robot platforms, enabling a team of robots to cooperatively generate a single map of their environment. Finally, an extension is described for acquiring three-dimensional maps, which capture the structure and visual appearance of indoor environments in three dimensions.

Probabilistic Algorithms and the Interactive Museum Tour-Guide Robot Minerva

This paper describes Minerva, an interactive tour-guide robot that was successfully deployed in a Smithsonian museum. Minerva’s software is pervasively probabilistic, relying on explicit representations of uncertainty in perception and control. During 2 weeks of operation, the robot interacted with thousands of people, both in the museum and through the Web, traversing more than 44 km at speeds of up to 163 cm/sec in the unmodified museum.

Mobile-Robot Map Building from an Advanced Sonar Array and Accurate Odometry

This paper describes a mobile robot equipped with a sonar sensor array in a guided, feature-based map-building task in an indoor environment. The landmarks common to indoor environments areplanes, corners, and edges, and these are located and classified with the sonar sensor array The map-building process makes use of accurate odometry information that is derivedfiom apair ofknife-edged unloaded encoder wheels. Discrete sonar observations are incrementally merged into partial planes to produce a realistic representation of the environment that is amenable to sonar localization. Collinearity constraints among featurs ar exploited to enhance both the map-featwe estimation and robot localization. The map update employs an iterated extended Kalmanfilter in the first implementation and subsequently a comparison is made with the Julier-Uhlmann-Durrant-Whyte Kalman filter which improves the accuracy of covariance propagation when nonlinear equations are involved. The map accounts for correlation among features and robot positions. Partial planes am also used to eliminate phantom targets caused by specular reflection of the sonar. Unclassifiable sonar targets are integrated into the map for the purpose of obstacle avoidance. The paper presents simulated and experimental data.

Recognizing Retro-Reflectors with an Obliquely-Oriented Multi-Point Sonar and Acoustic Flow

This paper applies acoustic principles and a novel scanning method motivated by acoustic flow to recognize retro-reflectors directly from sonar echoes while performing drive-by scanning. Right-angle corners and cylinders form specular retro-reflectors that produce strong echoes whose features can be easily identified. A multi-point sonar produces a point process whose density encodes the echo amplitude and allows strong echoes to be isolated. In drive-by scanning an obliquely-oriented sonar beam passes over a retro-reflector which then exhibits a sequence of strong echoes having a pattern predicted by a forward model. Drive-by scans of a simple retro-reflector, a complex hallway environment and distant objects illustrate the method. An algorithm recognizes and localizes the retro-reflector by performing a two-dimensional search to find coordinates producing range readings matching the data in a weighted-least-squared error sense. A conventional Polaroid 6500 ranging module produced 5,000 sonar points in a drive-by scan of a hallway, which the algorithm converted into six corner locations. Interfering reflectors were detected by large deviations from the predicted template. Echoes from occluded retro-reflectors up to 7 m range were analyzed. Recognizing objects directly from echoes extends sonar sensing from data acquisition to landmark identification for robot navigation.

An Efficient Data Association Approach to Simultaneous Localization and Map Building

In this paper we present an efficient integer programming (IP) based data association approach to simultaneous localization and mapping (SLAM). In this approach, the feature-based SLAM data association problem is formulated as a 0-1 IP problem. The IP problem is approached by first solving a relaxed linear programming (LP) problem. Based on the optimal LP solution, a suboptimal solution to the IP problem is then obtained by applying an iterative heuristic greedy rounding (IHGR) procedure. Unlike the traditional nearest-neighbor (NN) algorithm, the proposed algorithm deals with the global matching between existing features and measurements of each scan and is more robust for an environment of high-density features (the feature number is high and the distances between features are often very close) which is usually the case in outdoor applications. Detailed simulation and experimental studies show that the proposed IHGR-based algorithm has moderate computational requirement and offers a better performance with higher successful rate of SLAM for complex environments of high density of features than the NN algorithm.

A double refresh rate sonar ring with FPGA-based continuous matched filtering

Existing sonar rings are limited in their refresh rate to the transmit echo rate, that is, waiting for maximum range echoes to arrive before transmitting again. This paper presents a sonar ring refreshing at 60 Hz for 5.7-m range, which is twice the transmit echo rate, and this leads to lower latency, denser measurements. Two custom Field Programmable Gate Array signal processors provide real time continuous match filtering with dynamic templates. A new method is implemented to select the transmit time from a random set based on minimizing interference. Experiments demonstrate the increased refresh rate, interference rejection, and maps generated by the sonar ring.

CONCURRENT MAP BUILDING AND LOCALIZATION ON INDOOR DYNAMIC ENVIRONMENTS

A system that builds and maintains a dynamic map for a mobile robot is presented. A learning rule associated to each observed landmark is used to compute its robustness. The position of the robot during map construction is estimated by combining sensor readings, motion commands, and the current map state by means of an Extended Kalman Filter. The combination of landmark strength validation and Kalman filtering for map updating and robot position estimation allows for robust learning of moderately dynamic indoor environments.

The KCLBOT: Exploiting RGB-D Sensor Inputs for Navigation Environment Building and Mobile Robot Localization

This paper presents an alternative approach to implementing a stereo camera configuration for SLAM. The approach suggested implements a simplified method using a single RGB-D camera sensor mounted on a maneuverable non-holonomic mobile robot, the KCLBOT, used for extracting image feature depth information while maneuvering. Using a defined quadratic equation, based on the calibration of the camera, a depth computation model is derived base on the HSV color space map. Using this methodology it is possible to build navigation environment maps and carry out autonomous mobile robot path following and obstacle avoidance. This paper presents a calculation model which enables the distance estimation using the RGB-D sensor from Microsoft .NET micro framework device. Experimental results are presented to validate the distance estimation methodology.

Common Bayesian models for common cognitive issues

How can an incomplete and uncertain model of the environment be used to perceive, infer, decide and act efficiently? This is the challenge that both living and artificial cognitive systems have to face. Symbolic logic is, by its nature, unable to deal with this question. The subjectivist approach to probability is an extension to logic that is designed specifically to face this challenge. In this paper, we review a number of frequently encountered cognitive issues and cast them into a common Bayesian formalism. The concepts we review are ambiguities, fusion, multimodality, conflicts, modularity, hierarchies and loops. First, each of these concepts is introduced briefly using some examples from the neuroscience, psychophysics or robotics literature. Then, the concept is formalized using a template Bayesian model. The assumptions and common features of these models, as well as their major differences, are outlined and discussed.

An O(N(2)) square root unscented Kalman Filter for visual simultaneous localization and mapping

This paper develops a Square Root Unscented Kalman Filter (SRUKF) for performing video-rate visual simultaneous localization and mapping (SLAM) using a single camera. The conventional UKF has been proposed previously for SLAM, improving the handling of nonlinearities compared with the more widely used Extended Kalman Filter (EKF). However, no account was taken of the comparative complexity of the algorithms: In SLAM, the UKF scales as O(N;{3}) in the state length, compared to the EKF's O(N;{2}), making it unsuitable for video-rate applications with other than unrealistically few scene points. Here, it is shown that the SRUKF provides the same results as the UKF to within machine accuracy and that it can be reposed with complexity O(N;{2}) for state estimation in visual SLAM. This paper presents results from video-rate experiments on live imagery. Trials using synthesized data show that the consistency of the SRUKF is routinely better than that of the EKF, but that its overall cost settles at an order of magnitude greater than the EKF for large scenes.

Mobile robot localization using a single rotating sonar and two passive cylindrical beacons

This paper proposes a new method of estimating the position and heading angle of a mobile robot moving on a fiat surface. The proposed localization method utilizes two passive beacons and a single rotating ultrasonic sensor. The passive beacons consist of two cylinders with different diameters and reflect the ultrasonic pulses coming from the sonar sensor mounted on the mobile robot. The sonar sensor, again, mounted on a pan-tilt device then receives the reflected pulses while scanning over a wide area. The geometric parameter set of beaconis acquired from the sonar scan data obtained at a single mobile robot location using a new data processing algorithm. The presented algorithm is especially suitable for processing the sonar scan data obtained by ultrasonic sensor with wide beam spread. From this parameter set, the position and heading angle of the mobile robot is determined directly. The performance and validity of the proposed method are evaluated using two beacons and a single sonar sensor attached at the pan-tilt device mounted on a mobile robot, named LCAR, in our laboratory.

Simultaneous landmark classification, localization and map building for an advanced sonar ring

An autonomous mobile robot operating in an unknown indoor environment often needs to map the environment while localizing within the map. Feature-based world models including line and point features are widely used by researchers. This paper presents a novel delayed-classi-fication algorithm to categorize these features using a recently developed high-performance sonar ring within a simultaneous localization and map-building (SLAM) process. The sonar ring sensor accurately measures range and bearing to multiple targets at near real-time repetition rates of 11.5 Hz to 6 m range, and uses 24 simultaneously fired transmitters, 48 receivers and multiple echoes per receiver. The proposed algorithm is based on hypothesis generation and verification using the advanced sonar ring data and an extended Kalman filter (EKF) approach. It is capable of initiating new geometric features and classifying them within a short distance of travel of about 10 cm. For each new sonar reading not matching an existing feature, we initiate a pair of probational line and point features resulting from accurate range and bearing measurements. Later measurements are used to confirm or remove the probational features using EKF validation gates. The odometry error model of the filter allows for variations in effective wheel separation required by pneumatic robot tyres. The implementation of the novel classification and SLAM algorithm is discussed in this paper and experimental results using real sonar data are presented.