Haptic Manipulation of 3D Scans for Geometric Feature Enhancement

Localisation of geometric features like holes, edges, slots, etc. is vital to robotic planning in industrial automation settings. Low-cost 3D scanners are crucial in terms of improving accessibility, but pose a practical challenge to feature localisation because of poorer resolution and consequently affect robotic planning. In this work, we address the possibility of enhancing the quality of a 3D scan by a manual ’touch-up’ of task-relevant features, to ensure their automatic detection prior to automation. We propose a framework whereby the operator (i) has access to both the actual work-piece and its 3D scan; (ii) evaluates the missing salient features from the scan; (iii) uses a haptic stylus to physically interact with the actual work-piece, around such specific features; (iv) interactively updates the scan using the position and force information from the haptic stylus. The contribution of this work is the use of haptic mismatch for geometric update. Specifically, the geometry from the 3D scan is used to predict haptic feedback at a point on the work-piece surface. The haptic mismatch is derived as a measure of error between this prediction and the real interaction forces from physical contact at that point on the work-piece. The geometric update is driven until the haptic mismatch is minimised. Convergence of the proposed algorithm is first numerically verified on an analytical surface with simulated physical interaction. Error analysis of the surface position and orientations were also plotted. Experiments were conducted using a motion capture system providing sub-mm accuracy in position and a 6 axis F/T sensor. Missing features are successfully detected after the update of the scan using the proposed method in an experiment.

GPAtlasRRT: A Local Tactile Exploration Planner for Recovering the Shape of Novel Objects

Touch is an important modality to recover object shape. We present a method for a robot to complete a partial shape model by local tactile exploration. In local tactile exploration, the finger is constrained to follow the local surface. This is useful for recovering information about a contiguous portion of the object and is frequently employed by humans. There are three contributions. First, we show how to segment an initial point cloud of a grasped, unknown object into hand and object. Second, we present a local tactile exploration planner. This combines a Gaussian Process (GP) model of the object surface with an AtlasRRT planner. The GP predicts the unexplored surface and the uncertainty of that prediction. The AtlasRRT creates a tactile exploration path across this predicted surface, driving it towards the region of greatest uncertainty. Finally, we experimentally compare the planner with alternatives in simulation, and demonstrate the complete approach on a real robot. We show that our planner successfully traverses the object, and that the full object shape can be recovered with a good degree of accuracy.

Ergodic Exploration Using Binary Sensing for Nonparametric Shape Estimation

Current methods to estimate object shape-using either vision or touch-generally depend on high-resolution sensing. Here, we exploit ergodic exploration to demonstrate successful shape estimation when using a low-resolution binary contact sensor. The measurement model is posed as a collision-based tactile measurement, and classification methods are used to discriminate between shape boundary regions in the search space. Posterior likelihood estimates of the measurement model help the system actively seek out regions where the binary sensor is most likely to return informative measurements. Results show successful shape estimation of various objects as well as the ability to identify multiple objects in an environment. Interestingly, it is shown that ergodic exploration utilizes non-contact motion to gather significant information about shape. The algorithm is extended in three dimensions in simulation and we present two dimensional experimental results using the Rethink Baxter robot.

Haptic Perception of Unknown Object by Robot Hand: Exploration Strategy and Recognition Approach

In this paper, the exploration and recognition in unknown object perception by robot hand is discussed. Inspired by the touch and exploration of human hand, a haptic exploration strategy for multi-fingered robot hand is proposed. Based on the observations from human experiments, the proposed strategy can be used to guide the robot hand to plan a series of movements to get tactile information from different unknown objects, with the precondition of avoiding unexpected collisions with the objects. A recognition approach is then presented to recognize object shapes based on the tactile point data collected by the strategy. Geometric feature vectors are extracted from tactile point locations and normal vectors after clustering, and the object shapes are recognized by the random forests classifier. Simulations and experiments results show that the exploration strategy can be used to guide the robot to gather tactile information from unknown object automatically, and the recognition approach is effective and robust in object shape recognition work. This framework provides a sensible solution for robot unknown object perception problem, which is suitable for the multi-fingered robot hand with low-resolution tactile sensors.

Incrementally learning objects by touch: online discriminative and generative models for tactile-based recognition

Human beings not only possess the remarkable ability to distinguish objects through tactile feedback but are further able to improve upon recognition competence through experience. In this work, we explore tactile-based object recognition with learners capable of incremental learning. Using the sparse online infinite Echo-State Gaussian process (OIESGP), we propose and compare two novel discriminative and generative tactile learners that produce probability distributions over objects during object grasping/palpation. To enable iterative improvement, our online methods incorporate training samples as they become available. We also describe incremental unsupervised learning mechanisms, based on novelty scores and extreme value theory, when teacher labels are not available. We present experimental results for both supervised and unsupervised learning tasks using the iCub humanoid, with tactile sensors on its five-fingered anthropomorphic hand, and 10 different object classes. Our classifiers perform comparably to state-of-the-art methods (C4.5 and SVM classifiers) and findings indicate that tactile signals are highly relevant for making accurate object classifications. We also show that accurate "early" classifications are possible using only 20-30 percent of the grasp sequence. For unsupervised learning, our methods generate high quality clusterings relative to the widely-used sequential k-means and self-organising map (SOM), and we present analyses into the differences between the approaches.

Object-shape recognition and 3D reconstruction from tactile sensor images

This article presents a novel approach of edged and edgeless object-shape recognition and 3D reconstruction from gradient-based analysis of tactile images. We recognize an object's shape by visualizing a surface topology in our mind while grasping the object in our palm and also taking help from our past experience of exploring similar kind of objects. The proposed hybrid recognition strategy works in similar way in two stages. In the first stage, conventional object-shape recognition using linear support vector machine classifier is performed where regional descriptors features have been extracted from the tactile image. A 3D shape reconstruction is also performed depending upon the edged or edgeless objects classified from the tactile images. In the second stage, the hybrid recognition scheme utilizes the feature set comprising both the previously obtained regional descriptors features and some gradient-related information from the reconstructed object-shape image for the final recognition in corresponding four classes of objects viz. planar, one-edged object, two-edged object and cylindrical objects. The hybrid strategy achieves 97.62 % classification accuracy, while the conventional recognition scheme reaches only to 92.60 %. Moreover, the proposed algorithm has been proved to be less noise prone and more statistically robust.

Three-dimensional object reconstruction of symmetric objects by fusing visual and tactile sensing

In this work, we propose to reconstruct a complete three-dimensional (3-D) model of an unknown object by fusion of visual and tactile information while the object is grasped. Assuming the object is symmetric, a first hypothesis of its complete 3-D shape is generated. A grasp is executed on the object with a robotic manipulator equipped with tactile sensors. Given the detected contacts between the fingers and the object, the initial full object model including the symmetry parameters can be refined. This refined model will then allow the planning of more complex manipulation tasks. The main contribution of this work is an optimal estimation approach for the fusion of visual and tactile data applying the constraint of object symmetry. The fusion is formulated as a state estimation problem and solved with an iterated extended Kalman filter. The approach is validated experimentally using both artificial and real data from two different robotic platforms.