Discriminating Free Hand Movements Using Support Vector Machine and Recurrent Neural Network Algorithms

Decoding natural hand movements is of interest for human-computer interaction and may constitute a helpful tool in the diagnosis of motor diseases and rehabilitation monitoring. However, the accurate measurement of complex hand movements and the decoding of dynamic movement data remains challenging. Here, we introduce two algorithms, one based on support vector machine (SVM) classification combined with dynamic time warping, and the other based on a long short-term memory (LSTM) neural network, which were designed to discriminate small differences in defined sequences of hand movements. We recorded hand movement data from 17 younger and 17 older adults using an exoskeletal data glove while they were performing six different movement tasks. Accuracy rates in decoding the different movement types were similarly high for SVM and LSTM in across-subject classification, but, for within-subject classification, SVM outperformed LSTM. The SVM-based approach, therefore, appears particularly promising for the development of movement decoding tools, in particular if the goal is to generalize across age groups, for example for detecting specific motor disorders or tracking their progress over time.

Kinematic-Based Classification of Social Gestures and Grasping by Humans and Machine Learning Techniques

The affective motion of humans conveys messages that other humans perceive and understand without conventional linguistic processing. This ability to classify human movement into meaningful gestures or segments plays also a critical role in creating social interaction between humans and robots. In the research presented here, grasping and social gesture recognition by humans and four machine learning techniques (k-Nearest Neighbor, Locality-Sensitive Hashing Forest, Random Forest and Support Vector Machine) is assessed by using human classification data as a reference for evaluating the classification performance of machine learning techniques for thirty hand/arm gestures. The gestures are rated according to the extent of grasping motion on one task and the extent to which the same gestures are perceived as social according to another task. The results indicate that humans clearly rate differently according to the two different tasks. The machine learning techniques provide a similar classification of the actions according to grasping kinematics and social quality. Furthermore, there is a strong association between gesture kinematics and judgments of grasping and the social quality of the hand/arm gestures. Our results support previous research on intention-from-movement understanding that demonstrates the reliance on kinematic information for perceiving the social aspects and intentions in different grasping actions as well as communicative point-light actions.

Novel Tactile Sensor Technology and Smart Tactile Sensing Systems: A Review

During the last decades, smart tactile sensing systems based on different sensing techniques have been developed due to their high potential in industry and biomedical engineering. However, smart tactile sensing technologies and systems are still in their infancy, as many technological and system issues remain unresolved and require strong interdisciplinary efforts to address them. This paper provides an overview of smart tactile sensing systems, with a focus on signal processing technologies used to interpret the measured information from tactile sensors and/or sensors for other sensory modalities. The tactile sensing transduction and principles, fabrication and structures are also discussed with their merits and demerits. Finally, the challenges that tactile sensing technology needs to overcome are highlighted.

Asymptotically Optimal Motion Planning for Learned Tasks Using Time-Dependent Cost Maps

In unstructured environments in people's homes and workspaces, robots executing a task may need to avoid obstacles while satisfying task motion constraints, e.g., keeping a plate of food level to avoid spills or properly orienting a finger to push a button. We introduce a sampling-based method for computing motion plans that are collision-free and minimize a cost metric that encodes task motion constraints. Our time-dependent cost metric, learned from a set of demonstrations, encodes features of a task's motion that are consistent across the demonstrations and, hence, are likely required to successfully execute the task. Our sampling-based motion planner uses the learned cost metric to compute plans that simultaneously avoid obstacles and satisfy task constraints. The motion planner is asymptotically optimal and minimizes the Mahalanobis distance between the planned trajectory and the distribution of demonstrations in a feature space parameterized by the locations of task-relevant objects. The motion planner also leverages the distribution of the demonstrations to significantly reduce plan computation time. We demonstrate the method's effectiveness and speed using a small humanoid robot performing tasks requiring both obstacle avoidance and satisfaction of learned task constraints.

NOTE TO PRACTITIONERS

Motivated by the desire to enable robots to autonomously operate in cluttered home and workplace environments, this paper presents an approach for intuitively training a robot in a manner that enables it to repeat the task in novel scenarios and in the presence of unforeseen obstacles in the environment. Based on user-provided demonstrations of the task, our method learns features of the task that are consistent across the demonstrations and that we expect should be repeated by the robot when performing the task. We next present an efficient algorithm for planning robot motions to perform the task based on the learned features while avoiding obstacles. We demonstrate the effectiveness of our motion planner for scenarios requiring transferring a powder and pushing a button in environments with obstacles, and we plan to extend our results to more complex tasks in the future.

Learning Visual Object Categories for Robot Affordance Prediction

A fundamental requirement of any autonomous robot system is the ability to predict the affordances of its environment. The set of affordances define the actions that are available to the agent given the robot’s context. A standard approach to affordance learning is direct perception, which learns direct mappings from sensor measurements to affordance labels. For example, a robot designed for cross-country navigation could map stereo depth information and image features directly into predictions about the traversability of terrain regions. While this approach can succeed for a small number of affordances, it does not scale well as the number of affordances increases. In this paper, we show that visual object categories can be used as an intermediate representation that makes the affordance learning problem scalable. We develop a probabilistic graphical model which we call the Category—Affordance (CA) model, which describes the relationships between object categories, affordances, and appearance. This model casts visual object categorization as an intermediate inference step in affordance prediction. We describe several novel affordance learning and training strategies that are supported by our new model. Experimental results with indoor mobile robots evaluate these different strategies and demonstrate the advantages of the CA model in affordance learning, especially when learning from limited size data sets.

On Signature Invariants for Effective Motion Trajectory Recognition

Motion trajectory can be an informative and descriptive clue that is suitable for the characterization of motion. Studying motion trajectory for effective motion description and recognition is important in many applications. For instance, motion trajectory can play an im portant role in the representation, recognition and learning of most long-term human or robot actions, behaviors and activities. However, effective trajectory descriptors are lacking and most reported work just uses motion trajectory in its raw data form. In this paper, we propose a novel motion trajectory signature descriptor and study its rich descriptive invariants which benefit effective motion trajectory recognition. These invariants are key measures of the flexibility and effectiveness of a descriptor. Substantial descriptive invariants can be deduced from the proposed trajectory signature, which is attributed to the computational locality of the signature components. We first present the signature definition and its robust implementation. Then the signature's invariants are elaborated. A non-linear inter-signature matching algorithm is developed to measure the signature's similarity for trajectory recognition. Experiments are conducted to recognize human sign language, in which both synthetic and real data are used to verify the signature's invariants, and to illustrate the effectiveness in the signature recognition.

Incremental Learning, Clustering and Hierarchy Formation of Whole Body Motion Patterns using Adaptive Hidden Markov Chains

This paper describes a novel approach for autonomous and incremental learning of motion pattern primitives by observation of human motion. Human motion patterns are abstracted into a dynamic stochastic model, which can be used for both subsequent motion recognition and generation, analogous to the mirror neuron hypothesis in primates. The model size is adaptable based on the discrimination requirements in the associated region of the current knowledge base. A new algorithm for sequentially training the Markov chains is developed, to reduce the computation cost during model adaptation. As new motion patterns are observed, they are incrementally grouped together using hierarchical agglomerative clustering based on their relative distance in the model space. The clustering algorithm forms a tree structure, with specialized motions at the tree leaves, and generalized motions closer to the root. The generated tree structure will depend on the type of training data provided, so that the most specialized motions will be those for which the most training has been received. Tests with motion capture data for a variety of motion primitives demonstrate the efficacy of the algorithm.

Vision-based segmentation of continuous mechanomyographic grasping sequences

In detecting motor related activity from mechanomyographic (MMG) recordings, the acquisition of long, continuous streams of MMG signals is typically preferred over the painstaking collection of individual, isolated contractions. However, a major challenge with continuous collection is the subsequent separation of the MMG data stream into segments representing individual contractions. This paper proposes a method for segmenting continuously recorded MMG data streams using computer vision while providing a highly reduced set of key images for rapid human expert verification. Transverse plane video recordings of functional grasp sequences were synchronized with the acquisition of MMG signals from the forearm. An automatic, vision-based algorithm exploiting skin color detection, motion estimation, and template matching provided segmentation cues for MMG signals arising from multiple grips. The automatic segmentation method tolerated extraneous hand movements, differentiated among multiple grips and estimated grip transition times. Our implementation segmented two grips with an average accuracy of 97.8 -/+ 4%, and up to seven grips with an accuracy of 73 -/+ 20%. The automatically extracted contraction initiation and termination times were within 173 -/+ 133 ms of the times obtained via manual segmentation. It is suggested that the proposed method would be particularly conducive to the assembly of large collections of signals for training MMG-driven prostheses.