Using a human action database to recognize actions in monocular image sequences: recovering human whole body configurations

CNN-Based Hand Grasping Prediction and Control via Postural Synergy Basis Extraction

The prediction of hand grasping and control of a robotic manipulator for hand activity training is of great significance to assist stroke patients to recover their biomechanical functions. However, the human hand and the figure joints have multiple degrees of freedom; therefore, it is complex to process and analyze all the collected data in hand modeling. To simplify the description of grasping activities, it is necessary to extract and decompose the principal components of hand actions. In this paper, the relationships among hand grasping actions are explored by extracting the postural synergy basis of hand motions, aiming to simplify hand grasping actions and reduce the data dimensions for robot control. A convolutional neural network (CNN)-based hand activity prediction method is proposed, which utilizes motion data to estimate hand grasping actions. The prediction results were then used to control a stimulated robotic model according to the extracted postural synergy basis. The prediction accuracy of the proposed method for the selected hand motions could reach up to 94% and the robotic model could be operated naturally based on patient’s movement intention, so as to complete grasping tasks and achieve active rehabilitation.

Control and Guidance of Low-Cost Robots via Gesture Perception for Monitoring Activities in the Home

This paper describes the development of a low-cost mini-robot that is controlled by visual gestures. The prototype allows a person with disabilities to perform visual inspections indoors and in domestic spaces. Such a device could be used as the operator's eyes obviating the need for him to move about. The robot is equipped with a motorised webcam that is also controlled by visual gestures. This camera is used to monitor tasks in the home using the mini-robot while the operator remains quiet and motionless. The prototype was evaluated through several experiments testing the ability to use the mini-robot’s kinematics and communication systems to make it follow certain paths. The mini-robot can be programmed with specific orders and can be tele-operated by means of 3D hand gestures to enable the operator to perform movements and monitor tasks from a distance.