Force Myography-Based Human Robot Interactions via Deep Domain Adaptation and Generalization

Detecting Safety Anomalies in pHRI Activities via Force Myography

The potential application of using a wearable force myography (FMG) band for monitoring the occupational safety of a human participant working in collaboration with an industrial robot was studied. Regular physical human–robot interactions were considered as activities of daily life in pHRI (pHRI-ADL) to recognize human-intended motions during such interactions. The force myography technique was used to read volumetric changes in muscle movements while a human participant interacted with a robot. Data-driven models were used to observe human activities for useful insights. Using three unsupervised learning algorithms, isolation forest, one-class SVM, and Mahalanobis distance, models were trained to determine pHRI-ADL/regular, preset activities by learning the latent features’ distributions. The trained models were evaluated separately to recognize any unwanted interactions that differed from the normal activities, i.e., anomalies that were novel, inliers, or outliers to the normal distributions. The models were able to detect unusual, novel movements during a certain scenario that was considered an unsafe interaction. Once a safety hazard was detected, the control system generated a warning signal within seconds of the event. Hence, this study showed the viability of using FMG biofeedback to indicate risky interactions to prevent injuries, improve occupational health, and monitor safety in workplaces that require human participation.

Dataset on Force Myography for Human–Robot Interactions

Force myography (FMG) is a contemporary, non-invasive, wearable technology that can read the underlying muscle volumetric changes during muscle contractions and expansions. The FMG technique can be used in recognizing human applied hand forces during physical human robot interactions (pHRI) via data-driven models. Several FMG-based pHRI studies were conducted in 1D, 2D and 3D during dynamic interactions between a human participant and a robot to realize human applied forces in intended directions during certain tasks. Raw FMG signals were collected via 16-channel (forearm) and 32-channel (forearm and upper arm) FMG bands while interacting with a biaxial stage (linear robot) and a serial manipulator (Kuka robot). In this paper, we present the datasets and their structures, the pHRI environments, and the collaborative tasks performed during the studies. We believe these datasets can be useful in future studies on FMG biosignal-based pHRI control design.

Deep Unsupervised Domain Adaptation with Time Series Sensor Data: A Survey

Sensors are devices that output signals for sensing physical phenomena and are widely used in all aspects of our social production activities. The continuous recording of physical parameters allows effective analysis of the operational status of the monitored system and prediction of unknown risks. Thanks to the development of deep learning, the ability to analyze temporal signals collected by sensors has been greatly improved. However, models trained in the source domain do not perform well in the target domain due to the presence of domain gaps. In recent years, many researchers have used deep unsupervised domain adaptation techniques to address the domain gap between signals collected by sensors in different scenarios, i.e., using labeled data in the source domain and unlabeled data in the target domain to improve the performance of models in the target domain. This survey first summarizes the background of recent research on unsupervised domain adaptation with time series sensor data, the types of sensors used, the domain gap between the source and target domains, and commonly used datasets. Then, the paper classifies and compares different unsupervised domain adaptation methods according to the way of adaptation and summarizes different adaptation settings based on the number of source and target domains. Finally, this survey discusses the challenges of the current research and provides an outlook on future work. This survey systematically reviews and summarizes recent research on unsupervised domain adaptation for time series sensor data to provide the reader with a systematic understanding of the field.