CAM-Vtrans: real-time sports training utilizing multi-modal robot data

Introduction

Assistive robots and human-robot interaction have become integral parts of sports training. However, existing methods often fail to provide real-time and accurate feedback, and they often lack integration of comprehensive multi-modal data.

Methods

To address these issues, we propose a groundbreaking and innovative approach: CAM-Vtrans-Cross-Attention Multi-modal Visual Transformer. By leveraging the strengths of state-of-the-art techniques such as Visual Transformers (ViT) and models like CLIP, along with cross-attention mechanisms, CAM-Vtrans harnesses the power of visual and textual information to provide athletes with highly accurate and timely feedback. Through the utilization of multi-modal robot data, CAM-Vtrans offers valuable assistance, enabling athletes to optimize their performance while minimizing potential injury risks. This novel approach represents a significant advancement in the field, offering an innovative solution to overcome the limitations of existing methods and enhance the precision and efficiency of sports training programs.

DMPs-based skill learning for redundant dual-arm robotic synchronized cooperative manipulation

Dual-arm robot manipulation is applicable to many domains, such as industrial, medical, and home service scenes. Learning from demonstrations is a highly effective paradigm for robotic learning, where a robot learns from human actions directly and can be used autonomously for new tasks, avoiding the complicated analytical calculation for motion programming. However, the learned skills are not easy to generalize to new cases where special constraints such as varying relative distance limitation of robotic end effectors for human-like cooperative manipulations exist. In this paper, we propose a dynamic movement primitives (DMPs) based skills learning framework for redundant dual-arm robots. The method, with a coupling acceleration term to the DMPs function, is inspired by the transient performance control of Barrier Lyapunov Functions. The additional coupling acceleration term is calculated based on the constant joint distance and varying relative distance limitations of end effectors for object-approaching actions. In addition, we integrate the generated actions in joint space and the solution for a redundant dual-arm robot to complete a human-like manipulation. Simulations undertaken in Matlab and Gazebo environments certify the effectiveness of the proposed method.

A Framework for Composite Layup Skill Learning and Generalizing Through Teleoperation

In this article, an impedance control-based framework for human-robot composite layup skill transfer was developed, and the human-in-the-loop mechanism was investigated to achieve human-robot skill transfer. Although there are some works on human-robot skill transfer, it is still difficult to transfer the manipulation skill to robots through teleoperation efficiently and intuitively. In this article, we developed an impedance-based control architecture of telemanipulation in task space for the human-robot skill transfer through teleoperation. This framework not only achieves human-robot skill transfer but also provides a solution to human-robot collaboration through teleoperation. The variable impedance control system enables the compliant interaction between the robot and the environment, smooth transition between different stages. Dynamic movement primitives based learning from demonstration (LfD) is employed to model the human manipulation skills, and the learned skill can be generalized to different tasks and environments, such as the different shapes of components and different orientations of components. The performance of the proposed approach is evaluated on a 7 DoF Franka Panda through the robot-assisted composite layup on different shapes and orientations of the components.

Review: How Can Intelligent Robots and Smart Mechatronic Modules Facilitate Remote Assessment, Assistance, and Rehabilitation for Isolated Adults With Neuro-Musculoskeletal Conditions?

Worldwide, at the time this article was written, there are over 127 million cases of patients with a confirmed link to COVID-19 and about 2.78 million deaths reported. With limited access to vaccine or strong antiviral treatment for the novel coronavirus, actions in terms of prevention and containment of the virus transmission rely mostly on social distancing among susceptible and high-risk populations. Aside from the direct challenges posed by the novel coronavirus pandemic, there are serious and growing secondary consequences caused by the physical distancing and isolation guidelines, among vulnerable populations. Moreover, the healthcare system's resources and capacity have been focused on addressing the COVID-19 pandemic, causing less urgent care, such as physical neurorehabilitation and assessment, to be paused, canceled, or delayed. Overall, this has left elderly adults, in particular those with neuromusculoskeletal (NMSK) conditions, without the required service support. However, in many cases, such as stroke, the available time window of recovery through rehabilitation is limited since neural plasticity decays quickly with time. Given that future waves of the outbreak are expected in the coming months worldwide, it is important to discuss the possibility of using available technologies to address this issue, as societies have a duty to protect the most vulnerable populations. In this perspective review article, we argue that intelligent robotics and wearable technologies can help with remote delivery of assessment, assistance, and rehabilitation services while physical distancing and isolation measures are in place to curtail the spread of the virus. By supporting patients and medical professionals during this pandemic, robots, and smart digital mechatronic systems can reduce the non-COVID-19 burden on healthcare systems. Digital health and cloud telehealth solutions that can complement remote delivery of assessment and physical rehabilitation services will be the subject of discussion in this article due to their potential in enabling more effective and safer NMSDK rehabilitation, assistance, and assessment service delivery. This article will hopefully lead to an interdisciplinary dialogue between the medical and engineering sectors, stake holders, and policy makers for a better delivery of care for those with NMSK conditions during a global health crisis including future pandemics.

Learning for a Robot: Deep Reinforcement Learning, Imitation Learning, Transfer Learning

Dexterous manipulation of the robot is an important part of realizing intelligence, but manipulators can only perform simple tasks such as sorting and packing in a structured environment. In view of the existing problem, this paper presents a state-of-the-art survey on an intelligent robot with the capability of autonomous deciding and learning. The paper first reviews the main achievements and research of the robot, which were mainly based on the breakthrough of automatic control and hardware in mechanics. With the evolution of artificial intelligence, many pieces of research have made further progresses in adaptive and robust control. The survey reveals that the latest research in deep learning and reinforcement learning has paved the way for highly complex tasks to be performed by robots. Furthermore, deep reinforcement learning, imitation learning, and transfer learning in robot control are discussed in detail. Finally, major achievements based on these methods are summarized and analyzed thoroughly, and future research challenges are proposed.

Comprehensive Review on Reaching and Grasping of Objects in Robotics

Interaction between a robot and its environment requires perception about the environment, which helps the robot in making a clear decision about the object type and its location. After that, the end effector will be brought to the object’s location for grasping. There are many research studies on the reaching and grasping of objects using different techniques and mechanisms for increasing accuracy and robustness during grasping and reaching tasks. Thus, this paper presents an extensive review of research directions and topics of different approaches such as sensing, learning and gripping, which have been implemented within the current five years.