Skill-based human–robot cooperation in tele-operated path tracking

Real-time motion force-feedback system with predictive-vision for improving motor accuracy

Many haptic guidance systems have been studied over the years; however, most of them have been limited to predefined guidance methods. Calculating guidance according to the operator's motion is important for efficient human motor adaptation and learning. In this study, we developed a system that haptically provides guidance trajectory by sequential weighting between the operator's trajectory and the ideal trajectory calculated from a predictive-vision system. We investigated whether motion completion with a predictive-vision system affects human motor accuracy and adaptation in time-constrained goal-directed reaching and ball-hitting tasks through subject experiments. The experiment was conducted with 12 healthy participants, and all participants performed ball-hitting tasks. Half of the participants get forceful guidance from the proposed system in the middle of the experiment. We found that the use of the proposed system improved the operator's motor performance. Furthermore, we observed a trend in which the improvement in motor performance using this system correlated with that after the washout of this system. These results suggest that the predictive-vision system effectively enhances motor accuracy to the target error in dynamic and time-constrained reaching and hitting tasks and may contribute to facilitating motor learning.

A Digital Twin Approach for Contextual Assistance for Surgeons During Surgical Robotics Training

Minimally invasive robotic surgery copes with some disadvantages for the surgeon of minimally invasive surgery while preserving the advantages for the patient. Most commercially available robotic systems are telemanipulated with haptic input devices. The exploitation of the haptics channel, e.g., by means of Virtual Fixtures, would allow for an individualized enhancement of surgical performance with contextual assistance. However, it remains an open field of research as it is non-trivial to estimate the task context itself during a surgery. In contrast, surgical training allows to abstract away from a real operation and thus makes it possible to model the task accurately. The presented approach exploits this fact to parameterize Virtual Fixtures during surgical training, proposing a Shared Control Parametrization Engine that retrieves procedural context information from a Digital Twin. This approach accelerates a proficient use of the robotic system for novice surgeons by augmenting the surgeon's performance through haptic assistance. With this our aim is to reduce the required skill level and cognitive load of a surgeon performing minimally invasive robotic surgery. A pilot study is performed on the DLR MiroSurge system to evaluate the presented approach. The participants are tasked with two benchmark scenarios of surgical training. The execution of the benchmark scenarios requires basic skills as pick, place and path following. The evaluation of the pilot study shows the promising trend that novel users profit from the haptic augmentation during training of certain tasks.

Skill-Oriented and Performance-Driven Adaptive Curricula for Training in Robot-Assisted Surgery Using Simulators: A Feasibility Study

OBJECTIVE

Virtual Reality (VR) simulators represent a remarkable educational opportunity in order to acquire and refine surgical practical skills. Nevertheless, there exists no consensus regarding a standard curriculum of simulation-based training. This study introduces an automatic, adaptive curriculum where the training session is real-time scheduled on the basis of the trainee's performances.

METHODS

An experimental study using the master console of the da Vinci Research Kit (Intuitive Surgical Inc., Sunnyvale, US) was carried out to test this approach. Tasks involving fundamental skills of robotic surgery were designed and simulated in VR. Twelve participants without medical background along with twelve medical residents were randomly and equally divided into two groups: a control group, self-managing the training session, and an experimental group, undergoing the proposed adaptive training.

RESULTS

The performances of the experimental users were significantly better with respect to the ones of the control group after training (non-medical: p < 0.01; medical: p = 0.02). This trend was analogous in the non-medical and medical populations and no significant difference was identified between these two classes (even in the baseline assessment).

CONCLUSION

The analysis of the learning of the involved surgical skills highlighted how the proposed adaptive training managed to better identify and compensate for the trainee's gaps. The absence of initial difference between the non-medical and medical users underlines that robotic surgical devices require specific training before clinical practice.

SIGNIFICANCE

This feasibility study could pave the way towards the improvement of simulation-based training curricula.

Online human-like redundancy optimization for tele-operated anthropomorphic manipulators

Robot human-like behavior can enhance the performance of human–robot cooperation with prominently improved natural interaction. This also holds for redundant robots with an anthropomorphic kinematics. In this article, we translated human ability of managing redundancy to control a seven degrees of freedom anthropomorphic robot arm (LWR4+, KUKA, Germany) during tele-operated tasks. We implemented a nonlinear regression method—based on neural networks—between the human arm elbow swivel angle and the hand target pose to achieve an anthropomorphic arm posture during tele-operation tasks. The method was assessed in simulation and experiments were performed with virtual reality tracking tasks in a lab environment. The results showed that the robot achieves a human-like arm posture during tele-operation, and the subjects prefer to work with the biologically inspired robot. The proposed method can be applied in control of anthropomorphic robot manipulators for tele-operated collaborative tasks, such as in factories or in operating rooms.

An Uncontrolled Manifold Analysis of Arm Joint Variability in Virtual Planar Position and Orientation Telemanipulation

OBJECTIVE

In teleoperated robot-assisted tasks, the user interacts with manipulators to finely control remote tools. Manipulation of robotic devices, characterized by specific kinematic and dynamic proprieties, is a complex task for the human sensorimotor system due to the inherent biomechanical and neuronal redundancies that characterize the human arm and its control. We investigate how master devices with different kinematics structures and how different task constraints influence users capabilities in exploiting arm redundancy.

METHODS

A virtual teleoperation workbench was designed and the arm kinematics of seven users was acquired during the execution of two planar virtual tasks, involving either the control of position only or position-orientation of a tool. Using the uncontrolled manifold analysis of arm joint variability, we estimated the logarithmic ratio between the task irrelevant and the task relevant manifolds ( R_v).

RESULTS

The R_v values obtained in the position-orientation task were higher than in the position only task, while no differences were found between the master devices. A modulation of R_v was found through the execution of the position task and a positive correlation was found between task performance and redundancy exploitation.

CONCLUSION

Users exploited additional portions of arm redundancy when dealing with the tool orientation. The R_v modulation seems influenced by the task constraints and by the users possibility of reconfiguring the arm position.

SIGNIFICANCE

This paper advances the general understanding of the exploitation of arm redundancy in complex tasks, and can improve the development of future robotic devices.