Human-robot planar co-manipulation of extended objects: data-driven models and control from human-human dyads

Human teams are able to easily perform collaborative manipulation tasks. However, simultaneously manipulating a large extended object for a robot and human is a difficult task due to the inherent ambiguity in the desired motion. Our approach in this paper is to leverage data from human-human dyad experiments to determine motion intent for a physical human-robot co-manipulation task. We do this by showing that the human-human dyad data exhibits distinct torque triggers for a lateral movement. As an alternative intent estimation method, we also develop a deep neural network based on motion data from human-human trials to predict future trajectories based on past object motion. We then show how force and motion data can be used to determine robot control in a human-robot dyad. Finally, we compare human-human dyad performance to the performance of two controllers that we developed for human-robot co-manipulation. We evaluate these controllers in three-degree-of-freedom planar motion where determining if the task involves rotation or translation is ambiguous.

Collaborative robots can augment human cognition in regret-sensitive tasks

Despite theoretical benefits of collaborative robots, disappointing outcomes are well documented by clinical studies, spanning rehabilitation, prostheses, and surgery. Cognitive load theory provides a possible explanation for why humans in the real world are not realizing the benefits of collaborative robots: high cognitive loads may be impeding human performance. Measuring cognitive availability using an electrocardiogram, we ask 25 participants to complete a virtual-reality task alongside an invisible agent that determines optimal performance by iteratively updating the Bellman equation. Three robots assist by providing environmental information relevant to task performance. By enabling the robots to act more autonomously-managing more of their own behavior with fewer instructions from the human-here we show that robots can augment participants' cognitive availability and decision-making. The way in which robots describe and achieve their objective can improve the human's cognitive ability to reason about the task and contribute to human-robot collaboration outcomes. Augmenting human cognition provides a path to improve the efficacy of collaborative robots. By demonstrating how robots can improve human cognition, this work paves the way for improving the cognitive capabilities of first responders, manufacturing workers, surgeons, and other future users of collaborative autonomy systems.

Distributed Hierarchical Shared Control for Flexible Multirobot Maneuver Through Dense Undetectable Obstacles

When teleoperating a multirobot system (MRS) in outdoor environments, human operators can often detect obstacles that are not detected by robots and spot emergencies faster than robots do. However, the lack of efficient methods for operators to manipulate an MRS has limited the number of robots in a human-robot team. To handle this problem, a distributed hierarchical shared control scheme is proposed, aiming to provide a safe and flexible control interface for a few human operators to interact with a large MRS. The proposed hierarchical control scheme employs a two-layered structure. In the upper layer, intention field networks are designed to generate virtual human control signals. Two functionalities for human teleoperation, called: 1) group management and 2) motion intervention, are realized using intention fields, allowing the operators to split the robot formation into different groups and steer individual robots away from immediate danger. In parallel, a blending-based shared control algorithm is designed in the lower layer to resolve the conflict between human intervention inputs and autonomous formation control signals. The input-to-output stability (IOS) of the proposed distributed hierarchical shared control scheme is proved by exploiting the properties of weighting functions. Results from a usability testing experiment and a physical experiment are also presented to validate the effectiveness and practicability of the proposed method.

Synthesis With Guaranteed Cost and Less Human Intervention for Human-in-the-Loop Control Systems

This article studies the problem of synthesis with guaranteed cost and less human intervention for linear human-in-the-loop (HiTL) control systems. Initially, the human behaviors are modeled via a hidden controlled Markov process, which not only considers the inference's stochasticity and observation's uncertainty of the human internal state but also takes the control input to human into account. Then, to integrate both models of human and machine as well as their interaction, a hidden controlled Markov jump system (HCMJS) is constructed. With the aid of the stochastic Lyapunov functional together with the bilinear matrix inequality technique, a sufficient condition for the existence of human-assistance controllers is derived on the basis of the HCMJS model, which not only guarantees the stochastic stability of the closed-loop HiTL system but also provides a prescribed upper bound for the quadratic cost function. Moreover, to achieve less human intervention while meeting the desired cost level, an algorithm that mixes the particle swarm optimization and linear matrix inequality technique is proposed to seek a suitable feedback control law to the human and a human-assistance control law to the machine. Finally, the proposed method is applied to a driver-assistance system to verify its effectiveness.

A Confidence-Based Supervised-Autonomous Control Strategy for Robotic Vaginal Cuff Closure

Autonomous robotic suturing has the potential to improve surgery outcomes by leveraging accuracy, repeatability, and consistency compared to manual operations. However, achieving full autonomy in complex surgical environments is not practical and human supervision is required to guarantee safety. In this paper, we develop a confidence-based supervised autonomous suturing method to perform robotic suturing tasks via both Smart Tissue Autonomous Robot (STAR) and surgeon collaboratively with the highest possible degree of autonomy. Via the proposed method, STAR performs autonomous suturing when highly confident and otherwise asks the operator for possible assistance in suture positioning adjustments. We evaluate the accuracy of our proposed control method via robotic suturing tests on synthetic vaginal cuff tissues and compare them to the results of vaginal cuff closures performed by an experienced surgeon. Our test results indicate that by using the proposed confidence-based method, STAR can predict the success of pure autonomous suture placement with an accuracy of 94.74%. Moreover, via an additional 25% human intervention, STAR can achieve a 98.1% suture placement accuracy compared to an 85.4% accuracy of completely autonomous robotic suturing. Finally, our experiment results indicate that STAR using the proposed method achieves 1.6 times better consistency in suture spacing and 1.8 times better consistency in suture bite sizes than the manual results.

A Confidence-Based Shared Control Strategy for the Smart Tissue Autonomous Robot (STAR)

Autonomous robotic assisted surgery (RAS) systems aim to reduce human errors and improve patient outcomes leveraging robotic accuracy and repeatability during surgical procedures. However, full automation of RAS in complex surgical environments is still not feasible and collaboration with the surgeon is required for safe and effective use. In this work, we utilize our Smart Tissue Autonomous Robot (STAR) to develop and evaluate a shared control strategy for the collaboration of the robot with a human operator in surgical scenarios. We consider 2D pattern cutting tasks with partial blood occlusion of the cutting pattern using a robotic electrocautery tool. For this surgical task and RAS system, we i) develop a confidence-based shared control strategy, ii) assess the pattern tracking performances of manual and autonomous controls and identify the confidence models for human and robot as well as a confidence-based control allocation function, and iii) experimentally evaluate the accuracy of our proposed shared control strategy. In our experiments on porcine fat samples, by combining the best elements of autonomous robot controller with complementary skills of a human operator, our proposed control strategy improved the cutting accuracy by 6.4%, while reducing the operator work time to 44 % compared to a pure manual control.