Intuition-guided Reinforcement Learning for Soft Tissue Manipulation with Unknown Constraints

Intraoperative soft tissue manipulation is a critical challenge in autonomous robotic surgery. Furthermore, the intricate in vivo environment surrounding the target soft tissues poses additional hindrances to autonomous robotic decision-making. Previous studies assumed the grasping point was known and the target deformation could be achieved. The constraints were assumed to be constant during the operation, and there were no obstacles around the soft tissue. To address these problems, an intuition-guided deep reinforcement learning framework based on soft actor-critic (ID-SAC) was proposed for soft tissue manipulation under unknown constraints. The SAC algorithm is automatically activated upon encountering an obstacle, and the designed intuitive manipulation (IM) strategy is used to pull soft tissues toward the target shape directly when the obstacle is distant. A regulator factor is designed as an action within this framework to coordinate the IM approach and the SAC network. A reward function is designed to balance the exploration and exploitation of large deformations. Simultaneously, we proposed an autonomous grasp point selection neural network to prevent the impractical selection of grasp points, ensuring they can reach the target while avoiding grasping lesions and constrained areas. Successful simulation results confirmed that the proposed framework can manipulate the soft tissue while avoiding obstacles and adding new positional constraints. Compared with the SAC algorithm, the proposed framework can markedly increase the robotic soft tissue manipulation ability by automatically adjusting the regulator factors.

Autonomous countertraction for secure field of view in laparoscopic surgery using deep reinforcement learning

PURPOSE

Countertraction is a vital technique in laparoscopic surgery, stretching the tissue surface for incision and dissection. Due to the technical challenges and frequency of countertraction, autonomous countertraction has the potential to significantly reduce surgeons' workload. Despite several methods proposed for automation, achieving optimal tissue visibility and tension for incision remains unrealized. Therefore, we propose a method for autonomous countertraction that enhances tissue surface planarity and visibility.

METHODS

We constructed a neural network that integrates a point cloud convolutional neural network (CNN) with a deep reinforcement learning (RL) model. This network continuously controls the forceps position based on the surface shape observed by a camera and the forceps position. RL is conducted in a physical simulation environment, with verification experiments performed in both simulation and phantom environments. The evaluation was performed based on plane error, representing the average distance between the tissue surface and its least-squares plane, and angle error, indicating the angle between the tissue surface vector and the camera's optical axis vector.

RESULTS

The plane error decreased under all conditions both simulation and phantom environments, with 93.3% of case showing a reduction in angle error. In simulations, the plane error decreased from 3.6 ± 1.5 mm to 1.1 ± 1.8 mm , and the angle error from 29 ± 19 ∘ to 14 ± 13 ∘ . In the phantom environment, the plane error decreased from 0.96 ± 0.24 mm to 0.39 ± 0.23 mm , and the angle error from 32 ± 29 ∘ to 17 ± 20 ∘ .

CONCLUSION

The proposed neural network was validated in both simulation and phantom experimental settings, confirming that traction control improved tissue planarity and visibility. These results demonstrate the feasibility of automating countertraction using the proposed model.

Adaptive Online Learning and Robust 3-D Shape Servoing of Continuum and Soft Robots in Unstructured Environments

In this article, we present a novel and generic data-driven method to servo-control the 3-D shape of continuum and soft robots based on proprioceptive sensing feedback. Developments of 3-D shape perception and control technologies are crucial for continuum and soft robots to perform tasks autonomously in surgical interventions. However, owing to the nonlinear properties of continuum robots, one main difficulty lies in the modeling of them, especially for soft robots with variable stiffness. To address this problem, we propose a versatile learning-based adaptive shape controller by leveraging proprioception of 3-D configuration from fiber Bragg grating (FBG) sensors, which can online estimate the unknown model of continuum robot against unexpected disturbances and exhibit an adaptive behavior to the unmodeled system without priori data exploration. Based on a new composite adaptation algorithm, the asymptotic convergences of the closed-loop system with learning parameters have been proven by Lyapunov theory. To validate the proposed method, we present a comprehensive experimental study using two continuum and soft robots both integrated with multicore FBGs, including a robotic-assisted colonoscope and multisection extensible soft manipulators. The results demonstrate the feasibility, adaptability, and superiority of our controller in various unstructured environments, as well as phantom experiments.

Concepts and Trends n Autonomy for Robot-Assisted Surgery

Surgical robots have been widely adopted with over 4000 robots being used in practice daily. However, these are telerobots that are fully controlled by skilled human surgeons. Introducing "surgeon-assist"-some forms of autonomy-has the potential to reduce tedium and increase consistency, analogous to driver-assist functions for lanekeeping, cruise control, and parking. This article examines the scientific and technical backgrounds of robotic autonomy in surgery and some ethical, social, and legal implications. We describe several autonomous surgical tasks that have been automated in laboratory settings, and research concepts and trends.

Control of differential-drive mobile robots for soft object deformation

The aim of this work is a control scheme implementation to deform a nonrigid object in which deformation dynamics are modeled by the finite element method. The deformation of a soft object is highly difficult to model because of its non-linearity, time-dependency, and material-response characteristics. Thus, the control implementation for Differential Drive Mobile Robots (DDMR) to deform an elastic object, is a challenge. The proposed steps to solve it are: Position-control designed over DDMR kinematics. Alignment-control applied for DDMRs orientation. The desired shape of the object is achieved using two contact points as the control nodes. A centralized vision algorithm was employed in each stage to obtain positions. To show the usefulness of the proposed scheme, numerical simulation, and real-time implementation were carried out.

Modeling, learning, perception, and control methods for deformable object manipulation

Perceiving and handling deformable objects is an integral part of everyday life for humans. Automating tasks such as food handling, garment sorting, or assistive dressing requires open problems of modeling, perceiving, planning, and control to be solved. Recent advances in data-driven approaches, together with classical control and planning, can provide viable solutions to these open challenges. In addition, with the development of better simulation environments, we can generate and study scenarios that allow for benchmarking of various approaches and gain better understanding of what theoretical developments need to be made and how practical systems can be implemented and evaluated to provide flexible, scalable, and robust solutions. To this end, we survey more than 100 relevant studies in this area and use it as the basis to discuss open problems. We adopt a learning perspective to unify the discussion over analytical and data-driven approaches, addressing how to use and integrate model priors and task data in perceiving and manipulating a variety of deformable objects.

Blind Manipulation of Deformable Objects Based on Force Sensing and Finite Element Modeling

In this paper, we present a novel pipeline to simultaneously estimate and manipulate the deformation of an object using only force sensing and an FEM model. The pipeline is composed of a sensor model, a deformation model and a pose controller. The sensor model computes the contact forces that are used as input to the deformation model which updates the volumetric mesh of a manipulated object. The controller then deforms the object such that a given pose on the mesh reaches a desired pose. The proposed approach is thoroughly evaluated in real experiments using a robot manipulator and a force-torque sensor to show its accuracy in estimating and manipulating deformations without the use of vision sensors.

A Lyapunov-Stable Adaptive Method to Approximate Sensorimotor Models for Sensor-Based Control

In this article, we present a new scheme that approximates unknown sensorimotor models of robots by using feedback signals only. The formulation of the uncalibrated sensor-based regulation problem is first formulated, then, we develop a computational method that distributes the model estimation problem amongst multiple adaptive units that specialize in a local sensorimotor map. Different from traditional estimation algorithms, the proposed method requires little data to train and constrain it (the number of required data points can be analytically determined) and has rigorous stability properties (the conditions to satisfy Lyapunov stability are derived). Numerical simulations and experimental results are presented to validate the proposed method.

Modeling of Deformable Objects for Robotic Manipulation: A Tutorial and Review

Manipulation of deformable objects has given rise to an important set of open problems in the field of robotics. Application areas include robotic surgery, household robotics, manufacturing, logistics, and agriculture, to name a few. Related research problems span modeling and estimation of an object's shape, estimation of an object's material properties, such as elasticity and plasticity, object tracking and state estimation during manipulation, and manipulation planning and control. In this survey article, we start by providing a tutorial on foundational aspects of models of shape and shape dynamics. We then use this as the basis for a review of existing work on learning and estimation of these models and on motion planning and control to achieve desired deformations. We also discuss potential future lines of work.

Model-Based Manipulation of Linear Flexible Objects: Task Automation in Simulation and Real World

Manipulation of deformable objects is a desired skill in making robots ubiquitous in manufacturing, service, healthcare, and security. Common deformable objects (e.g., wires, clothes, bed sheets, etc.) are significantly more difficult to model than rigid objects. In this research, we contribute to the model-based manipulation of linear flexible objects such as cables. We propose a 3D geometric model of the linear flexible object that is subject to gravity and a physical model with multiple links connected by revolute joints and identified model parameters. These models enable task automation in manipulating linear flexible objects both in simulation and real world. To bridge the gap between simulation and real world and build a close-to-reality simulation of flexible objects, we propose a new strategy called Simulation-to-Real-to-Simulation (Sim2Real2Sim). We demonstrate the feasibility of our approach by completing the Plug Task used in the 2015 DARPA Robotics Challenge Finals both in simulation and real world, which involves unplugging a power cable from one socket and plugging it into another. Numerical experiments are implemented to validate our approach.

Manipulating deformable objects by interleaving prediction, planning, and control

We present a framework for deformable object manipulation that interleaves planning and control, enabling complex manipulation tasks without relying on high-fidelity modeling or simulation. The key question we address is when should we use planning and when should we use control to achieve the task? Planners are designed to find paths through complex configuration spaces, but for highly underactuated systems, such as deformable objects, achieving a specific configuration is very difficult even with high-fidelity models. Conversely, controllers can be designed to achieve specific configurations, but they can be trapped in undesirable local minima owing to obstacles. Our approach consists of three components: (1) a global motion planner to generate gross motion of the deformable object; (2) a local controller for refinement of the configuration of the deformable object; and (3) a novel deadlock prediction algorithm to determine when to use planning versus control. By separating planning from control we are able to use different representations of the deformable object, reducing overall complexity and enabling efficient computation of motion. We provide a detailed proof of probabilistic completeness for our planner, which is valid despite the fact that our system is underactuated and we do not have a steering function. We then demonstrate that our framework is able to successfully perform several manipulation tasks with rope and cloth in simulation, which cannot be performed using either our controller or planner alone. These experiments suggest that our planner can generate paths efficiently, taking under a second on average to find a feasible path in three out of four scenarios. We also show that our framework is effective on a 16-degree-of-freedom physical robot, where reachability and dual-arm constraints make the planning more difficult.

Design and Kinematic Control of the Cable-Driven Hyper-Redundant Manipulator for Potential Underwater Applications

Underwater manipulators are important robotic tools in the exploration of the ocean environment. Up to now, most existing underwater manipulators are rigid and with fixed 5 or 7 degrees of freedom (DOF), which may not be very suitable for some complicated underwater scenarios (e.g., pipe networks, narrow deep cavities, etc.). The biomimetic concept of muscles and tendons is also considered as continuum manipulators, but load capacity and operation accuracy are their essential drawbacks and thus limit their practical applications. Recently, the cable-driven technique has been developed for manipulators, which can include numerous joints and hyper-redundant DOF to execute tasks with dexterity and adaptability and thus they have strong potential for these complex underwater applications. In this paper, the design of a novel cable-driven hyper-redundant manipulator (CDHRM) is introduced, which is driven by multiple cables passing through the tubular structure from the base to the end-effector, and the joint numbers can be extended and decided by the specific underwater task requirements. The kinematic analysis of the proposed CDHRM is given which includes two parts: the cable-joint kinematics and the joint-end kinematics. The geometric relationship between the cable length and the joint angles are derived via the established geometric model for the cable-joint kinematics, and the projection relationship between the joint angles and end-effector’s pose is established via the spatial coordinate transformation matrix for the joint-end kinematics. Thus, the complex mapping relationships among the cables, joints and end-effectors are clearly achieved. To implement precise control, the kinematic control scheme is developed for the CDHRM with series-parallel connections and hyper-redundancy to achieve good tracking performance. The experiment on a real CDHRM system with five joints is carried out and the results verify the accuracy of kinematics solution, and the effectiveness of the proposed control design. Particularly, three experiments are tested in the underwater environment, which verifies its good tracking performance, load carrying and grasping capacity.

Multi-Modal Sensing and Robotic Manipulation of Non-Rigid Objects: A Survey

This paper aims to provide a comprehensive survey of recent advancements in modelling and autonomous manipulation of non-rigid objects. It first summarizes the recent advances in sensing and modelling of such objects with a focus on describing the methods and technologies used to measure their shape and estimate their material and physical properties. Formal representations considered to predict the deformation resulting from manipulation of non-rigid objects are then investigated. The third part provides a survey of planning and control strategies exploited to operate dexterous robotic systems while performing various tasks on objects made of different non-rigid materials.

Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey

We present a survey of recent work on robot manipulation and sensing of deformable objects, a field with relevant applications in diverse industries such as medicine (e.g. surgical assistance), food handling, manufacturing, and domestic chores (e.g. folding clothes). We classify the reviewed approaches into four categories based on the type of object they manipulate. Furthermore, within this object classification, we divide the approaches based on the particular task they perform on the deformable object. Finally, we conclude this survey with a discussion of the current state-of-the-art approaches and propose future directions within the proposed classification.