A Holistic Indirect Contact Identification Method for Soft Robot Proprioception

Soft robots hold great promise but are notoriously difficult to control due to their compliance and back-drivability. In order to implement useful controllers, improved methods of perceiving robot pose (position and orientation of the entire robot body) in free and perturbed states are needed. In this work, we present a holistic approach to robot pose perception in free bending and with external contact, using multiple soft strain sensors on the robot (not collocated with the point of contact). By comparing the deviation of these sensors from their value in an unperturbed pose, we are able to perceive the mode and magnitude of deformation and thereby estimate the resulting perturbed pose of the soft actuator. We develop a sample 2 degree-of-freedom soft finger with two sensors, and we characterize sensor response to front, lateral, and twist deformation to perceive the mode and magnitude of external perturbation. We develop a data-driven model of free-bending deformation, we impose our perturbation perception method, and we demonstrate the ability to perceive perturbed pose on a single-finger and a two-finger gripper. Our holistic contact identification method provides a generalizable approach to perturbed pose perception needed for the control of soft robots.

Adaptive Online Learning and Robust 3-D Shape Servoing of Continuum and Soft Robots in Unstructured Environments

In this article, we present a novel and generic data-driven method to servo-control the 3-D shape of continuum and soft robots based on proprioceptive sensing feedback. Developments of 3-D shape perception and control technologies are crucial for continuum and soft robots to perform tasks autonomously in surgical interventions. However, owing to the nonlinear properties of continuum robots, one main difficulty lies in the modeling of them, especially for soft robots with variable stiffness. To address this problem, we propose a versatile learning-based adaptive shape controller by leveraging proprioception of 3-D configuration from fiber Bragg grating (FBG) sensors, which can online estimate the unknown model of continuum robot against unexpected disturbances and exhibit an adaptive behavior to the unmodeled system without priori data exploration. Based on a new composite adaptation algorithm, the asymptotic convergences of the closed-loop system with learning parameters have been proven by Lyapunov theory. To validate the proposed method, we present a comprehensive experimental study using two continuum and soft robots both integrated with multicore FBGs, including a robotic-assisted colonoscope and multisection extensible soft manipulators. The results demonstrate the feasibility, adaptability, and superiority of our controller in various unstructured environments, as well as phantom experiments.

A Deep Learning Framework for Soft Robots with Synthetic Data

Data-driven methods with deep neural networks demonstrate promising results for accurate modeling in soft robots. However, deep neural network models rely on voluminous data in discovering the complex and nonlinear representations inherent in soft robots. Consequently, while it is not always possible, a substantial amount of effort is required for data acquisition, labeling, and annotation. This article introduces a data-driven learning framework based on synthetic data to circumvent the exhaustive data collection process. More specifically, we propose a novel time series generative adversarial network with a self-attention mechanism, Transformer TimeGAN (TTGAN) to precisely learn the complex dynamics of a soft robot. On top of that, the TTGAN is incorporated with a conditioning network that enables it to produce synthetic data for specific soft robot behaviors. The proposed framework is verified on a widely used pneumatic-based soft gripper as an exemplary experimental setup. Experimental results demonstrate that the TTGAN generates synthetic time series data with realistic soft robot dynamics. Critically, a combination of the synthetic and only partially available original data produces a data-driven model with estimation accuracy comparable to models obtained from using complete original data.

A review on model-based and model-free approaches to control soft actuators and their potentials in colonoscopy

Colorectal cancer (CRC) is the third most common cancer worldwide and responsible for approximately 1 million deaths annually. Early screening is essential to increase the chances of survival, and it can also reduce the cost of treatments for healthcare centres. Colonoscopy is the gold standard for CRC screening and treatment, but it has several drawbacks, including difficulty in manoeuvring the device, patient discomfort, and high cost. Soft endorobots, small and compliant devices thatcan reduce the force exerted on the colonic wall, offer a potential solution to these issues. However, controlling these soft robots is challenging due to their deformable materials and the limitations of mathematical models. In this Review, we discuss model-free and model-based approaches for controlling soft robots that can potentially be applied to endorobots for colonoscopy. We highlight the importance of selecting appropriate control methods based on various parameters, such as sensor and actuator solutions. This review aims to contribute to the development of smart control strategies for soft endorobots that can enhance the effectiveness and safety of robotics in colonoscopy. These strategies can be defined based on the available information about the robot and surrounding environment, control demands, mechanical design impact and characterization data based on calibration.

Soft DAgger: Sample-Efficient Imitation Learning for Control of Soft Robots

This paper presents Soft DAgger, an efficient imitation learning-based approach for training control solutions for soft robots. To demonstrate the effectiveness of the proposed algorithm, we implement it on a two-module soft robotic arm involved in the task of writing letters in 3D space. Soft DAgger uses a dynamic behavioral map of the soft robot, which maps the robot's task space to its actuation space. The map acts as a teacher and is responsible for predicting the optimal actions for the soft robot based on its previous state action history, expert demonstrations, and current position. This algorithm achieves generalization ability without depending on costly exploration techniques or reinforcement learning-based synthetic agents. We propose two variants of the control algorithm and demonstrate that good generalization capabilities and improved task reproducibility can be achieved, along with a consistent decrease in the optimization time and samples. Overall, Soft DAgger provides a practical control solution to perform complex tasks in fewer samples with soft robots. To the best of our knowledge, our study is an initial exploration of imitation learning with online optimization for soft robot control.

A Geometric Approach towards Inverse Kinematics of Soft Extensible Pneumatic Actuators Intended for Trajectory Tracking

Soft robots are interesting examples of hyper-redundancy in robotics. However, the nonlinear continuous dynamics of these robots and the use of hyper-elastic and visco-elastic materials make modeling these robots more complicated. This study presents a geometric inverse kinematics (IK) model for trajectory tracking of multi-segment extensible soft robots, where each segment of the soft actuator is geometrically approximated with a rigid links model to reduce the complexity. In this model, the links are connected with rotary and prismatic joints, which enable both the extension and rotation of the robot. Using optimization methods, the desired configuration variables of the soft actuator for the desired end-effector positions were obtained. Furthermore, the redundancy of the robot is applied for second task applications, such as tip angle control. The model's performance was investigated through kinematics and dynamics simulations and numerical benchmarks on multi-segment soft robots. The results showed lower computational costs and higher accuracy compared to most existing models. The method is easy to apply to multi-segment soft robots in both 2D and 3D, and it was experimentally validated on 3D-printed soft robotic manipulators. The results demonstrated the high accuracy in path following using this technique.

SoftSAR: The New Softer Side of Socially Assistive Robots-Soft Robotics with Social Human-Robot Interaction Skills

When we think of "soft" in terms of socially assistive robots (SARs), it is mainly in reference to the soft outer shells of these robots, ranging from robotic teddy bears to furry robot pets. However, soft robotics is a promising field that has not yet been leveraged by SAR design. Soft robotics is the incorporation of smart materials to achieve biomimetic motions, active deformations, and responsive sensing. By utilizing these distinctive characteristics, a new type of SAR can be developed that has the potential to be safer to interact with, more flexible, and uniquely uses novel interaction modes (colors/shapes) to engage in a heighted human-robot interaction. In this perspective article, we coin this new collaborative research area as SoftSAR. We provide extensive discussions on just how soft robotics can be utilized to positively impact SARs, from their actuation mechanisms to the sensory designs, and how valuable they will be in informing future SAR design and applications. With extensive discussions on the fundamental mechanisms of soft robotic technologies, we outline a number of key SAR research areas that can benefit from using unique soft robotic mechanisms, which will result in the creation of the new field of SoftSAR.

A Comprehensive Review of Vision-Based Robotic Applications: Current State, Components, Approaches, Barriers, and Potential Solutions

Being an emerging technology, robotic manipulation has encountered tremendous advancements due to technological developments starting from using sensors to artificial intelligence. Over the decades, robotic manipulation has advanced in terms of the versatility and flexibility of mobile robot platforms. Thus, robots are now capable of interacting with the world around them. To interact with the real world, robots require various sensory inputs from their surroundings, and the use of vision is rapidly increasing nowadays, as vision is unquestionably a rich source of information for a robotic system. In recent years, robotic manipulators have made significant progress towards achieving human-like abilities. There is still a large gap between human and robot dexterity, especially when it comes to executing complex and long-lasting manipulations. This paper comprehensively investigates the state-of-the-art development of vision-based robotic application, which includes the current state, components, and approaches used along with the algorithms with respect to the control and application of robots. Furthermore, a comprehensive analysis of those vision-based applied algorithms, their effectiveness, and their complexity has been enlightened here. To conclude, there is a discussion over the constraints while performing the research and potential solutions to develop a robust and accurate vision-based robot manipulation.

Design of a Lightweight and Deployable Soft Robotic Arm

Soft robotics represents a rising trend in recent years, due to the ability to work in unstructured environments or in strict contact with humans. Introducing soft parts, robots can adapt to various contexts overcoming limits relative to the rigid structure of traditional ones. Main issues of soft robotics systems concern the relatively low force exertion and control complexity. Moreover, several fields of application, as space industry, need to develop novel lightweight and deployable robotic systems, that can be stored into a relatively small volume and deployed when required. In this paper, POPUP robot is introduced: a soft manipulator having inflatable links and rigid joints. Its hybrid structure aims to match the advantages of rigid robots and the useful properties of having a lightweight and deployable parts, ensuring simple control, low energy consumption and low compressed gas requirement. The first robot prototype and the system architecture are described highlighting design criteria and effect of internal pressure on the performances. A pseudo-rigid body model is used to describe the behavior of inflatable links looking forward to control design. Finally, the model is extended to the whole robot: multi-body simulations are performed to highlight the importance of suitable sensor equipment for control development, proposing a visual servoing solution.

Task Autonomy of a Flexible Endoscopic System for Laser-Assisted Surgery

Laser beam steering has been widely studied for the automation of surgery. Currently, flexible instruments for laser surgery are operated entirely by surgeons, which keeps the automation of endoluminal surgery at the initial level. This paper introduces the design of a new workflow that enables the task autonomy of laser-assisted surgery in constrained environments such as the gastrointestinal (GI) tract with a flexible continuum robotic system. Unlike current, laser steering systems driven by piezoelectric require the use of high voltage and are risky. This paper describes a tendon-driven 2 mm diameter flexible manipulator integrated with an endoscope to steer the laser beam. By separating its motion from the total endoscopic system, the designed flexible manipulator can automatically manipulate the laser beam. After the surgical site is searched by the surgeon with a master/slave control, a population-based model-free control method is applied for the flexible manipulator to achieve accurate laser beam steering while overcoming the noise from the visual feedback and disturbances from environment during operation. Simulations and experiments are performed with the system and control methods to demonstrate the proposed framework in a simulated constrained environment.

A Survey on Design, Actuation, Modeling, and Control of Continuum Robot

In this paper, we describe the advances in the design, actuation, modeling, and control field of continuum robots. After decades of pioneering research, many innovative structural design and actuation methods have arisen. Untethered magnetic robots are a good example; its external actuation characteristic allows for miniaturization, and they have gotten a lot of interest from academics. Furthermore, continuum robots with proprioceptive abilities are also studied. In modeling, modeling approaches based on continuum mechanics and geometric shaping hypothesis have made significant progress after years of research. Geometric exact continuum mechanics yields apparent computing efficiency via discrete modeling when combined with numerical analytic methods such that many effective model-based control methods have been realized. In the control, closed-loop and hybrid control methods offer great accuracy and resilience of motion control when combined with sensor feedback information. On the other hand, the advancement of machine learning has made modeling and control of continuum robots easier. The data-driven modeling technique simplifies modeling and improves anti-interference and generalization abilities. This paper discusses the current development and challenges of continuum robots in the above fields and provides prospects for the future.

Visual servoing of continuum robots: Methods, challenges, and prospects

BACKGROUND

Recent advancements in continuum robotics have accentuated developing efficient and stable controllers to handle shape deformation and compliance. The control of continuum robots (CRs) using physical sensors attached to the robot, particularly in confined spaces, is difficult due to their limited accuracy in three-dimensional deflections and challenging localisation. Therefore, using non-contact imaging sensors finds noticeable importance, particularly in medical scenarios. Accordingly, given the need for direct control of the robot tip and notable uncertainties in the kinematics and dynamics of CRs, many papers have focussed on the visual servoing (VS) of CRs in recent years.

METHODS

The significance of this research towards safe human-robot interaction has fuelled our survey on the previous methods, current challenges, and future opportunities.

RESULTS

Beginning with actuation modalities and modelling approaches, the paper investigates VS methods in medical and non-medical scenarios.

CONCLUSIONS

Finally, challenges and prospects of VS for CRs are discussed, followed by concluding remarks.

Review on generic methods for mechanical modeling, simulation and control of soft robots

In this review paper, we are interested in the models and algorithms that allow generic simulation and control of a soft robot. First, we start with a quick overview of modeling approaches for soft robots and available methods for calculating the mechanical compliance, and in particular numerical methods, like real-time Finite Element Method (FEM). We also show how these models can be updated based on sensor data. Then, we are interested in the problem of inverse kinematics, under constraints, with generic solutions without assumption on the robot shape, the type, the placement or the redundancy of the actuators, the material behavior… We are also interested by the use of these models and algorithms in case of contact with the environment. Moreover, we refer to dynamic control algorithms based on mechanical models, allowing for robust control of the positioning of the robot. For each of these aspects, this paper gives a quick overview of the existing methods and a focus on the use of FEM. Finally, we discuss the implementation and our contribution in the field for an open soft robotics research.

A Survey for Machine Learning-Based Control of Continuum Robots

Soft continuum robots have been accepted as a promising category of biomedical robots, accredited to the robots’ inherent compliance that makes them safely interact with their surroundings. In its application of minimally invasive surgery, such a continuum concept shares the same view of robotization for conventional endoscopy/laparoscopy. Different from rigid-link robots with accurate analytical kinematics/dynamics, soft robots encounter modeling uncertainties due to intrinsic and extrinsic factors, which would deteriorate the model-based control performances. However, the trade-off between flexibility and controllability of soft manipulators may not be readily optimized but would be demanded for specific kinds of modeling approaches. To this end, data-driven modeling strategies making use of machine learning algorithms would be an encouraging way out for the control of soft continuum robots. In this article, we attempt to overview the current state of kinematic/dynamic model-free control schemes for continuum manipulators, particularly by learning-based means, and discuss their similarities and differences. Perspectives and trends in the development of new control methods are also investigated through the review of existing limitations and challenges.

Soft robotic manipulator for intraoperative MRI-guided transoral laser microsurgery

Magnetic resonance (MR) imaging (MRI) provides compelling features for the guidance of interventional procedures, including high-contrast soft tissue imaging, detailed visualization of physiological changes, and thermometry. Laser-based tumor ablation stands to benefit greatly from MRI guidance because 3D resection margins alongside thermal distributions can be evaluated in real time to protect critical structures while ensuring adequate resection margins. However, few studies have investigated the use of projection-based lasers like those for transoral laser microsurgery, potentially because dexterous laser steering is required at the ablation site, raising substantial challenges in the confined MRI bore and its strong magnetic field. Here, we propose an MR-safe soft robotic system for MRI-guided transoral laser microsurgery. Owing to its miniature size (Ø12 × 100 mm), inherent compliance, and five degrees of freedom, the soft robot ensures zero electromagnetic interference with MRI and enables safe and dexterous operation within the confined oral and pharyngeal cavities. The laser manipulator is rapidly fabricated with hybrid soft and hard structures and is powered by microvolume (<0.004 milliter) fluid flow to enable laser steering with enhanced stiffness and lowered hysteresis. A learning-based controller accommodates the inherent nonlinear robot actuation, which was validated with laser path-following tests. Submillimeter laser steering accuracy was demonstrated with a mean error < 0.20 mm. MRI compatibility testing demonstrated zero observable image artifacts during robot operation. Ex vivo tissue ablation and a cadaveric head-and-neck trial were carried out under MRI, where we employed MR thermometry to monitor the tissue ablation margin and thermal diffusion intraoperatively.

Control and benchmarking of a 7-DOF robotic arm using Gazebo and ROS

The robot controller plays an important role in controlling the robot. The controller mainly aims to eliminate or suppress the influence of uncertain factors on the control robot. Furthermore, there are many types of controllers, and different types of controllers have different features. To explore the differences between controllers of the same category, this article studies some controllers from basic controllers and advanced controllers. This article conducts the benchmarking of the selected controller through pre-set tests. The test task is the most commonly used pick and place. Furthermore, to complete the robustness test, a task of external force interference is also set to observe whether the controller can control the robot arm to return to a normal state. Subsequently, the accuracy, control efficiency, jitter and robustness of the robot arm controlled by the controller are analyzed by comparing the Position and Effort data. Finally, some future works of the benchmarking and reasonable improvement methods are discussed.

Review of machine learning methods in soft robotics

Soft robots have been extensively researched due to their flexible, deformable, and adaptive characteristics. However, compared to rigid robots, soft robots have issues in modeling, calibration, and control in that the innate characteristics of the soft materials can cause complex behaviors due to non-linearity and hysteresis. To overcome these limitations, recent studies have applied various approaches based on machine learning. This paper presents existing machine learning techniques in the soft robotic fields and categorizes the implementation of machine learning approaches in different soft robotic applications, which include soft sensors, soft actuators, and applications such as soft wearable robots. An analysis of the trends of different machine learning approaches with respect to different types of soft robot applications is presented; in addition to the current limitations in the research field, followed by a summary of the existing machine learning methods for soft robots.

Model-Based Control and External Load Estimation of an Extensible Soft Robotic Arm

Soft robotics has widely been known for its compliant characteristics when dealing with contraction or manipulation. These soft behavior patterns provide safe and adaptive interactions, greatly relieving the complexity of active control policies. However, another promising aspect of soft robotics, which is to achieve useful information from compliant behavior, is not widely studied. This characteristic could help to reduce the dependence of sensors, gain a better knowledge of the environment, and enrich high-level control strategies. In this paper, we have developed a state-change model of a soft robotic arm, and we demonstrate how compliant behavior could be used to estimate external load based on this model. Moreover, we propose an improved version of the estimation procedure, further reducing the estimation error by compensating the influcence of pressure deadzone. Experiments of both methods are compared, displaying the potential effectiveness of applying these methods.

Continuum Robots for Manipulation Applications: A Survey

This paper presents a literature survey documenting the evolution of continuum robots over the past two decades (1999–present). Attention is paid to bioinspired soft robots with respect to the following three design parameters: structure, materials, and actuation. Using this three-faced prism, we identify the uniqueness and novelty of robots that have hitherto not been publicly disclosed. The motivation for this study comes from the fact that continuum soft robots can make inroads in industrial manufacturing, and their adoption will be accelerated if their key advantages over counterparts with rigid links are clear. Four different taxonomies of continuum robots are included in this study, enabling researchers to quickly identify robots of relevance to their studies. The kinematics and dynamics of these robots are not covered, nor is their application in surgical manipulation.