Analysis of Suture Manipulation Forces for Teleoperation with Force Feedback

Joint-level force sensing for indirect hybrid force/position control of continuum robots with friction

Continuum robots offer the dexterity and obstacle circumvention capabilities necessary to enable surgery in deep surgical sites. They also can enable joint-level ex situ force sensing (JEFS), which provides an estimate of end-effector wrenches given joint-level forces. Prior works on JEFS relied on a restrictive embodiment with minimal actuation line friction and captured model and frictional actuation transmission uncertainties using a configuration space formulation. In this work, we overcome these limitations. First, frictional losses are canceled using a feed-forward term based on support vector regression in joint space. Then, regression maps and their interpolation are used to account for actuation hysteresis. The residual joint-force error is then further minimized using a least-squares model parameter update. An indirect hybrid force/position controller using JEFS is presented with evaluation carried out on a realistic pre-clinically deployable insertable robotic effectors platform (IREP) for single-port access surgery. Automated mock force-controlled ablation, exploration, and knot tightening are evaluated. A user study involving the daVinci Research Kit surgeon console and the IREP as a surgical slave was carried out to compare the performance of users with and without force feedback based on JEFS for force-controlled ablation and knot tightening. Results in automated experiments and a user study of telemanipulated experiments suggest that intrinsic force-sensing can achieve levels of force uncertainty and force regulation errors of the order of 0.2 N. Using JEFS and automated task execution, repeatability, and force regulation accuracy is shown to be comparable to using a commercial force sensor for human-in-the-loop feedback.

Suture Breakage Warning System for Robotic Surgery

As robotic surgery has increased in popularity, the lack of haptic feedback has become a growing issue due to the application of excessive forces that may lead to clinical problems such as intraoperative and postoperative suture breakage. Previous suture breakage warning systems have largely depended on visual and/or auditory feedback modalities, which have been shown to increase cognitive load and reduce operator performance. This work catalogues a new sensing technology and haptic feedback system (HFS) that can reduce instances of suture failure without negatively impacting performance outcomes including knot quality. Suture breakage is common in knot-tying as the pulling motion introduces prominent shear forces. A shear sensor mountable on the da Vinci robotic surgical system's Cadiere grasper detects forces that correlate to the suture's internal tension. HFS then provides vibration feedback to the operator as forces near a particular material's failure load. To validate the system, subjects tightened a total of four knots, two with the Haptic Feedback System (HFS) and two without feedback. The number of suture breakages were recorded and knot fidelity was evaluated by measuring knot slippage. Results showed that instances of suture failure were significantly reduced when HFS was enabled (p = 0.0078). Notably, knots tied with HFS also showed improved quality compared to those tied without feedback (p = 0.010). The results highlight the value of HFS in improving robotic procedure outcomes by reducing instances of suture failures, producing better knots, and reducing the need for corrective measures.

Supervised autonomous robotic soft tissue surgery

The current paradigm of robot-assisted surgeries (RASs) depends entirely on an individual surgeon's manual capability. Autonomous robotic surgery-removing the surgeon's hands-promises enhanced efficacy, safety, and improved access to optimized surgical techniques. Surgeries involving soft tissue have not been performed autonomously because of technological limitations, including lack of vision systems that can distinguish and track the target tissues in dynamic surgical environments and lack of intelligent algorithms that can execute complex surgical tasks. We demonstrate in vivo supervised autonomous soft tissue surgery in an open surgical setting, enabled by a plenoptic three-dimensional and near-infrared fluorescent (NIRF) imaging system and an autonomous suturing algorithm. Inspired by the best human surgical practices, a computer program generates a plan to complete complex surgical tasks on deformable soft tissue, such as suturing and intestinal anastomosis. We compared metrics of anastomosis-including the consistency of suturing informed by the average suture spacing, the pressure at which the anastomosis leaked, the number of mistakes that required removing the needle from the tissue, completion time, and lumen reduction in intestinal anastomoses-between our supervised autonomous system, manual laparoscopic surgery, and clinically used RAS approaches. Despite dynamic scene changes and tissue movement during surgery, we demonstrate that the outcome of supervised autonomous procedures is superior to surgery performed by expert surgeons and RAS techniques in ex vivo porcine tissues and in living pigs. These results demonstrate the potential for autonomous robots to improve the efficacy, consistency, functional outcome, and accessibility of surgical techniques.

The Role of Direct and Visual Force Feedback in Suturing Using a 7-DOF Dual-Arm Teleoperated System

The lack of haptic feedback in robotics-assisted surgery can result in tissue damage or accidental tool-tissue hits. This paper focuses on exploring the effect of haptic feedback via direct force reflection and visual presentation of force magnitudes on performance during suturing in robotics-assisted minimally invasive surgery (RAMIS). For this purpose, a haptics-enabled dual-arm master-slave teleoperation system capable of measuring tool-tissue interaction forces in all seven Degrees-of-Freedom (DOFs) was used. Two suturing tasks, tissue puncturing and knot-tightening, were chosen to assess user skills when suturing on phantom tissue. Sixteen subjects participated in the trials and their performance was evaluated from various points of view: force consistency, number of accidental hits with tissue, amount of tissue damage, quality of the suture knot, and the time required to accomplish the task. According to the results, visual force feedback was not very useful during the tissue puncturing task as different users needed different amounts of force depending on the penetration of the needle into the tissue. Direct force feedback, however, was more useful for this task to apply less force and to minimize the amount of damage to the tissue. Statistical results also reveal that both visual and direct force feedback were required for effective knot tightening: direct force feedback could reduce the number of accidental hits with the tissue and also the amount of tissue damage, while visual force feedback could help to securely tighten the suture knots and maintain force consistency among different trials/users. These results provide evidence of the importance of 7-DOF force reflection when performing complex tasks in a RAMIS setting.

Pneumatically Driven Multi-DOF Surgical Forceps Manipulator with a Bending Joint Mechanism Using Elastic Bodies

[abstFig src='/00280004/13.jpg' width='300' text='Proposed pneumatically driven multi-DOF surgical forceps manipulator' ] In this study, a pneumatically-driven forceps manipulator was developed for a master-slave-type surgical robot. The proposed manipulator had two flexible joints, one for the bending joint at the tip and the other for transmitting a bending force from the actuators to the wires of the forceps. The manipulator had two degree-of-freedoms (DOFs) of bending driven by only two pneumatic cylinders and a gripper driven by a cylinder. Given the interoperability in real surgery, a mechanism was proposed such that the clean forceps part could be easily attached to and detached from the filthy drive unit. An experiment of the master-slave-system was conducted with the proposed manipulator to verify the tracking performance of the cylinders’ position and the bending angle of the forceps manipulator.

A Novel Device for Measuring Forces in Endoluminal Procedures

In this paper a simple but effective measuring system for endoluminal procedures is presented. The device allows measuring forces during the endoluminal manipulation of tissues with a standard surgical instrument for laparoscopic procedures. The force measurement is performed by recording both the forces applied directly by the surgeon at the instrument handle and the reaction forces on the access port. The measuring system was used to measure the forces necessary for appropriate surgical manipulation of tissues during transanal endoscopic microsurgery (TEM). Ex-vivo and in-vivo measurements were performed, reported and discussed. The obtained data can be used for developing and appropriately dimensioning novel dedicated instrumentation for TEM procedures.

Selective automation and skill transfer in medical robotics: a demonstration on surgical knot-tying

BACKGROUND

Transferring non-trivial human manipulation skills to robot systems is a challenging task. There have been a number of attempts to design research systems for skill transfer, but the level of the complexity of the actual skills transferable to the robot was rather limited, and delicate operations requiring a high dexterity and long action sequences with many sub-operations were impossible to transfer.

METHODS

A novel approach to human-machine skill transfer for multi-arm robot systems is presented. The methodology capitalizes on the metaphor of 'scaffolded learning', which has gained widespread acceptance in psychology. The main idea is to formalize the superior knowledge of a teacher in a certain way to generate support for a trainee. In our case, the scaffolding is constituted by abstract patterns, which facilitate the structuring and segmentation of information during 'learning by demonstration'. The actual skill generalization is then based on simulating fluid dynamics.

RESULTS

The approach has been successfully evaluated in the medical domain for the delicate task of automated knot-tying for suturing with standard surgical instruments and a realistic minimally invasive robotic surgery system.

Development of optical fiber Bragg grating force-reflection sensor system of medical application for safe minimally invasive robotic surgery

Force feedback plays a very important role in medical surgery. In minimally invasive surgery (MIS), however, the very long and stiff bars of surgical instruments greatly diminish force feedback for the surgeon. In the case of minimally invasive robotic surgery (MIRS), force feedback is totally eliminated. Previous researchers have reported that the absence of force feedback increased the average force magnitude applied to the tissue by at least 50%, and increased the peak force magnitude by at least a factor of two. Therefore, it is very important to provide force information in MIRS. Recently, many sensors are being developed for MIS and MIRS, but some obstacles to their application in actual medical surgery must be surmounted. The most critical problems are size limit and sterilizability. Optical fiber sensors are among the most suitable sensors for the surgical environment. The optical fiber Bragg grating (FBG) sensor, in particular, offers an important additional advantage over other optical fiber sensors in that it is not influenced by the intensity of the light source. In this paper, we present the initial results of a study on the application of a FBG sensor to measure reflected forces in MIRS environments and suggest the possibility of successful application to MIRS systems.

Mechanical analysis of end-to-end silk-sutured anastomosis for robot-assisted surgery

BACKGROUND

Robot-assisted anastomosis holds great promise for the future. To secure surgery quality, some key process factors, such as the force arrangement of sutures, should be provided because of the lack of haptic feedback in robotics systems

METHODS

A model of anastomosis is presented to establish the mechanical relationship between vessel and sutures. Stress distribution of the vessel loaded by the suture was then achieved through finite-element simulations, based on the material property test results. Further, experiments were performed to validate the reliability of the FEM simulation of the anastomosis process.

RESULTS

To avoid blood osmosis, the allowable lower limit of the suture tension was 0.05 N. To keep the tissue free from injury, the allowable upper limit of tension on the suture was 0.4 N.

CONCLUSIONS

The study provided meaningful results for directing the robot-assisted anastomosis procedure and design of the surgical tools.

Can surgeons think and operate with haptics at the same time?

Much effort has been devoted to incorporating haptic feedback into surgical simulators. However, the benefits of haptics for novice trainees in the early stages of learning are not clear. Presumably, novices have less spare attentional resources to attend to haptic cues while learning basic laparoscopic skills. The aim of this study was to determine whether novice surgeons have adequate cognitive resources to attend to haptic information. Thirty surgical residents and attendings performed a TransferPlace task in a simulator, with and without haptics. Cognitive loading was imposed using a mental arithmetic task. Subjects performed 10 trials (five with cognitive loading and five without) with and without haptics. Results showed that all subjects performed significantly slower (27%) when they were cognitively loaded than unloaded, but equally accurately in both cases, suggesting a speed-accuracy tradeoff. On average, subjects performed 36% faster and 97% more accurately with haptics than without, even while cognitively loaded. Haptic feedback can not only enhance performance, but also counter the effect of cognitive load. This effect is greater for more experienced surgeons than less experienced ones, indicating greater spare cognitive capacity in surgeons with more experience.

Robotics in urologic surgery: an evolving new technology

Rapid technological developments in the past two decades have produced new inventions such as robots and incorporated them into our daily lives. Today, robots perform vital functions in homes, outer space, hospitals and on military instillations. The development of robotic surgery has given hospitals and health care providers a valuable tool that is making a profound impact on highly technical surgical procedures. The field of urology is one area of medicine that has adopted and incorporated robotic surgery into its armamentarium. Innovative robotic urologic surgical applications and techniques are being developed and reported everyday. Increased utilization and development will ultimately fuel the discovery of newer applications of robotic systems in urologic surgery. Herein we provide an overview of the history, development, and applications of robotics in surgery with a focus on urologic surgery.

Effect of sensory substitution on suture-manipulation forces for robotic surgical systems

OBJECTIVES

Direct haptic (force or tactile) feedback is not yet available in commercial robotic surgical systems. Previous work by our group and others suggests that haptic feedback might significantly enhance the execution of surgical tasks requiring fine suture manipulation, specifically those encountered in cardiothoracic surgery. We studied the effects of substituting direct haptic feedback with visual and auditory cues to provide the operating surgeon with a representation of the forces he or she is applying with robotic telemanipulators.

METHODS

Using the robotic da Vinci surgical system (Intuitive Surgical, Inc, Sunnyvale, Calif), we compared applied forces during a standardized surgical knot-tying task under 4 different sensory-substitution scenarios: no feedback, auditory feedback, visual feedback, and combined auditory-visual feedback.

RESULTS

The forces applied with these sensory-substitution modes more closely approximate suture tensions achieved under ideal haptic conditions (ie, hand ties) than forces applied without such sensory feedback. The consistency of applied forces during robot-assisted suture tying aided by visual feedback or combined auditory-visual feedback sensory substitution is superior to that achieved with hand ties. Robot-assisted ties aided with auditory feedback revealed levels of consistency that were generally equivalent or superior to those attained with hand ties. Visual feedback and auditory feedback improve the consistency of robotically applied forces.

CONCLUSIONS

Sensory substitution, in the form of visual feedback, auditory feedback, or both, confers quantifiable advantages in applied force accuracy and consistency during the performance of a simple surgical task.

Overview and fundamentals of urologic robot-integrated systems

Advances in technology have revolutionized urology. Minimally invasive tools now form the core of the urologist's armamentarium. Laparoscopic surgery has become the favored approach for treating many complicated urologic ailments. Surgical robots represent the next evolutionary step in the fruitful man-machine partnership. The introduction of robotic technology in urology changes how urologists learn, teach, plan, and operate. As technology evolves, robots not only will improve performance in minimally invasive procedures, but also enhance other procedures or enable new kinds of operations.

Methods for haptic feedback in teleoperated robot-assisted surgery

Teleoperated minimally invasive surgical robots can significantly enhance a surgeon's accuracy, dexterity and visualization. However, current commercially available systems do not include significant haptic (force and tactile) feedback to the operator. This paper describes experiments to characterize this problem, as well as several methods to provide haptic feedback in order to improve surgeon's performance. There exist a variety of sensing and control methods that enable haptic feedback, although a number of practical considerations, e.g. cost, complexity and biocompatibility, present significant challenges. The ability of teleoperated robot-assisted surgical systems to measure and display haptic information leads to a number of additional exciting clinical and scientific opportunities, such as active operator assistance through "virtual fixtures" and the automatic acquisition of tissue properties.