Automatic Initialization and Dynamic Tracking of Surgical Suture Threads

The current state of autonomous suturing: a systematic review

BACKGROUND

Robotic technology is an important tool in surgical innovation, with robots increasingly being used in the clinical setting. Robots can be used to enhance accuracy, perform remote actions, or to automate tasks. One such surgical task is suturing, a repetitive, fundamental component of surgery that can be tedious and time consuming. Suturing is a promising automation target because of its ubiquity, repetitive nature, and defined constraints. This systematic review examines research to date on autonomous suturing.

METHODS

A systematic review of the literature focused on autonomous suturing was conducted in accordance with PRISMA guidelines.

RESULTS

6850 articles were identified by searching PubMed, Embase, Compendex, and Inspec. Duplicates and non-English articles were removed. 4389 articles were screened and 4305 were excluded. Of the 84 remaining, 43 articles did not meet criteria, leaving 41 articles for final review. Among these, 34 (81%) were published after 2014. 31 (76%) were published in an engineering journal9 in a robotics journal, and 1 in a medical journal. The great majority of articles (33, 80%) did not have a specific clinical specialty focus, whereas 6 (15%) were focused on applications in MIS/laparoscopic surgery and 2 (5%) on applications in ophthalmology. Several suturing subtasks were identified, including knot tying, suture passing/needle insertion, needle passing, needle and suture grasping, needle tracking/kinesthesia, suture thread detection, suture needle shape production, instrument assignment, and suture accuracy. 14 articles were considered multi-component because they referred to several previously mentioned subtasks.

CONCLUSION

In this systematic review exploring research to date on autonomous suturing, 41 articles demonstrated significant progress in robotic suturing. This summary revealed significant heterogeneity of work, with authors focused on different aspects of suturing and a multitude of engineering problems. The review demonstrates increasing academic and commercial interest in surgical automation, with significant technological advances toward feasibility.

Real-Time Suture Thread Detection with an Image Classifier

Micro-anastomosis is considered to be a difficult task even for skilled surgeons. Our group has developed a surgical robotic system to assist surgeons. Going further, the detection of surgically relevant objects in the microscopic view is indispensable for the automation or semi-automation of the system. This paper proposes a novel surgical thread detector inspired by an automatic crack detection method. The proposed method achieved a Dice score of 76.30% and an intersection over union (IOU) of 66.08% at 34.50 fps.

"Hey Siri! Perform a type 3 hysterectomy. Please watch out for the ureter!" What is autonomous surgery and what are the latest developments?

As a result of major advances in deep learning algorithms and computer processing power, there have been important developments in the fields of medicine and robotics. Although fully autonomous surgery systems where human impact will be minimized are still a long way off, systems with partial autonomy have gradually entered clinical use. In this review, articles on autonomous surgery classified and summarized, with the aim of informing the reader about questions such as "What is autonomic surgery?" and in which areas studies are progressing.

Real-Time Visual Tracking of Dynamic Surgical Suture Threads

In order to realize many of the potential benefits associated with robotically assisted minimally invasive surgery, the robot must be more than a remote controlled device. Currently, using a surgical robot can be challenging, fatiguing, and time consuming. Teaching the robot to actively assist surgical tasks, such as suturing, has the potential to vastly improve both patient outlook and the surgeon's efficiency. One obstacle to completing surgical sutures autonomously is the difficulty in tracking surgical suture threads. This paper presents novel stereo image processing algorithms for the detection, initialization, and tracking of a surgical suture thread. A Non Uniform Rational B-Spline (NURBS) curve is used to model a thin, deformable, and dynamic length thread. The NURBS model is initialized and grown from a single selected point located on the thread. The NURBS curve is optimized by minimizing the image matching energy between the projected stereo NURBS image and the segmented thread image. The algorithms are evaluated using suture threads, a calibrated test pattern, and a simulated thread image. Additionally, the accuracy of the algorithms presented are validated as they track a suture thread undergoing translation, deformation, and apparent length changes. All of the tracking is in real-time. Note to Practioners: Abstract-The problem of tracking a surgical suture thread was addressed in this work. Since the suture thread is highly deformable, any tracking algorithm must be robust to intersections, occlusions, knot tying, and length changes. The detection algorithm introduced in this paper is capable of distinguishing different threads when they intersect. The tracking algorithm presented here demonstrate that it is possible, using polynomial curves, to track a suture thread as it deforms, becomes occluded, changes length, and even ties a knot in real time. The detection algorithm can enhance directional thin features while the polynomial curve modeling can track any string like structure. Further integration of the polynomial curve with a feed-forward thread model could improve the stability and robustness of the thread tracking.

Three-Dimensional Surgical Needle Localization and Tracking Using Stereo Endoscopic Image Streams

This paper presents algorithms for three-dimensional tracking of surgical needles using the stereo endoscopic camera images obtained from the da Vinci ® Surgical Robotic System. The proposed method employs Bayesian state estimation, computer vision techniques, and robot kinematics. A virtual needle rendering procedure is implemented to create simulated images of the surgical needle under the da Vinci ® robot endoscope, which makes it possible to measure the similarity between the rendered needle image and the real needle. A particle filter algorithm using the mentioned techniques is then used for tracking the surgical needle. The performance of the tracking is experimentally evaluated using an actual da Vinci ® surgical robotic system and quantitatively validated in a ROS/Gazebo simulation thereof.

Catadioptric Stereo Tracking for Three Dimensional Shape Measurement of MRI Guided Catheters

The recent introduction of Magnetic Resonance Imaging (MRI)-actuated steerable catheters lays the ground work for increasing the efficacy of cardiac catheter procedures. The MRI, while capable of imaging the catheter for tracking and control, does not fulfill all of the needs required to identify and develop a complete catheter model. Specifially, the frequency response of the catheter must be identified to ensure stable control of the catheter system. This requires a higher frequency imaging than the MRI can achieve. This work uses a catadioptric stereo camera system consisting of a mirror and a single camera in order to track a MRI actuated catheter inside a MRI machine. The catadioptric system works in parallel to the MRI and is capable of recording the catheter at 60 fps for post processing. The accuracy of the catadioptric system is verified in imaging conditions that would be found inside the MRI. The stereo camera is then used to track a catheter as it is actuated inside the MRI.