Autonomous catheter insertion system using magnetic motion capture sensor for endovascular surgery

Actuation technologies for magnetically guided catheters

Due to their wide range of clinical application possibilities, magnetic actuation technologies have grabbed the attention of researchers worldwide. The design, execution, and analysis of magnetic catheter systems have advanced significantly during the last decade. The review focuses on magnetic actuation for catheter steering and control of the device, which will be explored in detail in the following sections. There is a discussion of future work and the challenges of the review systems, and the conclusions are finally addressed.

Learning-based autonomous vascular guidewire navigation without human demonstration in the venous system of a porcine liver

Purpose

The navigation of endovascular guidewires is a dexterous task where physicians and patients can benefit from automation. Machine learning-based controllers are promising to help master this task. However, human-generated training data are scarce and resource-intensive to generate. We investigate if a neural network-based controller trained without human-generated data can learn human-like behaviors.

Methods

We trained and evaluated a neural network-based controller via deep reinforcement learning in a finite element simulation to navigate the venous system of a porcine liver without human-generated data. The behavior is compared to manual expert navigation, and real-world transferability is evaluated.

Results

The controller achieves a success rate of 100% in simulation. The controller applies a wiggling behavior, where the guidewire tip is continuously rotated alternately clockwise and counterclockwise like the human expert applies. In the ex vivo porcine liver, the success rate drops to 30%, because either the wrong branch is probed, or the guidewire becomes entangled.

Conclusion

In this work, we prove that a learning-based controller is capable of learning human-like guidewire navigation behavior without human-generated data, therefore, mitigating the requirement to produce resource-intensive human-generated training data. Limitations are the restriction to one vessel geometry, the neglected safeness of navigation, and the reduced transferability to the real world.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11548-022-02646-8.

Remote vascular interventional surgery robotics: a literature review

Vascular interventional doctors are exposed to radiation hazards during surgery and endure high work intensity. Remote vascular interventional surgery robotics is a hot research field, in which researchers aim to not only protect the health of interventional doctors, but to also improve surgical accuracy and efficiency. However, the current vascular interventional robots have numerous shortcomings, such as poor haptic feedback, few compatible surgeries and instruments, and cumbersome maintenance and operational procedures. Nevertheless, vascular interventional surgery combined with robotics provides more cutting-edge directions, such as Internet remote surgery combined with 5G network technology and the application of artificial intelligence in surgical procedures. To summarize the developmental status and key technical points of intravascular interventional surgical robotics research, we performed a systematic literature search to retrieve original articles related to remote vascular interventional surgery robotics published up to December 2020. This review, which includes 113 articles published in English, introduces the mechanical and structural characteristics of various aspects of vascular interventional surgical robotics, discusses the current key features of vascular interventional surgical robotics in force sensing, haptic feedback, and control methods, and summarizes current frontiers in autonomous surgery, long-distance robotic telesurgery, and magnetic resonance imaging (MRI)-compatible structures. On the basis of summarizing the current research status of remote vascular interventional surgery robotics, we aim to propose a variety of prospects for future robotic systems.

Development of a Robotic Catheter Manipulation System Based on BP Neural Network PID Controller

In the process of artificial interventional therapy, the operation of artificial catheter is not accurate, which will bring strong radiation damage to surgeons. The purpose of this study is to develop a catheter operating system of surgical robot to assist doctors in remote operation and avoid the influence of radiation. BP neural network plays an important role in the flexibility and rapidity of control. According to the actual output of the system, the control parameters of the controller are constantly adjusted to achieve better output effect. This paper introduces the practical application of BP neural network PID controller in the remote operation of the system and compares with the traditional PID controller. The results show that the new control algorithm is feasible and effective. The results show that the synchronization performance of BP neural network PID controller is better than that of traditional PID controller.

Autonomous guidewire navigation in a two dimensional vascular phantom

The treatment of cerebro- and cardiovascular diseases requires complex and challenging navigation of a catheter. Previous attempts to automate catheter navigation lack the ability to be generalizable. Methods of Deep Reinforcement Learning show promising results and may be the key to automate catheter navigation through the tortuous vascular tree. This work investigates Deep Reinforcement Learning for guidewire manipulation in a complex and rigid vascular model in 2D. The neural network trained by Deep Deterministic Policy Gradients with Hindsight Experience Replay performs well on the low-level control task, however the high-level control of the path planning must be improved further.

A novel noncontact detection method of surgeon's operation for a master-slave endovascular surgery robot

Master-slave endovascular interventional surgery (EIS) robots have brought revolutionary advantages to traditional EIS, such as avoiding X-ray radiation to the surgeon and improving surgical precision and safety. However, the master controllers of most of the current EIS robots always lead to bad human-machine interaction, because of the difference in nature between the rigid operating handle and the flexible medical catheter used in EIS. In this paper, a noncontact detection method is proposed, and a novel master controller is developed to realize real-time detection of surgeon's operation without interference to the surgeon. A medical catheter is used as the operating handle. It is enabled by using FAST corner detection algorithm and optical flow algorithm to track the corner points of the continuous markers on a designed sensing pipe. A mathematical model is established to calculate the axial and rotational motion of the sensing pipe according to the moving distance of the corner points in image coordinates. A master-slave EIS robot system is constructed by integrating the proposed master controller and a developed slave robot. Surgical task performance evaluation in an endovascular evaluator (EVE) is conducted, and the results indicate that the proposed detection method breaks through the axial measuring range limitation of the previous marker-based detection method. In addition, the rotational detection error is reduced by 92.5% compared with the previous laser-based detection method. The results also demonstrate the capability and efficiency of the proposed master controller to control the slave robot for surgical task implementation. Graphical abstract A novel master controller is developed to realize real-time noncontact detection of surgeon's operation without interference to the surgeon. The master controller is used to remotely control the slave robot to implement certain surgical tasks.

Haptic Telerobotic Cardiovascular Intervention: A Review of Approaches, Methods, and Future Perspectives

Cardiac diseases are recognized as the leading cause of mortality, hospitalization, and medical prescription globally. The gold standard for the treatment of coronary artery stenosis is the percutaneous cardiac intervention that is performed under live X-ray imaging. Substantial clinical evidence shows that the surgeon and staff are prone to serious health problems due to X-ray exposure and occupational hazards. Telerobotic vascular intervention systems with a master-slave architecture reduced the X-ray exposure and enhanced the clinical outcomes; however, the loss of haptic feedback during surgery has been the main limitation of such systems. This paper is a review of the state of the art for haptic telerobotic cardiovascular interventions. A survey on the literature published between 2000 and 2019 was performed. Results of the survey were screened based on their relevance to this paper. Also, the leading research disciplines were identified based on the results of the survey. Furthermore, different approaches for sensor-based and model-based haptic telerobotic cardiovascular intervention, haptic rendering and actuation, and the pertinent methods were critically reviewed and compared. In the end, the current limitations of the state of the art, unexplored research areas as well as the future perspective of the research on this technology were laid out.

A marker-based contactless catheter-sensing method to detect surgeons' operations for catheterization training systems

It is challenging to position a catheter or a guidewire within a patient's complicated and delicate vascular structure due to the lack of intuitive visual feedback by only manipulating the proximal part of the surgical instruments. Training is therefore critical before an actual surgery because any mistake due to the surgeon's inexperience can be fatal for the patient. The catheter manipulation skills of experienced surgeons can be useful as input for training novice surgeons. However, few research groups focused on designs with consideration of the contactless catheter motion measurement, which allows obtaining expert surgeons' catheter manipulation trajectories whilst still allowing them to employ an actual catheter and apply conventional pull, push and twist of the catheter as used in bedside intravascular interventional surgeries. In this paper, a novel contactless catheter-sensing method is proposed to measure the catheter motions by detecting and tracking a passive marker with four feature-point groups. The passive marker is designed to allow simultaneously sensing the translational and rotational motions of the input catheter. Finally, the effectiveness of the proposed contactless catheter-sensing method is validated by conducting a series of comparison experiments. The accuracy and error analysis are quantified based on the absolute error, relative error, mean absolute error, and the success rate of the detection.

Compensatory force measurement and multimodal force feedback for remote-controlled vascular interventional robot

Minimally invasive vascular interventional surgery is widely used and remote-controlled vascular interventional surgery robots (RVIRs) are being developed to reduce the occupational risk of the intervening physician in minimally invasive vascular interventional surgeries. Skilled surgeon performs surgeries mainly depending on the detection of collisions. Inaccurate force feedback will be difficult for surgeons to perform surgeries or even results in medical accidents. In addition, the surgeon cannot quickly and easily distinguish whether the proximal force exceeds the safety threshold of blood vessels or not, and thus it results in damage to the blood vessels. In this paper, we present a novel method comprising compensatory force measurement and multimodal force feedback (MFF). Calibration experiments and performance evaluation experiments were carried out. Experimental results demonstrated that the proposed method can measure the proximal force of catheter/guidewire accurately and assist surgeons to distinguish the change of proximal force more easily. This novel method is suitable for use in actual surgical operations.

Operation evaluation in-human of a novel remote-controlled vascular interventional robot

Remote-controlled vascular interventional robots (RVIRs) are being developed to increase the accuracy of surgical operations and reduce the number of occupational risks sustained by intervening physicians, such as radiation exposure and chronic neck/back pain. However, complex control of the RVIRs improves the doctor's operation difficulty and reduces the operation efficiency. Furthermore, incomplete sterilization of the RVIRs will increase the risk of infection, or even cause medical accidents. In this study, we introduced a novel method that provides higher operation efficiency than a previous prototype and allows for complete robot sterilization. A prototype was fabricated and validated through laboratory setting experiments and an in-human experiment. The results illustrated that the proposed RVIR has better performance compared with the previous prototype, and preliminarily demonstrated that the proposed RVIR has good safety and reliability and can be used in clinical surgeries.

A cooperation of catheters and guidewires-based novel remote-controlled vascular interventional robot

Remote-controlled vascular interventional robots (RVIRs) are being developed to increase the overall accuracy of surgical operations and reduce the occupational risks of intervening physicians, such as radiation exposure and chronic neck/back pain. Several RVIRs have been used to operate catheters or guidewires accurately. However, a lack of cooperation between the catheters and guidewires results in the surgeon being unable to complete complex surgery by propelling the catheter/guidewire to the target position. Furthermore, it is a significant challenge to operate the catheter/guidewire accurately and detect their proximal force without damaging their surfaces. In this study, we introduce a novel method that allows catheters and guidewires to be operated simultaneously in complex surgery. Our method accurately captures force measurements and enables precisely controlled catheter and guidewire operation. A prototype is validated through various experiments. The results demonstrate the feasibility of the proposed RVIR to operate a catheter and guidewire accurately, detect the resistance forces, and complete complex surgical operations in a cooperative manner.

3D Catheter Shape Reconstruction Using Electromagnetic and Image Sensors

In current practice, fluoroscopy remains the gold standard for guiding surgeons during endovascular catheterization. The poor visibility of anatomical structures and the absence of depth information make accurate catheter localization and manipulation a difficult task. Overexposure to radiation and use of risk-prone contrast agent also compromise surgeons’ and patients’ health. Alternative approaches using embedded electromagnetic (EM) sensors have been developed to overcome the limitations of fluoroscopy-based interventions. As only a finite number of sensors can be integrated within a catheter, methods that rely on such sensors require the use of interpolation schemes to recover the catheter shape. Since EM sensors are sensitive to external interferences, the outcome is not robust. This paper introduces a probabilistic framework that improves the catheter localization and reduces the dependency on fluoroscopy and contrast agents. Within this framework, the dense 2D information extracted from fluoroscopic images is combined with the discrete pose information of EM sensors to provide a reliable reconstruction of the full three-dimensional catheter shape. Validation in a physics-based simulation environment and in a real-world experimental setup provides promising results and indicates that the proposed framework allows reconstructing the 3D catheter shape with a median root-mean-square error of 3.7[Formula: see text]mm with an interquartile range of 0.3[Formula: see text]mm.

Real-time in vitro intravascular reconstruction and navigation for endovascular aortic stent grafting

BACKGROUND

Trans-catheter endovascular stent grafting minimizes trauma and increases the benefitting patient population. However, the alignment between stent graft branches and vasculature branches remains time-consuming and challenging, and such techniques require a significant amount of contrast agent for imaging.

METHODS

A new framework for intravascular reconstruction based on sensor fusion between intravascular ultrasound (IVUS) imaging and electromagnetic (EM) tracking was proposed. A new image processing method was presented to realize fully automatic processing of IVUS imaging and 3D reconstruction in real time, as well as branch detection for alignment and deployment. Complementary navigation using CT data allows for efficient catheter advancement and assistant clinical judgement.

RESULTS

The reconstruction of an in vitro descending aorta phantom with branches was realized at 35 Hz, with cross-section radius average error of 0.64 mm.

CONCLUSION

The proposed method demonstrates significant potential for clinical applications, enables navigation for precise alignment and placement for stent grafting to reduce surgical time, and decreases hemorrhagic collisions and the use of contrast agent. Copyright © 2016 John Wiley & Sons, Ltd.

Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions

PURPOSE

Advances in technology and computing play an increasingly important role in the evolution of modern surgical techniques and paradigms. This article reviews the current role of machine learning (ML) techniques in the context of surgery with a focus on surgical robotics (SR). Also, we provide a perspective on the future possibilities for enhancing the effectiveness of procedures by integrating ML in the operating room.

METHODS

The review is focused on ML techniques directly applied to surgery, surgical robotics, surgical training and assessment. The widespread use of ML methods in diagnosis and medical image computing is beyond the scope of the review. Searches were performed on PubMed and IEEE Explore using combinations of keywords: ML, surgery, robotics, surgical and medical robotics, skill learning, skill analysis and learning to perceive.

RESULTS

Studies making use of ML methods in the context of surgery are increasingly being reported. In particular, there is an increasing interest in using ML for developing tools to understand and model surgical skill and competence or to extract surgical workflow. Many researchers begin to integrate this understanding into the control of recent surgical robots and devices.

CONCLUSION

ML is an expanding field. It is popular as it allows efficient processing of vast amounts of data for interpreting and real-time decision making. Already widely used in imaging and diagnosis, it is believed that ML will also play an important role in surgery and interventional treatments. In particular, ML could become a game changer into the conception of cognitive surgical robots. Such robots endowed with cognitive skills would assist the surgical team also on a cognitive level, such as possibly lowering the mental load of the team. For example, ML could help extracting surgical skill, learned through demonstration by human experts, and could transfer this to robotic skills. Such intelligent surgical assistance would significantly surpass the state of the art in surgical robotics. Current devices possess no intelligence whatsoever and are merely advanced and expensive instruments.

Current and emerging robot-assisted endovascular catheterization technologies: a review

Endovascular techniques have been embraced as a minimally-invasive treatment approach within different disciplines of interventional radiology and cardiology. The current practice of endovascular procedures, however, is limited by a number of factors including exposure to high doses of X-ray radiation, limited 3D imaging, and lack of contact force sensing from the endovascular tools and the vascular anatomy. More recently, advances in steerable catheters and development of master/slave robots have aimed to improve these practices by removing the operator from the radiation source and increasing the precision and stability of catheter motion with added degrees-of-freedom. Despite their increased application and a growing research interest in this area, many such systems have been designed without considering the natural manipulation skills and ergonomic preferences of the operators. Existing studies on tool interactions and natural manipulation skills of the operators are limited. In this manuscript, new technical developments in different aspects of robotic endovascular intervention including catheter instrumentation, intra-operative imaging and navigation techniques, as well as master/slave based robotic catheterization platforms are reviewed. We further address emerging trends and new research opportunities towards more widespread clinical acceptance of robotically assisted endovascular technologies.

A remote-controlled vascular interventional robot: system structure and image guidance

BACKGROUND

Robot-assisted vascular interventional surgery (VIS) enables the surgeon to teleoperate a catheter in a safe cabinet, such that exposure to X-ray radiation is reduced. For safe and accurate teleoperation, system structure and image guidance is important.

METHODS

The system structure of the developed remote-controlled vascular interventional robot (RVIR) and its image guidance system (IGS) are introduced. RVIR is based on a master-slave structure. Key technologies of IGS are addressed, including C-arm calibration, distortion correction, catheter localization and 3D vasculature reconstruction.

RESULTS

Experiments show that the RMS error of distortion correction is 0.35 pixels, and 0.53 mm for distance reconstruction. The error in catheter localization between the IGS and the encoders is small. In vitro and in vivo tests verified the feasibility of RVIR.

CONCLUSIONS

Experiments indicate that the RVIR is feasible and valid to help the surgeon perform VIS remotely; the function and reconstruction accuracy of IGS can satisfy the surgeon's requirement to guide the RVIR.

Electromagnetic navigation platform for endovascular surgery: how to develop sensorized catheters and guidewires

BACKGROUND

Endovascular procedures are nowadays limited by difficulties arising from the use of 2D images and are associated with dangerous X-ray exposure and the injection of nephrotoxic contrast medium.

METHODS

An electromagnetic navigator is proposed to guide endovascular procedures with reduced radiation dose and contrast medium injection. Five DOF electromagnetic sensors are calibrated and used to track in real time the positions and orientation of endovascular catheters and guidewires, while intraoperative 3D rotational angiography is used to acquire 3D models of patient anatomy. A preliminary prototype is developed to prove the feasibility of the system using an anthropomorphic phantom.

RESULTS

The spatial accuracy of the system was evaluated during 70 targeting trials obtaining an overall accuracy of 1.2 ± 0.3 mm; system usability was positively evaluated by three surgeons.

CONCLUSIONS

The strategy proposed to sensorize endovascular instruments paves the way for the development of surgical strategies with reduced radiation dose and contrast medium injection. Further in vitro, animal and clinical experiments are necessary for complete surgical validation.

The requirement for smart catheters for advanced endovascular applications

Endovascular techniques are well established in minimally invasive vascular surgery. Stent-graft technology is evolving. This will allow more complex aortic pathology to be treated. Current limitations in this equipment include the reliance on selective catheters along with trial and error to cannulate visceral or aortic arch branch vessels. This can lead to prolonged operations with increased radiation and contrast load to the patient with inevitable increase in peri-operative morbidity. A smart catheter that could expedite target vessel catheterization, minimize repeat catheter changes, and provide greater haptic feedback would enhance the endovascular surgeon's armamentarium. This would lead to broader and safer application of endovascular surgery.

Robot Manipulation and Guidance Using Magnetic Motion Capture Sensor and a Rule-Based Controller

Magnetic motion capture sensors (MMCS) are not commonly used for robot control due to the need for complex, resource-consuming calibration to correct error introduced by the magnetic sensor. We propose avoiding such calibration using a rule-based controller that only uses spatial coordinates from the magnetic sensor. This controller uses a sparse look-up table of spatial coordinates and actions conducted by the robot and reacts to the presence of the sensor near reference points. The control method was applied to manipulate a robotic camera to track a catheter-shaped sensor inside vessels silicone models. A second evaluation was done guiding a mechanism to reconstruct catheter insertion in major silicone vasculature models. The robotic camera tracked the catheter by reacting to the sensor within 10 mm of each reference point. The catheter insertion mechanism reconstructed the catheter trajectory by reacting to the sensor within 6 mm of each reference point. We found that the proposed method allowed robot control in a bounded space without having to correct for the magnetic tracker output distortion.