Adaptive Kinematic Control of a Robotic Venipuncture Device Based on Stereo Vision, Ultrasound, and Force Guidance

Advances of surgical robotics: image-guided classification and application

Surgical robotics application in the field of minimally invasive surgery has developed rapidly and has been attracting increasingly more research attention in recent years. A common consensus has been reached that surgical procedures are to become less traumatic and with the implementation of more intelligence and higher autonomy, which is a serious challenge faced by the environmental sensing capabilities of robotic systems. One of the main sources of environmental information for robots are images, which are the basis of robot vision. In this review article, we divide clinical image into direct and indirect based on the object of information acquisition, and into continuous, intermittent continuous, and discontinuous according to the target-tracking frequency. The characteristics and applications of the existing surgical robots in each category are introduced based on these two dimensions. Our purpose in conducting this review was to analyze, summarize, and discuss the current evidence on the general rules on the application of image technologies for medical purposes. Our analysis gives insight and provides guidance conducive to the development of more advanced surgical robotics systems in the future.

Spatial sensitivity distribution assessment and Monte Carlo simulations for needle-based bioimpedance imaging during venipuncture using the finite element method

Despite being among the most common medical procedures, needle insertions suffer from a high error rate. Impedance measurements using electrode-equipped needles offer promise for improved tissue targeting and reduced errors. Impedance visualization usually requires an extensive pre-measured impedance dataset for tissue differentiation and knowledge of the electric fields contributing to the resulting impedances. This work presents two finite element simulation approaches for both problems. The first approach describes the generation of a multitude of impedances with Monte Carlo simulations for both, homogeneous and inhomogeneous tissue to circumvent the need to rely on previously measured data. These datasets could be used for tissue discrimination. The second method describes the simulation of the spatial sensitivity distribution of an electrode layout. Two singularity analysis methods were employed to determine the bulk of the sensitivity within a finite volume, which in turn enables consistent 3D visualization. The modeled electrode layout consists of 12 electrodes radially placed around a hypodermic needle. Electrical excitation was simulated using two neighboring electrodes for current carriage and voltage pickup, which resulted in 12 distinct bipolar excitation states. Both, the impedance simulations and the respective singularity analysis methods were compared with each other. The results show that the statistical spread of impedances is highly dependent on the tissue type and its inhomogeneities. The bounded bulk of sensitivities of both methods are of similar extent and symmetry. Future models should incorporate more detailed tissue properties such as anisotropy or changing material properties due to tissue deformation to gain more accurate predictions.

Design of composite puncture blood collection system and research on puncture force

Venous blood collection testing is one of the most commonly used medical diagnostic methods. Compared with conventional venous blood collection, robotic collection can reduce needle-stick injuries, medical staff workload, and infection risk; allow doctor-patient isolation; and improve collection reliability. Existing venous blood collection robots use rigid puncture needles, which can easily puncture the lower wall of blood vessels, causing vessel damage and collection failure. This paper proposes a bionic blood collection strategy based on a composite puncture needle that mimics the structure and function of mosquito mouthparts. A bionic composite puncture needle insertion system with puncture-force sensing was designed, and venipuncture forces were simulated and mathematically modelled. A prototype insertion system was built and used in an experiment, which demonstrated effective composite puncture blood collection and explored the factors influencing puncture force. Puncture force decreases with increased puncture speed and angle and with a decreased needle diameter. This provides a basis for optimising the parameters of blood collection robots.

Autonomous Robotic Ultrasound Vascular Imaging System With Decoupled Control Strategy for External-Vision-Free Environments

OBJECTIVE

Ultrasound (US) probes scan over the surface of the human body to acquire US images in clinical vascular US diagnosis. However, due to the deformation and specificity of different human surfaces, the relationship between the scan trajectory of the skin and the internal tissues is not fully correlated, which poses a challenge for autonomous robotic US imaging in a dynamic and external-vision-free environment. Here, we propose a decoupled control strategy for autonomous robotic vascular US imaging in an environment without external vision.

METHODS

The proposed system is divided into outer-loop posture control and inner-loop orientation control, which are separately determined by a deep learning (DL) agent and a reinforcement learning (RL) agent. First, we use a weakly supervised US vessel segmentation network to estimate the probe orientation. In the outer loop control, we use a force-guided reinforcement learning agent to maintain a specific angle between the US probe and the skin in the dynamic imaging processes. Finally, the orientation and the posture are integrated to complete the imaging process.

RESULTS

Evaluation experiments on several volunteers showed that our RUS could autonomously perform vascular imaging in arms with different stiffness, curvature, and size without additional system adjustments. Furthermore, our system achieved reproducible imaging and reconstruction of dynamic targets without relying on vision-based surface information.

CONCLUSION AND SIGNIFICANCE

Our system and control strategy provides a novel framework for the application of US robots in complex and external-vision-free environments.

Application of Medical Image Navigation Technology in Minimally Invasive Puncture Robot

Microneedle puncture is a standard minimally invasive treatment and surgical method, which is widely used in extracting blood, tissues, and their secretions for pathological examination, needle-puncture-directed drug therapy, local anaesthesia, microwave ablation needle therapy, radiotherapy, and other procedures. The use of robots for microneedle puncture has become a worldwide research hotspot, and medical imaging navigation technology plays an essential role in preoperative robotic puncture path planning, intraoperative assisted puncture, and surgical efficacy detection. This paper introduces medical imaging technology and minimally invasive puncture robots, reviews the current status of research on the application of medical imaging navigation technology in minimally invasive puncture robots, and points out its future development trends and challenges.

A venipuncture robot with decoupled position and attitude guided by near-infrared vision and force feedback

BACKGROUND

This study aims to develop a venipuncture robot to replace manual venipuncture to ease the heavy workload, lower the risk of 2019-nCoV infection, and boost venipuncture success rates.

METHOD

The robot is designed with decoupled position and attitude. It consists of a 3-degree-of-freedom positioning manipulator to locate the needle and a 3-degree-of-freedom end-effector that is always vertical to adjust the yaw and pitch angles of the needle. The near-infrared vision and laser sensors obtain the three-dimensional information of puncture positions, while the change in force detects the state feedback of punctures.

RESULTS

The experimental results demonstrate that the venipuncture robot has a compact design, flexible motion, high positioning accuracy and repeatability (0.11 and 0.04 mm), and a high success rate when puncturing the phantom.

CONCLUSION

This paper presents a decoupled position and attitude venipuncture robot guided by near-infrared vision and force feedback to replace manual venipuncture. The robot is compact, dexterous, and accurate, which helps to improve the success rate of venipuncture, and it is expected to achieve fully automatic venipuncture in the future.

Puncture site decision method for venipuncture robot based on near-infrared vision and multiobjective optimization

Venipuncture robots have superior perception and stability to humans and are expected to replace manual venipuncture. However, their use is greatly restricted because they cannot make decisions regarding the puncture sites. Thus, this study presents a multi-information fusion method for determining puncture sites for venipuncture robots to improve their autonomy in the case of limited resources. Here, numerous images have been gathered and processed to establish an image dataset of human forearms for training the U-Net with the soft attention mechanism (SAU-Net) for vein segmentation. Then, the veins are segmented from the images, feature information is extracted based on near-infrared vision, and a multiobjective optimization model for puncture site decision is provided by considering the depth, diameter, curvature, and length of the vein to determine the optimal puncture site. Experiments demonstrate that the method achieves a segmentation accuracy of 91.2% and a vein extraction rate of 86.7% while achieving the Pareto solution set (average time: 1.458 s) and optimal results for each vessel. Finally, a near-infrared camera is applied to the venipuncture robot to segment veins and determine puncture sites in real time, with the results transmitted back to the robot for an attitude adjustment. Consequently, this method can enhance the autonomy of venipuncture robots if implemented dramatically.

Design and Evaluation of a Handheld Robotic Device for Peripheral Catheterization

Medical robots provide enhanced dexterity, vision, and safety for a broad range of procedures. In this article, we present a handheld, robotic device capable of performing peripheral catheter insertions with high accuracy and repeatability. The device utilizes a combination of ultrasound imaging, miniaturized robotics, and machine learning to safely and efficiently introduce a catheter sheath into a peripheral blood vessel. Here, we present the mechanical design and experimental validation of the device, known as VeniBot. Additionally, we present results on our ultrasound deep learning algorithm for vessel segmentation, and performance on tissue-mimicking phantom models that simulate difficult peripheral catheter placement. Overall, the device achieved first-attempt success rates of 97 ± 4% for vessel punctures and 89 ± 7% for sheath cannulations on the tissue mimicking models (n = 240). The results from these studies demonstrate the viability of a handheld device for performing semi-automated peripheral catheterization. In the future, the use of this device has the potential to improve clinical workflow and reduce patient discomfort by assuring a safe and efficient procedure.

Autonomic Robotic Ultrasound Imaging System Based on Reinforcement Learning

OBJECTIVE

In this paper, we introduce an autonomous robotic ultrasound (US) imaging system based on reinforcement learning (RL). The proposed system and framework are committed to controlling the US probe to perform fully autonomous imaging of a soft, moving and marker-less target based only on single RGB images of the scene.

METHODS

We propose several different approaches and methods to achieve the following objectives: real-time US probe controlling, soft surface constant force tracking and automatic imaging. First, to express the state of the robotic US imaging task, we proposed a state representation model to reduce the dimensionality of the imaging state and encode the force and US information into the scene image space. Then, an RL agent is trained by a policy gradient theorem based RL model with the single RGB image as the only observation. To achieve adaptable constant force tracking between the US probe and the soft moving target, we propose a force-to-displacement control method based on an admittance controller.

RESULTS

In the simulation experiment, we verified the feasibility of the integrated method. Furthermore, we evaluated the proposed force-to-displacement method to demonstrate the safety and effectiveness of adaptable constant force tracking. Finally, we conducted phantom and volunteer experiments to verify the feasibility of the method on a real system.

CONCLUSION

The experiments indicated that our approaches were stable and feasible in the autonomic and accurate control of the US probe.

SIGNIFICANCE

The proposed system has potential application value in the image-guided surgery and robotic surgery.

Coronary vessel detection methods for organ-mounted robots

BACKGROUND

HeartLander is a tethered robot walker that utilizes suction to adhere to the beating heart. HeartLander can be used for minimally invasive administration of cardiac medications or ablation of tissue. In order to administer injections safely, HeartLander must avoid coronary vasculature.

METHODS

Doppler ultrasound signals were recorded using a custom-made cardiac phantom and used to classify different coronary vessel properties. The classification was performed by two machine learning algorithms, the support vector machines and a deep convolutional neural network. These algorithms were then validated in animal trials.

RESULTS

Accuracy of identifying vessels above turbulent flow reached greater than 92% in phantom trials and greater than 98% in animal trials.

CONCLUSIONS

Through the use of two machine learning algorithms, HeartLander has shown the ability to identify different sized vasculature proximally above turbulent flow. These results indicate that it is feasible to use Doppler ultrasound to identify and avoid coronary vasculature during cardiac interventions using HeartLander.

Symbol recognition system based on 3D stereo vision

With the continuous development of computer science and technology, symbol recognition systems may be converted from two-dimensional space to three-dimensional space. Therefore, this article mainly introduces the symbol recognition system based on 3D stereo vision. The three-dimensional image is taken by the visual coordinate measuring machine in two places on the left and right. Perform binocular stereo matching on the edge of the feature points of the two images. A corner detection algorithm combining SUSAN and Harris is used to detect the left and right camera calibration templates. The two-dimensional coordinate points of the object are determined by the image stereo matching module, and the three-dimensional discrete coordinate points of the object space can be obtained according to the transformation relationship between the image coordinates and the actual object coordinates. Then draw the three-dimensional model of the object through the three-dimensional drawing software. Experimental data shows that the logic resources and memory resources occupied by image preprocessing account for 30.4% and 27.4% of the entire system, respectively. The results show that the system can calibrate the internal and external parameters of the camera. In this way, the camera calibration result will be more accurate and the range will be wider. At the same time, it can effectively make up for the shortcomings of traditional modeling techniques to ensure the measurement accuracy of the detection system.

Telehealth's New Horizon: Providing Smart Hospital-Level Care in the Home

During the COVID-19 pandemic, medical providers have expanded telehealth into daily practice, with many medical and behavioral health care visits provided remotely over video or through phone. The telehealth market was already facilitating home health care with increasing levels of sophistication before COVID-19. Among the emerging telehealth practices, telephysical therapy; teleneurology; telemental health; chronic care management of congestive heart failure, chronic obstructive pulmonary disease, diabetes; home hospice; home mechanical ventilation; and home dialysis are some of the most prominent. Home telehealth helps streamline hospital/clinic operations and ensure the safety of health care workers and patients. The authors recommend that we expand home telehealth to a comprehensive delivery of medical care across a distributed network of hospitals and homes, linking patients to health care workers through the Internet of Medical Things using in-home equipment, including smart medical monitoring devices to create a "medical smart home." This expanded telehealth capability will help doctors care for patients flexibly, remotely, and safely as a part of standard operations and during emergencies such as a pandemic. This model of "telehomecare" is already being implemented, as shown herein with examples. The authors envision a future in which providers and hospitals transition medical care delivery to the home just as, during the COVID-19 pandemic, students adapted to distance learning and adults transitioned to remote work from home. Many of our homes in the future may have a "smart medical suite" as well as a "smart home office."

Ultrasound-guided needle insertion robotic system for percutaneous puncture

PURPOSE

Ultrasound (US)-guided percutaneous puncture technology can realize real-time, minimally invasive interventional therapy without radiation. The location accuracy of the puncture needle directly determines the precision and safety of the operation. It is a challenge for novices and young surgeons to perform a free-hand puncture guided by the ultrasound images to achieve the desired accuracy. This work aims to develop a robotic system to assist surgeons to perform percutaneous punctures with high precision.

METHODS

An US-guided puncture robot was designed to allow the mounting and control of the needle to achieve localization and insertion. The US probe fitted within the puncture robot was held by a passive arm. Moreover, the puncture robot was calibrated with a novel calibration method to achieve coordinate transformation between the robot and the US image. The system allowed the operators to plan the puncture target and puncture path on US images, and the robot performed needle insertion automatically. Five groups of puncture experiments were performed to verify the validity and accuracy of the proposed robotic system.

RESULTS

Assisted by the robotic system, the positioning and orientation accuracies of the needle insertion were 0.9 ± 0.29 mm and 0.76 ± 0.34°, respectively. These are improved compared with the results obtained with the free-hand puncture (1.82 ± 0.51 mm and 2.79 ± 1.32°, respectively). Moreover, the proposed robotic system can reduce the operation time and number of needle insertions (14.28 ± 3.21 s and one needle insertion, respectively), compared with the free-hand puncture (25.14 ± 6.09 s and 1.96 ± 0.68 needle insertions, respectively).

CONCLUSION

A robotic system for percutaneous puncture guided by US images was developed and demonstrated. The experimental results indicate that the proposed system is accurate and feasible. It can assist novices and young surgeons to perform the puncture operation with increased accuracy.

Towards Sonomyography-Based Real-Time Control of Powered Prosthesis Grasp Synergies

Sonomyography (ultrasound imaging) offers a way of classifying complex muscle activity and configuration, with higher SNR and lower hardware requirements than sEMG, using various supervised learning algorithms. The physiological image obtained from an ultrasound probe can be used to train a classification algorithm which can run on real time ultrasound images. The predicted values can then be mapped onto assistive or teleoperated robots. This paper describes the classification of ultrasound information and its subsequent mapping onto a soft robotic gripper as a step toward direct synergy control. Support Vector Classification algorithm has been used to classify ultrasound information into a set of defined states: open, closed, pinch and hook grasps. Once the model was trained with the ultrasound image data, real time input from the forearm was used to predict these states. The final predicted state output then set joint stiffnesses in the soft actuators, changing their interactions or synergies, to obtain the corresponding soft robotic gripper states. Data collection was carried out on five different test subjects for eight trials each. An average accuracy percentage of 93% was obtained averaged over all data. This real-time ultrasound-based control of a soft robotic gripper constitutes a promising step toward intuitive and robust biosignal-based control methods for robots.

WITHDRAWN: The Role of Intercostal Nerve Block Combined with Puncture Surgery Robot Controlled by Fuzzy Proportion Integral Differential Algorithm under the Guidance of MRI Image in the Treatment of Lung Cancer

This article has been withdrawn at the request of the Editor-in-Chief. The Publisher apologizes for any inconvenience this may cause. The full Elsevier Policy on Article Withdrawal can be found at https://www.elsevier.com/about/our-business/policies/article-withdrawal.

Health Care-Associated Infections and the Radiology Department

Health care-associated infections (HCAIs) are a significant concern for both health care workers (HCWs) and patients. They are a major contributing factor of disease in industrialized countries, and are responsible for significant morbidity, mortality, and a direct annual financial loss of $6-7 billion in North America alone. They are an increasingly challenging health issue due to multidrug-resistant pathogens such as methicillin-resistant Staphylococcus aureus and vancomycin-resistant Enterococci among others, along with an increasing number of susceptible patients. Over the last three decades, the risk of HCAIs has increased in the radiology department (RD) in part because of an increased number of patients visiting the department and an increase in the utilization of imaging modalities. In this review, we will discuss how patients and staff can be exposed to HCAIs in the RD, including contaminated inanimate surfaces, radiology equipment, and associated medical devices. As the role of medical imaging has extended from primarily diagnosis to include more interventions, the implementation and development of standardized infection minimization protocols and infection control procedures are vital in the RD, particularly in interventional radiology. With globalisation and the rapid movement of people regionally, nationally, and globally, there is greater risk of exposure to contagious diseases such as Ebola, especially if infected patients are undiagnosed when they travel. For effective infection control, advanced training and education of HCWs in the RD is essential. The purpose of this article is to provide an overview of HCAIs as related to activities of the RD. We will discuss the following major topics including the variety of HCAIs commonly encountered, the role of the RD in HCAIs, transmission of infections to patients and HCWs in the RD, standard infection prevention measures, and the management of susceptible/infected patients in the RD. We shall also examine the role of, and the preparedness of, HCWs, including RD technologists and interventional radiologists, who may be exposed to undiagnosed, yet infected patients. We shall conclude with a brief discussion of the role of further research related to HCAIs. Learning Objectives After the completion of this review article, the readers will • Understand the exposure and role of radiology department in health care-associated infections, • Know the causes/modes/transmission of infections in radiology department, • Be conscious of standard disinfection protocols, • Be aware of current and future strategies required for the effective control of health care-associated infection in the radiology department. This is a CME article and provides the equivalent of 2 hours of continuing education that may be applied to your professional development credit system. A 10-question multiple-choice quiz follows this reading. Please note that no formalized credit (category A) is available from CAMRT.

First-in-human evaluation of a hand-held automated venipuncture device for rapid venous blood draws

Obtaining venous access for blood sampling or intravenous (IV) fluid delivery is an essential first step in patient care. However, success rates rely heavily on clinician experience and patient physiology. Difficulties in obtaining venous access result in missed sticks and injury to patients, and typically require alternative access pathways and additional personnel that lengthen procedure times, thereby creating unnecessary costs to healthcare facilities. Here, we present the first-in-human assessment of an automated robotic venipuncture device designed to safely perform blood draws on peripheral forearm veins. The device combines ultrasound imaging and miniaturized robotics to identify suitable vessels for cannulation and robotically guide an attached needle toward the lumen center. The device demonstrated results comparable to or exceeding that of clinical standards, with a success rate of 87% on all participants (n = 31), a 97% success rate on nondifficult venous access participants (n = 25), and an average procedure time of 93 ± 30 s (n = 31). In the future, this device can be extended to other areas of vascular access such as IV catheterization, central venous access, dialysis, and arterial line placement.

Automated end-to-end blood testing at the point-of-care: Integration of robotic phlebotomy with downstream sample processing

Diagnostic blood testing is the most commonly performed clinical procedure in the world, and influences the majority of medical decisions made in hospital and laboratory settings. However, manual blood draw success rates are dependent on clinician skill and patient physiology, and results are generated almost exclusively in centralized labs from large-volume samples using labor-intensive analytical techniques. This paper presents a medical device that enables end-to-end blood testing by performing blood draws and providing diagnostic results in a fully automated fashion at the point-of-care. The system couples an image-guided venipuncture robot, developed to address the challenges of routine venous access, with a centrifuge-based blood analyzer to obtain quantitative measurements of hematology. We first demonstrate a white blood cell assay on the analyzer, using a blood mimicking fluid spiked with fluorescent microbeads, where the area of the packed bead layer is correlated with the bead concentration. Next we perform experiments to evaluate the pumping efficiency of the sample handling module. Finally, studies are conducted on the integrated device - from blood draw to analysis - using blood vessel phantoms to assess the accuracy and repeatability of the resulting white blood cell assay.

Design and Evaluation of a Robotic Device for Automated Tail Vein Cannulations in Rodent Models

Preclinical testing in rodent models is a ubiquitous part of modern biomedical research and commonly involves accessing the venous bloodstream for blood sampling and drug delivery. Manual tail vein cannulation is a time-consuming process and requires significant skill and training, particularly since improperly inserted needles can affect the experimental results and study outcomes. In this paper, we present a miniaturized, robotic medical device for automated, image-guided tail vein cannulations in rodent models. The device is composed of an actuated three degrees-of-freedom (DOFs) needle manipulator, three-dimensional (3D) near-infrared (NIR) stereo cameras, and an animal holding platform. Evaluating the system through a series of workspace simulations and free-space positioning tests, the device exhibited a sufficient work volume for the needle insertion task and submillimeter accuracy over the calibration targets. The results indicate that the device is capable of cannulating tail veins in rodent models as small as 0.3 mm in diameter, the smallest diameter vein required to target.

System Design and Development of a Robotic Device for Automated Venipuncture and Diagnostic Blood Cell Analysis

Diagnostic blood testing is the most prevalent medical procedure performed in the world and forms the cornerstone of modern health care delivery. Yet blood tests are still predominantly carried out in centralized labs using large-volume samples acquired by manual venipuncture, and no end-to-end solution from blood draw to sample analysis exists today. Our group is developing a platform device that merges robotic phlebotomy with automated diagnostics to rapidly deliver patient information at the site of the blood draw. The system couples an image-guided venipuncture robot, designed to address the challenges of routine venous access, with a centrifuge-based blood analyzer to obtain quantitative measurements of hematology. In this paper, we first present the system design and architecture of the integrated device. We then perform a series of in vitro experiments to evaluate the cannulation accuracy of the system on blood vessel phantoms. Next, we assess the effects of vessel diameter, needle gauge, flow rate, and viscosity on the rate of sample collection. Finally, we demonstrate proof-of-concept of a white cell assay on the blood analyzer using in vitro human samples spiked with fluorescently labeled microbeads.

3D Near Infrared and Ultrasound Imaging of Peripheral Blood Vessels for Real-Time Localization and Needle Guidance

This paper presents a portable imaging device designed to detect peripheral blood vessels for cannula insertion that are otherwise difficult to visualize beneath the skin. The device combines near infrared stereo vision, ultrasound, and real-time image analysis to map the 3D structure of subcutaneous vessels. We show that the device can identify adult forearm vessels and be used to guide manual insertions in tissue phantoms with increased first-stick accuracy compared to unassisted cannulation. We also demonstrate that the system may be coupled with a robotic manipulator to perform automated, image-guided venipuncture.