Ultrasound guided robotic biopsy using augmented reality and human-robot cooperative control

A venipuncture robot with decoupled position and attitude guided by near-infrared vision and force feedback

BACKGROUND

This study aims to develop a venipuncture robot to replace manual venipuncture to ease the heavy workload, lower the risk of 2019-nCoV infection, and boost venipuncture success rates.

METHOD

The robot is designed with decoupled position and attitude. It consists of a 3-degree-of-freedom positioning manipulator to locate the needle and a 3-degree-of-freedom end-effector that is always vertical to adjust the yaw and pitch angles of the needle. The near-infrared vision and laser sensors obtain the three-dimensional information of puncture positions, while the change in force detects the state feedback of punctures.

RESULTS

The experimental results demonstrate that the venipuncture robot has a compact design, flexible motion, high positioning accuracy and repeatability (0.11 and 0.04 mm), and a high success rate when puncturing the phantom.

CONCLUSION

This paper presents a decoupled position and attitude venipuncture robot guided by near-infrared vision and force feedback to replace manual venipuncture. The robot is compact, dexterous, and accurate, which helps to improve the success rate of venipuncture, and it is expected to achieve fully automatic venipuncture in the future.

A recent review and a taxonomy for hard and soft tissue visualization-based mixed reality

BACKGROUND

Mixed reality (MR) visualization is gaining popularity in image-guided surgery (IGS) systems, especially for hard and soft tissue surgeries. However, a few MR systems are implemented in real time. Some factors are limiting MR technology and creating a difficulty in setting up and evaluating the MR system in real environments. Some of these factors include: the end users are not considered, the limitations in the operating room, and the medical images are not fully unified into the operating interventions.

METHODOLOGY

The purpose of this article is to use Data, Visualization processing, and View (DVV) taxonomy to evaluate the current MR systems. DVV includes all the components required to be considered and validated for the MR used in hard and soft tissue surgeries. This taxonomy helps the developers and end users like researchers and surgeons to enhance MR system for the surgical field.

RESULTS

We evaluated, validated, and verified the taxonomy based on system comparison, completeness, and acceptance criteria. Around 24 state-of-the-art solutions that are picked relate to MR visualization, which is then used to demonstrate and validate this taxonomy. The results showed that most of the findings are evaluated and others are validated.

CONCLUSION

The DVV taxonomy acts as a great resource for MR visualization in IGS. State-of-the-art solutions are classified, evaluated, validated, and verified to elaborate the process of MR visualization during surgery. The DVV taxonomy provides the benefits to the end users and future improvements in MR.

Hybrid simulation using mixed reality for interventional ultrasound imaging training

PURPOSE

Ultrasound (US) imaging offers advantages over other imaging modalities and has become the most widespread modality for many diagnostic and interventional procedures. However, traditional 2D US requires a long training period, especially to learn how to manipulate the probe. A hybrid interactive system based on mixed reality was designed, implemented and tested for hand-eye coordination training in diagnostic and interventional US.

METHODS

A hybrid simulator was developed integrating a physical US phantom and a software application with a 3D virtual scene. In this scene, a 3D model of the probe with its relative scan plane is coherently displayed with a 3D representation of the phantom internal structures. An evaluation study of the diagnostic module was performed by recruiting thirty-six novices and four experts. The performances of the hybrid (HG) versus physical (PG) simulator were compared. After the training session, each novice was required to visualize a particular target structure. The four experts completed a 5-point Likert scale questionnaire.

RESULTS

Seventy-eight percentage of the HG novices successfully visualized the target structure, whereas only 45% of the PG reached this goal. The mean scores from the questionnaires were 5.00 for usefulness, 4.25 for ease of use, 4.75 for 3D perception, and 3.25 for phantom realism.

CONCLUSIONS

The hybrid US training simulator provides ease of use and is effective as a hand-eye coordination teaching tool. Mixed reality can improve US probe manipulation training.

Robotic ultrasound systems in medicine

Robots ultrasound (RUS) can be defined as the combination of ultrasound imaging with a robotic system in medical interventions. With their potential for high precision, dexterity, and repeatability, robots are often uniquely suited for ultrasound integration. Although the field is relatively young, it has already generated a multitude of robotic systems for application in dozens of medical procedures. This paper reviews the robotic ultrasound systems that have been developed over the past two decades and describes their potential impact on modern medicine. The RUS projects reviewed include extracorporeal devices, needle guidance systems, and intraoperative systems.