Robotic ultrasound systems in medicine

Development and Preliminary Evaluation of a Robotic Device for MRI-Guided Needle Breast Biopsy

This study concerns the development and evaluation of a simple and ergonomic robotic system for Magnetic Resonance Imaging (MRI)-guided needle breast biopsy with lateral needle approach. The device comprises two piezoelectrically actuated linear motion stages intended to align a needle supporter with the target for manual needle insertion. The device demonstrated submillimeter accuracy and safe operation within a 3 T clinical MRI scanner. In phantom studies, tumor simulators of varying sizes were successfully targeted in both laboratory and MRI settings.

Satisfaction analysis of 5G remote ultrasound robot for diagnostics based on a structural equation model

Objectives

With the increasing application of 5G remote ultrasound robots in healthcare, robust methods are in critical demand to assess participant satisfaction and identify its influencing factors. At present, there is limited empirical research on multi-parametric and multidimensional satisfaction evaluation of participants with 5G remote ultrasound robot examination. Previous studies have demonstrated that structural equation modeling (SEM) effectively integrates various statistical techniques to examine the relationships among multiple variables. Therefore, this study aimed to evaluate the satisfaction of participants with 5G remote ultrasound robot examination and its influencing factors using SEM.

Methods

Between April and June 2022, 213 participants from Wuhan Automobile Manufacturing Company underwent remote ultrasound examinations using the MGIUS-R3 remote ultrasound robot system. After these examinations, the participants evaluated the performance of the 5G remote ultrasound robot based on their personal experiences and emotional responses. They completed a satisfaction survey using a self-developed questionnaire, which included 19 items across five dimensions: examination efficiency, examination perception, communication perception, value perception, and examination willingness. A SEM was established to assess the satisfaction of participants with the 5G remote ultrasound robot examinations and the influencing factors.

Results

A total of 201 valid questionnaires were collected. The overall satisfaction of participants with the 5G remote ultrasound robot examination was 45.43 ± 11.60, with 169 participants (84%) expressing satisfaction. In the path hypothesis relationship test, the dimensions of examination efficiency, examination perception, communication perception, and value perception had positive effects on satisfaction, with standardized path coefficients of 0.168, 0.170, 0.175, and 0.191. Satisfaction had a direct positive effect on examination willingness, with a standardized path coefficient of 0.260. Significant differences were observed across different educational levels in the dimensions of examination perception, communication perception, value perception, and examination willingness. Participants with different body mass indices also showed significant differences in examination perception; all p-values were less than 0.05.

Conclusion

In this study, value perception was identified as the most significant factor influencing satisfaction. It could be improved by enhancing participants' understanding of the accuracy and safety of 5G remote ultrasound robot examinations. This enhances satisfaction and the willingness to undergo examinations. Such improvements not only facilitate the widespread adoption of this technology but also promote the development of telemedicine services.

Autonomous Robotic Ultrasound Vascular Imaging System With Decoupled Control Strategy for External-Vision-Free Environments

OBJECTIVE

Ultrasound (US) probes scan over the surface of the human body to acquire US images in clinical vascular US diagnosis. However, due to the deformation and specificity of different human surfaces, the relationship between the scan trajectory of the skin and the internal tissues is not fully correlated, which poses a challenge for autonomous robotic US imaging in a dynamic and external-vision-free environment. Here, we propose a decoupled control strategy for autonomous robotic vascular US imaging in an environment without external vision.

METHODS

The proposed system is divided into outer-loop posture control and inner-loop orientation control, which are separately determined by a deep learning (DL) agent and a reinforcement learning (RL) agent. First, we use a weakly supervised US vessel segmentation network to estimate the probe orientation. In the outer loop control, we use a force-guided reinforcement learning agent to maintain a specific angle between the US probe and the skin in the dynamic imaging processes. Finally, the orientation and the posture are integrated to complete the imaging process.

RESULTS

Evaluation experiments on several volunteers showed that our RUS could autonomously perform vascular imaging in arms with different stiffness, curvature, and size without additional system adjustments. Furthermore, our system achieved reproducible imaging and reconstruction of dynamic targets without relying on vision-based surface information.

CONCLUSION AND SIGNIFICANCE

Our system and control strategy provides a novel framework for the application of US robots in complex and external-vision-free environments.

SAPM: Self-Adaptive Parallel Manipulator with Pose and Force Adjustment for Robotic Ultrasonography

Robotic ultrasonography potentially acts as an essential aid to medical diagnosis. To overcome the limitations in robotic ultrasonography, in this paper, we proposed a novel self-adaptive parallel manipulator (SAPM) that can automatically adjust the ultrasound (US) probe pose to adapt to various contours of scanned areas, provide approximate constant operating forces/torques, achieve mechanical measurement, and cushion undesired produced forces. A novel parallel adjustment mechanism is proposed to attain automatic pose adjustment with 3 degrees of freedom (DOFs). This mechanism enables the US probe to adapt to different scanned areas and to perform the scanning with approximate constant forces and torques. Besides, we present a mechanical measurement and safety protection method that can be integrated into the SAPM and used as operation status monitoring and early warning during scanning procedures by capturing operating forces and torques. Experiments were carried out to calibrate the measurement and buffer units and evaluate the performance of the SAPM. Experimental results show the ability of the SAPM to provide 3-DoFs motion and operating force/torque measurement and automatically adjust the US probe pose to capture US images of equally good quality compared to a manual sonographer scan. Moreover, it has characteristics similar to soft robots that could significantly improve operation safety, and could be extended to some other engineering or medical applications.

Robotic ultrasound imaging: State-of-the-art and future perspectives

Ultrasound (US) is one of the most widely used modalities for clinical intervention and diagnosis due to the merits of providing non-invasive, radiation-free, and real-time images. However, free-hand US examinations are highly operator-dependent. Robotic US System (RUSS) aims at overcoming this shortcoming by offering reproducibility, while also aiming at improving dexterity, and intelligent anatomy and disease-aware imaging. In addition to enhancing diagnostic outcomes, RUSS also holds the potential to provide medical interventions for populations suffering from the shortage of experienced sonographers. In this paper, we categorize RUSS as teleoperated or autonomous. Regarding teleoperated RUSS, we summarize their technical developments, and clinical evaluations, respectively. This survey then focuses on the review of recent work on autonomous robotic US imaging. We demonstrate that machine learning and artificial intelligence present the key techniques, which enable intelligent patient and process-specific, motion and deformation-aware robotic image acquisition. We also show that the research on artificial intelligence for autonomous RUSS has directed the research community toward understanding and modeling expert sonographers' semantic reasoning and action. Here, we call this process, the recovery of the "language of sonography". This side result of research on autonomous robotic US acquisitions could be considered as valuable and essential as the progress made in the robotic US examination itself. This article will provide both engineers and clinicians with a comprehensive understanding of RUSS by surveying underlying techniques. Additionally, we present the challenges that the scientific community needs to face in the coming years in order to achieve its ultimate goal of developing intelligent robotic sonographer colleagues. These colleagues are expected to be capable of collaborating with human sonographers in dynamic environments to enhance both diagnostic and intraoperative imaging.

Fully Automatic Dual-Probe Lung Ultrasound Scanning Robot for Screening Triage

Two-dimensional lung ultrasound (LUS) has widely emerged as a rapid and noninvasive imaging tool for the detection and diagnosis of coronavirus disease 2019 (COVID-19). However, image differences will be magnified due to changes in ultrasound (US) imaging experience, such as US probe attitude control and force control, which will directly affect the diagnosis results. In addition, the risk of virus transmission between sonographer and patients is increased due to frequent physical contact. In this study, a fully automatic dual-probe US scanning robot for the acquisition of LUS images is proposed and developed. Furthermore, the trajectory was optimized based on the velocity look-ahead strategy, the stability of contact force of the system and the scanning efficiency were improved by 24.13% and 29.46%, respectively. Also, the control ability of the contact force of robotic automatic scanning was 34.14 times higher than that of traditional manual scanning, which significantly improves the smoothness of scanning. Importantly, there was no significant difference in image quality obtained by robotic automatic scanning and manual scanning. Furthermore, the scanning time for a single person is less than 4 min, which greatly improves the efficiency of screening triage of group COVID-19 diagnosis and suspected patients and reduces the risk of virus exposure and spread.

Visual Perception and Convolutional Neural Network-Based Robotic Autonomous Lung Ultrasound Scanning Localization System

Under the situation of severe COVID-19 epidemic, lung ultrasound (LUS) has been proved to be an effective and convenient method to diagnose and evaluate the extent of respiratory disease. However, the traditional clinical ultrasound (US) scanning requires doctors not only to be in close contact with patients but also to have rich experience. In order to alleviate the shortage of medical resources and reduce the work stress and risk of infection for doctors, we propose a visual perception and convolutional neural network (CNN)-based robotic autonomous LUS scanning localization system to realize scanned target recognition, probe pose solution and movement, and the acquisition of US images. The LUS scanned targets are identified through the target segmentation and localization algorithm based on the improved CNN, which is using the depth camera to collect the image information; furthermore, the method based on multiscale compensation normal vector is used to solve the attitude of the probe; finally, a position control strategy based on force feedback is designed to optimize the position and attitude of the probe, which can not only obtain high-quality US images but also ensure the safety of patients and the system. The results of human LUS scanning experiment verify the accuracy and feasibility of the system. The positioning accuracy of the scanned targets is 15.63 ± 0.18 mm, and the distance accuracy and rotation angle accuracy of the probe position calculation are 6.38 ± 0.25 mm and 8.60° ±2.29° , respectively. More importantly, the obtained high-quality US images can clearly capture the main pathological features of the lung. The system is expected to be applied in clinical practice.

A review on tele-manipulators for remote diagnostic procedures and surgery

With modern medicine and healthcare services improving in leaps and bounds, the integration of telemedicine has helped in expanding these specialised healthcare services to remote locations. Healthcare telerobotic systems form a component of telemedicine, which allows medical intervention from a distance. It has been nearly 40 years since a robotic technology, PUMA 560, was introduced to perform a stereotaxic biopsy in the brain. The use of telemanipulators for remote surgical procedures began around 1995, with the Aesop, the Zeus, and the da Vinci robotic surgery systems. Since then, the utilisation of robots has steadily increased in diverse healthcare disciplines, from clinical diagnosis to telesurgery. The telemanipulator system functions in a master–slave protocol mode, with the doctor operating the master system, aided by audio-visual and haptic feedback. Based on the control commands from the master, the slave system, a remote manipulator, interacts directly with the patient. It eliminates the requirement for the doctor to be physically present in the spatial vicinity of the patient by virtually bringing expert-guided medical services to them. Post the Covid-19 pandemic, an exponential surge in the utilisation of telerobotic systems has been observed. This study aims to present an organised review of the state-of-the-art telemanipulators used for remote diagnostic procedures and surgeries, highlighting their challenges and scope for future research and development.

Suppression of Clothing-Induced Acoustic Attenuation in Robotic Auscultation

For patients who are often embarrassed and uncomfortable when exposing their breasts and having them touched by physicians of different genders during auscultation, we are developing a robotic system that performs auscultation over clothing. As the technical issue, the sound obtained through the clothing is often attenuated. This study aims to investigate clothing-induced acoustic attenuation and develop a suppression method for it. Because the attenuation is due to the loss of energy as sound propagates through a medium with viscosity, we hypothesized that the attenuation is improved by compressing clothing and shortening the sound propagation distance. Then, the amplitude spectrum of the heart sound was obtained over clothes of different thicknesses and materials in a phantom study and human trial at varying contact forces with a developed passive-actuated end-effector. Our results demonstrate the feasibility of the attenuation suppression method by applying an optimum contact force, which varied according to the clothing condition. In the phantom experiments, the attenuation rate was improved maximumly by 48% when applying the optimal contact force (1 N). In human trials, the attenuation rate was under the acceptable attenuation (40%) when applying the optimal contact force in all combinations in each subject. The proposed method promises the potential of robotic auscultation toward eliminating gender bias.

Strategy for automatic ultrasound (US) probe positioning in robot-assisted ultrasound guided radiation therapy

Objective. As part of image-guided radiotherapy, ultrasound-guided radiotherapy is currently already in use and under investigation for robot assisted systems Ipsen 2021. It promises a real-time tumor localization during irradiation (intrafractional) without extra dose. The ultrasound probe is held and guided by a robot. However, there is a lack of basic safety mechanisms and interaction strategies to enable a safe clinical procedure. In this study we investigate potential positioning strategies with safety mechanisms for a safe robot-human-interaction.Approach. A compact setup of ultrasound device, lightweight robot, tracking camera, force sensor and control computer were integrated in a software application to represent a potential USgRT setup. For the realization of a clinical procedure, positioning strategies for the ultrasound head with the help of the robot were developed, implemented, and tested. In addition, basic safety mechanisms for the robot have been implemented, using the integrated force sensor, and have been tested by intentional collisions.Main results. Various positioning methods from manual guidance to completely automated procedures were tested. Robot-guided methods achieved higher positioning accuracy and were faster in execution compared to conventional hand-guided methods. The developed safety mechanisms worked as intended and the detected collision force were below 20 N.Significance. The study demonstrates the feasibility of a new approach for safe robotic ultrasound imaging, with a focus on abdominal usage (liver, prostate, kidney). The safety measures applied here can be extended to other human-robot interactions and present the basic for further studies in medical applications.

Towards fully automated robotic platform for remote auscultation

BACKGROUND

Since most developed countries are facing an increase in the number of patients per healthcare worker due to a declining birth rate and an ageing population, relatively simple and safe diagnosis tasks may need to be performed using robotics and automation technologies, without specialists and hospitals. This study presents an automated robotic platform for remote auscultation, which is a highly cost-effective screening tool for detecting abnormal clinical signs.

METHOD

The developed robotic platform is composed of a 6-degree-of-freedom cooperative robotic arm, LiDAR camera, and a spring-based mechanism holding an electric stethoscope. The platform enables autonomous stethoscope positioning based on external body information acquired using the LiDAR camera-based multi-way registration; the platform also ensures safe and flexible contact, maintaining the contact force within a certain range through the passive-actuated mechanism.

RESULTS

Our preliminary results confirm that the robotic platform enables estimation of the landing positions required for cardiac examinations based on the depth and landmark information of the body surface. It also handles the stethoscope while maintaining the contact force without relying on the push-in displacement by the robotic arm.

CONCLUSION

The developed robotic platform enables the estimation of the landing positions and handling the stethoscope while maintaining the contact force, which promises the potential of automatic remote auscultation.

A Novel Ultrasound Robot with Force/torque Measurement and Control for Safe and Efficient Scanning

Medical ultrasound is of increasing importance in medical diagnosis and intraoperative assistance and possesses great potential advantages when integrated with robotics. However, some concerns, including the operation efficiency, operation safety, image quality, and comfort of patients, remain after introducing robotics into medical ultrasound. In this paper, an ultrasound robot integrating a force control mechanism, force/torque measurement mechanism, and online adjustment method, is proposed to overcome the current limitations. The ultrasound robot can measure operating forces and torques, provide adjustable constant operating forces, eliminate great operating forces introduced by accidental operations, and achieve various scanning depths based on clinical requirements. The proposed ultrasound robot would potentially facilitate sonographers to find the targets quickly, improve operation safety and efficiency, and decrease patients' discomfort. Simulations and experiments were carried out to evaluate the performance of the ultrasound robot. Experimental results show that the proposed ultrasound robot is able to detect operating force in the z-direction and torques around the x- and y- directions with errors of 3.53% F.S., 6.68% F.S., and 6.11% F.S., respectively, maintain the constant operating force with errors of less than 0.57N, and achieve various scanning depths for target searching and imaging. This proposed ultrasound robot has good performance and would potentially be used in medical ultrasound.

A-SEE: Active-Sensing End-effector Enabled Probe Self-Normal-Positioning for Robotic Ultrasound Imaging Applications

Conventional manual ultrasound (US) imaging is a physically demanding procedure for sonographers. A robotic US system (RUSS) has the potential to overcome this limitation by automating and standardizing the imaging procedure. It also extends ultrasound accessibility in resource-limited environments with the shortage of human operators by enabling remote diagnosis. During imaging, keeping the US probe normal to the skin surface largely benefits the US image quality. However, an autonomous, real-time, low-cost method to align the probe towards the direction orthogonal to the skin surface without pre-operative information is absent in RUSS. We propose a novel end-effector design to achieve self-normal-positioning of the US probe. The end-effector embeds four laser distance sensors to estimate the desired rotation towards the normal direction. We then integrate the proposed end-effector with a RUSS system which allows the probe to be automatically and dynamically kept to normal direction during US imaging. We evaluated the normal positioning accuracy and the US image quality using a flat surface phantom, an upper torso mannequin, and a lung ultrasound phantom. Results show that the normal positioning accuracy is 4.17 ± 2.24 degrees on the flat surface and 14.67 ± 8.46 degrees on the mannequin. The quality of the RUSS collected US images from the lung ultrasound phantom was equivalent to that of the manually collected ones.

Dual Mode pHRI-teleHRI Control System with a Hybrid Admittance-Force Controller for Ultrasound Imaging

The COVID-19 pandemic has brought unprecedented extreme pressure on the medical system due to the physical distance policy, especially for procedures such as ultrasound (US) imaging, which are usually carried out in person. Tele-operation systems are a promising way to avoid physical human-robot interaction (pHRI). However, the system usually requires another robot on the remote doctor side to provide haptic feedback, which makes it expensive and complex. To reduce the cost and system complexity, in this paper, we present a low-cost, easy-to-use, dual-mode pHRI-teleHRI control system with a custom-designed hybrid admittance-force controller for US imaging. The proposed system requires only a tracking camera rather than a sophisticated robot on the remote side. An audio feedback is designed for replacing haptic feedback on the remote side, and its sufficiency is experimentally verified. The experimental results indicate that the designed hybrid controller can significantly improve the task performance in both modes. Furthermore, the proposed system enables the user to conduct US imaging while complying with the physical distance policy, and allows them to seamlessly switch modes from one to another in an online manner. The novel system can be easily adapted to other medical applications beyond the pandemic, such as tele-healthcare, palpation, and auscultation.

Remote Ultrasound Scan Procedures with Medical Robots: Towards New Perspectives between Medicine and Engineering

Background

This review explores state-of-the-art teleoperated robots for medical ultrasound scan procedures, providing a comprehensive look including the recent trends arising from the COVID-19 pandemic.

Methods

Physicians' experience is included to indicate the importance of their role in the design of improved medical robots. From this perspective, novel classes of equipment for remote diagnostics based on medical robotics are discussed in terms of innovative engineering technologies.

Results

Relevant literature is reviewed under the system engineering point of view, organizing the discussion on the basis of the main technological focus of each contribution.

Conclusions

This contribution is aimed at stimulating new research to obtain faster results on teleoperated robotics for ultrasound diagnostics in response to the high demand raised by the ongoing pandemic.

Enabling quantitative robot-assisted compressional elastography via the extended Kalman filter

Compressional or quasi-static elastography has demonstrated the capability to detect occult cancers in a variety of tissue types, however it has a serious limitation in that the resulting elastograms are generally qualitative whereas other forms of elastography, such as shear-wave, can produce absolute measures of elasticity for histopathological classification. We address this limitation by introducing a stochastic method using an extended Kalman filter and robot-assistance to obtain quantitative elastograms which are resilient to measurement noise and system uncertainty. In this paper, the probabilistic framework is described, which utilizes many ultrasound acquisitions obtained from multiple palpations, to fuse data and uncertainty from a robotic manipulator's joint encoders and force/torque sensor directly into the inverse reconstruction of the elastogram. Quantitative results are demonstrated over homogeneous and inclusion gelatin phantoms using a seven degree of freedom manipulator for a range of initial elasticity assumptions. Results imply resilience to poorly assumed initial conditions as all trials were within 5 kPa of the elasticity measured by a mechanical testing system. Moreover, the presence or absence of an inclusion is clear in all reconstructed elastograms even when artifacts are present in displacement fields, indicating further robustness to measurement noise. The proposed stochastic method allows fusion of data from a robot's sensors directly into compressional elastography image reconstruction which may stabilize optimization and improve accuracy. This approach provides a mathematical framework to readily incorporate measurements from additional sensors in future applications which may extend the capabilities of compressional elastography beyond that of producing quantitative elasticity measurements.

Autonomous Scanning Target Localization for Robotic Lung Ultrasound Imaging

Under the ceaseless global COVID-19 pandemic, lung ultrasound (LUS) is the emerging way for effective diagnosis and severeness evaluation of respiratory diseases. However, close physical contact is unavoidable in conventional clinical ultrasound, increasing the infection risk for health-care workers. Hence, a scanning approach involving minimal physical contact between an operator and a patient is vital to maximize the safety of clinical ultrasound procedures. A robotic ultrasound platform can satisfy this need by remotely manipulating the ultrasound probe with a robotic arm. This paper proposes a robotic LUS system that incorporates the automatic identification and execution of the ultrasound probe placement pose without manual input. An RGB-D camera is utilized to recognize the scanning targets on the patient through a learning-based human pose estimation algorithm and solve for the landing pose to attach the probe vertically to the tissue surface; A position/force controller is designed to handle intraoperative probe pose adjustment for maintaining the contact force. We evaluated the scanning area localization accuracy, motion execution accuracy, and ultrasound image acquisition capability using an upper torso mannequin and a realistic lung ultrasound phantom with healthy and COVID-19-infected lung anatomy. Results demonstrated the overall scanning target localization accuracy of 19.67 ± 4.92 mm and the probe landing pose estimation accuracy of 6.92 ± 2.75 mm in translation, 10.35 ± 2.97 deg in rotation. The contact force-controlled robotic scanning allowed the successful ultrasound image collection, capturing pathological landmarks.

Autonomic Robotic Ultrasound Imaging System Based on Reinforcement Learning

OBJECTIVE

In this paper, we introduce an autonomous robotic ultrasound (US) imaging system based on reinforcement learning (RL). The proposed system and framework are committed to controlling the US probe to perform fully autonomous imaging of a soft, moving and marker-less target based only on single RGB images of the scene.

METHODS

We propose several different approaches and methods to achieve the following objectives: real-time US probe controlling, soft surface constant force tracking and automatic imaging. First, to express the state of the robotic US imaging task, we proposed a state representation model to reduce the dimensionality of the imaging state and encode the force and US information into the scene image space. Then, an RL agent is trained by a policy gradient theorem based RL model with the single RGB image as the only observation. To achieve adaptable constant force tracking between the US probe and the soft moving target, we propose a force-to-displacement control method based on an admittance controller.

RESULTS

In the simulation experiment, we verified the feasibility of the integrated method. Furthermore, we evaluated the proposed force-to-displacement method to demonstrate the safety and effectiveness of adaptable constant force tracking. Finally, we conducted phantom and volunteer experiments to verify the feasibility of the method on a real system.

CONCLUSION

The experiments indicated that our approaches were stable and feasible in the autonomic and accurate control of the US probe.

SIGNIFICANCE

The proposed system has potential application value in the image-guided surgery and robotic surgery.

Force-guided autonomous robotic ultrasound scanning control method for soft uncertain environment

PURPOSE

Autonomous ultrasound imaging by robotic ultrasound scanning systems in complex soft uncertain clinical environments is important and challenging to assist in therapy. To cope with the complex environment faced by the ultrasound probe during the scanning process, we propose an autonomous robotic ultrasound (US) control method based on reinforcement learning (RL) model to build the relationship between the environment and the system. The proposed method requires only contact force as input information to achieve robot control of the posture and contact force of the probe without any a priori information about the target and the environment.

METHODS

First, an RL agent is proposed and trained by a policy gradient theorem-based RL model with the 6-degree-of-freedom (DOF) contact force of the US probe to learn the relationship between contact force and output force directly. Then, a force control strategy based on the admittance controller is proposed for synchronous force, orientation and position control by defining the desired contact force as the action space.

RESULTS

The proposed method was evaluated via collected US images, contact force and scan trajectories by scanning an unknown soft phantom. The experimental results indicated that the proposed method differs from the free-hand scanned approach in the US images within 3 ± 0.4%. The analysis results of contact forces and trajectories indicated that our method could make stable scanning processes on a soft uncertain skin surface and obtained US images.

CONCLUSION

We propose a concise and efficient force-guided US robot scanning control method for soft uncertain environment based on reinforcement learning. Experimental results validated our method's feasibility and validity for complex skin surface scanning, and the volunteer experiments indicated the potential application value in the complex clinical environment of robotic US imaging system especially with limited visual information.

Tele-Operative Low-Cost Robotic Lung Ultrasound Scanning Platform for Triage of COVID-19 Patients

Novel severe acute respiratory syndrome coronavirus 2 (COVID-19) has become a pandemic of epic proportions, and global response to prepare health systems worldwide is of utmost importance. 2-dimensional (2D) lung ultrasound (LUS) has emerged as a rapid, noninvasive imaging tool for diagnosing COVID-19 infected patients. Concerns surrounding LUS include the disparity of infected patients and healthcare providers, and importantly, the requirement for substantial physical contact between the patient and operator, increasing the risk of transmission. New variants of COVID-19 will continue to emerge; therefore, mitigation of the virus's spread is of paramount importance. A tele-operative robotic ultrasound platform capable of performing LUS in COVID-19 infected patients may be of significant benefit, especially in low- and middle-income countries. The authors address the issues mentioned above surrounding the use of LUS in COVID-19 infected patients and the potential for extension of this technology in a resource-limited environment. Additionally, first-time application, feasibility, and safety were validated in healthy subjects. Preliminary results demonstrate that our platform allows for the successful acquisition and application of robotic LUS in humans.

Towards automated ultrasound imaging-robotic image acquisition in liver and prostate for long-term motion monitoring

Real-time volumetric (4D) ultrasound has shown high potential for diagnostic and therapy guidance tasks. One of the main drawbacks of ultrasound imaging to date is the reliance on manual probe positioning and the resulting user dependence. Robotic assistance could help overcome this issue and facilitate the acquisition of long-term image data to observe dynamic processesin vivoover time. The aim of this study is to assess the feasibility of robotic probe manipulation and organ motion quantification during extended imaging sessions. The system consists of a collaborative robot and a 4D ultrasound system providing real-time data access. Five healthy volunteers received liver and prostate scans during free breathing over 30 min. Initial probe placement was performed with real-time remote control with a predefined contact force of 10 N. During scan acquisition, the probe position was continuously adjusted to the body surface motion using impedance control. Ultrasound volumes, the pose of the end-effector and the estimated contact forces were recorded. For motion analysis, one anatomical landmark was manually annotated in a subset of ultrasound frames for each experiment. Probe contact was uninterrupted over the entire scan duration in all ten sessions. Organ drift and imaging artefacts were successfully compensated using remote control. The median contact force along the probe's longitudinal axis was 10.0 N with maximum values of 13.2 and 21.3 N for liver and prostate, respectively. Forces exceeding 11 N only occurred in 0.3% of the time. Probe and landmark motion were more pronounced in the liver, with median interquartile ranges of 1.5 and 9.6 mm, compared to 0.6 and 2.7 mm in the prostate. The results show that robotic ultrasound imaging with dynamic force control can be used for stable, long-term imaging of anatomical regions affected by motion. The system facilitates the acquisition of 4D image datain vivoover extended scanning periods for the first time and holds the potential to be used for motion monitoring for therapy guidance as well as diagnostic tasks.

Towards Standardized Acquisition with a Dual-probe Ultrasound Robot for Fetal Imaging

Standardized acquisitions and diagnoses using robots and AI would potentially increase the general usability and reliability of medical ultrasound. Working towards this prospect, this paper presents the recent developments of a standardized acquisition workflow using a novel dual-probe ultrasound robot, for a project known as intelligent Fetal Imaging and Diagnosis (iFIND). The workflow includes an abdominal surface mapping step to obtain a non-parametric spline surface, a rule-based end-point calculation method to position each individual joint, and a motor synchronization method to achieve a smooth motion towards a target point. The design and implementation of the robot are first presented in this paper and the proposed workflow is then explained in detail with simulation and volunteer experiments performed and analyzed. The closed-form analytical solution to the specific motion planning problem has demonstrated a reliable performance controlling the robot to move towards the expected scanning areas and the calculated proximity of the robot to the surface shows that the robot maintains a safe distance while moving around the abdomen. The volunteer study has successfully demonstrated the reliable working and controllability of the robot in terms of acquiring desired ultrasound views. Our future work will focus on improving the motion planning, and on integrating the proposed standardized acquisition workflow with newly- developed ultrasound image processing methods to obtain diagnostic results in an accurate and consistent way.

A Constant-Force End-Effector With Online Force Adjustment for Robotic Ultrasonography

In this letter, we propose a novel constant-force end-effector (CFEE) to address current limitations in robotic ultrasonography. The CFEE uses a parallel, motor-spring-based solution to precisely generate constant operating forces over a wide range and enable the ultrasound (US) probe to adapt to the abdominal contours autonomously. A displacement measurement unit was developed to realize the acquisition of probe position and precise control of the operating force. Moreover, the operating force can be adjusted online to maintain safety and continuity of operation. Simulations and experiments were carried out to evaluate the performance. Results show that the proposed CFEE can provide constant forces of 4-12 N with displacements of 0-8 mm. The maximum relative error of force generation is 8.28%, and the accuracy and precision for displacement measurement are 0.29 mm and ±0.16 mm, respectively. Various operating forces can be adjusted online during the same operation. Ultrasound images acquired by the proposed CFEE are of equally good quality compared to a manual sonographer scan. The proposed CFEE would have potential further medical applications.

Medical Robotics for Ultrasound Imaging: Current Systems and Future Trends

Purpose of Review

This review provides an overview of the most recent robotic ultrasound systems that have contemporary emerged over the past five years, highlighting their status and future directions. The systems are categorized based on their level of robot autonomy (LORA).

Recent Findings

Teleoperating systems show the highest level of technical maturity. Collaborative assisting and autonomous systems are still in the research phase, with a focus on ultrasound image processing and force adaptation strategies. However, missing key factors are clinical studies and appropriate safety strategies. Future research will likely focus on artificial intelligence and virtual/augmented reality to improve image understanding and ergonomics.

Summary

A review on robotic ultrasound systems is presented in which first technical specifications are outlined. Hereafter, the literature of the past five years is subdivided into teleoperation, collaborative assistance, or autonomous systems based on LORA. Finally, future trends for robotic ultrasound systems are reviewed with a focus on artificial intelligence and virtual/augmented reality.

Application of a Robotic Tele-Echography System for COVID-19 Pneumonia

To date, coronavirus disease 2019 (COVID-19) has infected millions of people worldwide. Ultrasound plays an indispensable role in the diagnosis, monitoring, and follow-up of patients with COVID-19. In this study, we used a robotic tele-echography system based on a 5G communication network for remote diagnosis. The system has great potential for lung, heart, and vasculature information, medical staff protection, and resource sharing, can be a valuable tool for treating patients during the pandemic, and can be expected to expand to more specialized fields.

Ultrasound-guided needle insertion robotic system for percutaneous puncture

PURPOSE

Ultrasound (US)-guided percutaneous puncture technology can realize real-time, minimally invasive interventional therapy without radiation. The location accuracy of the puncture needle directly determines the precision and safety of the operation. It is a challenge for novices and young surgeons to perform a free-hand puncture guided by the ultrasound images to achieve the desired accuracy. This work aims to develop a robotic system to assist surgeons to perform percutaneous punctures with high precision.

METHODS

An US-guided puncture robot was designed to allow the mounting and control of the needle to achieve localization and insertion. The US probe fitted within the puncture robot was held by a passive arm. Moreover, the puncture robot was calibrated with a novel calibration method to achieve coordinate transformation between the robot and the US image. The system allowed the operators to plan the puncture target and puncture path on US images, and the robot performed needle insertion automatically. Five groups of puncture experiments were performed to verify the validity and accuracy of the proposed robotic system.

RESULTS

Assisted by the robotic system, the positioning and orientation accuracies of the needle insertion were 0.9 ± 0.29 mm and 0.76 ± 0.34°, respectively. These are improved compared with the results obtained with the free-hand puncture (1.82 ± 0.51 mm and 2.79 ± 1.32°, respectively). Moreover, the proposed robotic system can reduce the operation time and number of needle insertions (14.28 ± 3.21 s and one needle insertion, respectively), compared with the free-hand puncture (25.14 ± 6.09 s and 1.96 ± 0.68 needle insertions, respectively).

CONCLUSION

A robotic system for percutaneous puncture guided by US images was developed and demonstrated. The experimental results indicate that the proposed system is accurate and feasible. It can assist novices and young surgeons to perform the puncture operation with increased accuracy.

Roboterbasierte Ultraschallsteuerung

Hintergrund

Die periphere endovaskuläre Chirurgie ist nach wie vor durch die Anwendung von Röntgenstrahlen und Röntgenkontrastmittel für die intraprozedurale Navigation der Instrumentarien ein Verfahren mit potenziellen Risiken und Nebenwirkungen.

Projektziel

Ziel des RoGUS-PAD (Robotic-Guided Ultrasound System for Peripheral Arterial Disease)-Projektes ist die Entwicklung eines roboterbasierten ultraschallgesteuerten Assistenzsystems für periphere endovaskuläre Interventionen zur Verringerung und ggf. Vermeidung von Röntgenstrahlung und Röntgenkontrastmittel sowie Verbesserung der Echtzeitvisualisierung.

Material und Methoden

Für die Bildgebung wurde ein 2‑D-Ultraschall-Lineartastkopf (L12‑3, Philips Healthcare, Best, Niederlande) am Endeffektor eines Roboterarms (LBR iiwa 7 R800, KUKA, Augsburg, Deutschland) montiert. Die ersten Versuche wurden an einem eigens für dieses Projekt entwickelten ultraschallfähigen Phantom durchgeführt. Die Bildverarbeitung und Robotersteuerung erfolgten durch ein speziell entwickeltes Programm in C++.

Ergebnisse

Zur Testung der technischen Umsetzbarkeit des Projektes konnten wir einen semiautomatischen 2‑D-Ultraschallscan einer peripheren Arterie am Phantom durchführen. In 27 von 30 Durchläufen zeigte sich ein erfolgreicher Scanvorgang.

Schlussfolgerung

Unsere ersten Ergebnisse bestätigten, dass die Entwicklung eines roboterbasierten Assistenzsystems für ultraschallgesteuerte periphere endovaskuläre Interventionen technisch umsetzbar ist. Dies stützt unsere Ambitionen einer Translation des Systems in die tägliche klinische Praxis.

Force Feedback Assistance in Remote Ultrasound Scan Procedures

In this contribution, we approached a new aspect in robotic applications. We investigated human–machine modeling for remote ultrasound scan equipment. While robotic systems for ultrasound scan applications with remote operations have been widely studied, in this research, remote force-feedback control was tested. The goal is for the human operator to receive, as physical input, the correct force perception transmitted by the remote ultrasound scan equipment in analyzing the body of the patient. Two principal aspects were investigated. The first was an artificial body model to receive the control signals from the remote equipment. The second aspect was to study a suitable feedback control law that attempts to compensate for the uncertainty between the artificial body and the patient’s body, while also taking into account the transmission delay. Therefore, the task was to give the operator relevant information while considering the force effect; thus, providing a reliable and efficient platform in order to work in remote conditions with ultrasound scan equipment.

Artificial Intelligence and Echocardiography: A Primer for Cardiac Sonographers

Artificial intelligence (AI) is emerging as a key component in diagnostic medical imaging, including echocardiography. AI with deep learning has already been used with automated view labeling, measurements, and interpretation. As the development and use of AI in echocardiography increase, potential concerns may be raised by cardiac sonographers and the profession. This report, from a sonographer's perspective, focuses on defining AI, the basics of the technology, identifying some current applications of AI, and how the use of AI may improve patient care in the future.

•

AI will have a strong role in echocardiography.

•

AI will guide image acquisition and optimization.

•

AI for image analysis may aid in interpretation.

•

AI is a tool that will not replace sonographers but will help them be more efficient.

Biomechanical modelling of probe to tissue interaction during ultrasound scanning

PURPOSE

Biomechanical simulation of anatomical deformations caused by ultrasound probe pressure is of outstanding importance for several applications, from the testing of robotic acquisition systems to multi-modal image fusion and development of ultrasound training platforms. Different approaches can be exploited for modelling the probe-tissue interaction, each achieving different trade-offs among accuracy, computation time and stability.

METHODS

We assess the performances of different strategies based on the finite element method for modelling the interaction between the rigid probe and soft tissues. Probe-tissue contact is modelled using (i) penalty forces, (ii) constraint forces, and (iii) by prescribing the displacement of the mesh surface nodes. These methods are tested in the challenging context of ultrasound scanning of the breast, an organ undergoing large nonlinear deformations during the procedure.

RESULTS

The obtained results are evaluated against those of a non-physically based method. While all methods achieve similar accuracy, performance in terms of stability and speed shows high variability, especially for those methods modelling the contacts explicitly. Overall, prescribing surface displacements is the approach with best performances, but it requires prior knowledge of the contact area and probe trajectory.

CONCLUSIONS

In this work, we present different strategies for modelling probe-tissue interaction, each able to achieve different compromises among accuracy, speed and stability. The choice of the preferred approach highly depends on the requirements of the specific clinical application. Since the presented methodologies can be applied to describe general tool-tissue interactions, this work can be seen as a reference for researchers seeking the most appropriate strategy to model anatomical deformation induced by the interaction with medical tools.

User-centred design and evaluation of a tele-operated echocardiography robot

We present the collected findings of a user-centred approach for developing a tele-operated robot for remote echocardiography examinations. During the three-year development of the robot, we involved users in all development stages of the robot, to increase the usability of the system for the doctors. For requirement compilation, we conducted a literature review, observed two traditional examinations, arranged focus groups with doctors and patients, and conducted two online surveys. During the development of the robot, we regularly involved doctors in usability tests to receive feedback from them on the user interface for the robot and on the robot’s hardware. For evaluation of the robot, we conducted two eye tracking studies. In the first study, doctors executed a traditional echocardiography examination. In the second study, the doctors conducted a remote examination with our robot. The results of the studies show that all doctors were able to successfully complete a correct ultrasonography examination with the tele-operated robot. In comparison to a traditional examination, the doctors on average only need a short amount of additional time to successfully examine a patient when using our remote echocardiography robot. The results also show that the doctors fixate considerably more often, but with shorter fixation times, on the USG screen in the traditional examination compared to the remote examination. We found further that some of the user-centred design methods we applied had to be adjusted to the clinical context and the hectic schedule of the doctors. Overall, our experience and results suggest that the usage of user-centred design methodology is well suited for developing medical robots and leads to a usable product that meets the end users’ needs.

Robotic fetal ultrasonography platform with a passive scan mechanism

PURPOSE

The shortage of obstetricians and gynecologists has intensified in developed countries. Our long-term goal is to develop a robotic prenatal care platform for automatic ultrasound (US) scanning to improve the workflow efficiency of obstetricians and gynecologists. This paper develops a hardware platform for the positioning of the US probe to obtain diagnostic US images while satisfying safety requirements of the fetus and pregnant woman.

METHOD

The proposed system includes a mechanism that maintains the contact force in a certain range and passively adjusts the US probe posture relative to the body surface. The system is designed according to clinical survey data. For proof of concept, we conducted a robotic US scan with an agar phantom and three pregnant women under the operation of a physician.

RESULTS

Experimental results show the passive US scan motion followed the phantom surface with an acceptable contact force (< 15 N). Clinical trials were safely carried out with observations of fetal body parts.

CONCLUSION

Our proposed platform acquired US images with satisfactory contact forces in the phantom study. The feasibility of the platform was demonstrated in a clinical study.

Tip Estimation Method in Phantoms for Curved Needle Using 2D Transverse Ultrasound Images

Flexible needles have been widely used in minimally invasive surgeries, especially in percutaneous interventions. Among the interventions, tip position of the curved needle is very important, since it directly affects the success of the surgeries. In this paper, we present a method to estimate the tip position of a long-curved needle by using 2D transverse ultrasound images from a robotic ultrasound system. Ultrasound is first used to detect the cross section of long-flexible needle. A new imaging approach is proposed based on the selection of numbers of pixels with a higher gray level, which can directly remove the lower gray level to highlight the needle. After that, the needle shape tracking method is proposed by combining the image processing with the Kalman filter by using 3D needle positions, which develop a robust needle tracking procedure from 1 mm to 8 mm scan intervals. Shape reconstruction is then achieved using the curve fitting method. Finally, the needle tip position is estimated based on the curve fitting result. Experimental results showed that the estimation error of tip position is less than 1 mm within 4 mm scan intervals. The advantage of the proposed method is that the shape and tip position can be estimated through scanning the needle’s cross sections at intervals along the direction of needle insertion without detecting the tip.

A practical guide to graphic communication for quality assurance, education, and patient care in echocardiography

Graphic communication (GC) is useful for continuous quality improvement (CQI), education, and patient care when in‐person discussion is not possible because of geographic and schedule constraints. In echocardiography, these constraints can be mitigated by (a) capturing screenshots and device photos or videos and sharing them by email or text message, (b) simultaneous viewing of images on digital displays, and (c) broadcasting the study real time during acquisition to other mobile or stationary devices. Screenshots are useful for CQI and education and can be acquired, annotated, and shared with minimal impact on the flow of clinical echo interpretation. Providers at different locations can employ GC for shared clinical decision making by viewing echo studies from the same server, video conferencing or accessing real‐time broadcast from a device. Which GC tool is selected is determined by its ease of use, the provider's goals and whether immediate image review is needed.

Robotic-Assisted Ultrasound for Fetal Imaging: Evolution from Single-Arm to Dual-Arm System

The development of robotic-assisted extracorporeal ultrasound systems has a long history and a number of projects have been proposed since the 1990s focusing on different technical aspects. These aim to resolve the deficiencies of on-site manual manipulation of hand-held ultrasound probes. This paper presents the recent ongoing developments of a series of bespoke robotic systems, including both single-arm and dual-arm versions, for a project known as intelligent Fetal Imaging and Diagnosis (iFIND). After a brief review of the development history of the extracorporeal ultrasound robotic system used for fetal and abdominal examinations, the specific aim of the iFIND robots, the design evolution, the implementation details of each version, and the initial clinical feedback of the iFIND robot series are presented. Based on the preliminary testing of these newly-proposed robots on 42 volunteers, the successful and reliable working of the mechatronic systems were validated. Analysis of a participant questionnaire indicates a comfortable scanning experience for the volunteers and a good acceptance rate to being scanned by the robots.

Analysis of a Customized Clutch Joint Designed for the Safety Management of an Ultrasound Robot

Robotic systems have great potential to assist ultrasound (US) examination. Currently, the safety management method to limit the force that a US robot can apply mostly relies on force sensing and software-based algorithms. This causes the concern that the potential failure of sensors, electrical systems, or software could lead to patient injuries. In this paper, we investigated a customized spring-loaded ball clutch joint designed for a newly developed US robot to passively limit the force applied. The working mechanism of the clutch was modelled and the kinematic-based analysis was performed to understand the variation of the limited force at different postures of the robot. The triggering torque of the clutch was found to be 3928 N·mm, which results in the mean limited force 22.10 ± 1.76 N at the US probe end based on potential postures. The real measurement of the implemented design indicated that the limited force could be set between 17 and 24 N at the neutral posture depending on the preload. With the maximum preload, the mean limited force was found to be 21.98 ± 0.96 N based on 30 repeated measurements. The practically measured results meet the expectation from the theoretical calculation, and the resulting small variation has indicated a good repeatability of the clutch. Based on this evidence, it is concluded that the proposed clutch meets the design aim that it can limit the force applied within a safe range while at the same time ensuring that the required force is applied at different postures.

Design and Implementation of a Bespoke Robotic Manipulator for Extra-corporeal Ultrasound

With the potential for high precision, dexterity, and repeatability, a self-tracked robotic system can be employed to assist the acquisition of real-time ultrasound. However, limited numbers of robots designed for extra-corporeal ultrasound have been successfully translated into clinical use. In this study, we aim to build a bespoke robotic manipulator for extra-corporeal ultrasound examination, which is lightweight and has a small footprint. The robot is formed by five specially shaped links and custom-made joint mechanisms for probe manipulation, to cover the necessary range of motion with redundant degrees of freedom to ensure the patient's safety. The mechanical safety is emphasized with a clutch mechanism, to limit the force applied to patients. As a result of the design, the total weight of the manipulator is less than 2 kg and the length of the manipulator is about 25 cm. The design has been implemented, and simulation, phantom, and volunteer studies have been performed, to validate the range of motion, the ability to make fine adjustments, mechanical reliability, and the safe operation of the clutch. This paper details the design and implementation of the bespoke robotic ultrasound manipulator, with the design and assembly methods illustrated. Testing results to demonstrate the design features and clinical experience of using the system are presented. It is concluded that the current proposed robotic manipulator meets the requirements as a bespoke system for extra-corporeal ultrasound examination and has great potential to be translated into clinical use.

Fully Automatic Three-Dimensional Ultrasound Imaging Based on Conventional B-Scan

Robotic ultrasound systems have turned into clinical use over the past few decades, increasing precision and quality of medical operations. In this paper, we propose a fully automatic scanning system for three-dimensional (3-D) ultrasound imaging. A depth camera was first used to obtain the depth data and color data of the tissue surface. Based on the depth image, the 3-D contour of the tissue was rendered and the scan path of ultrasound probe was automatically planned. Following the scan path, a 3-D translating device drove the probe to move on the tissue surface. Simultaneously, the B-scans and their positional information were recorded for subsequent volume reconstruction. In order to stop the scanning process when the pressure on the skin exceeded a preset threshold, two force sensors were attached to the front side of the probe for force measurement. In vitro and in vivo experiments were conducted for assessing the performance of the proposed system. Quantitative results show that the error of volume measurement was less than 1%, indicating that the system is capable of automatic ultrasound scanning and 3-D imaging. It is expected that the proposed system can be well used in clinical practices.

Development of an AVF Stenosis Assessment Tool for Hemodialysis Patients Using Robotic Ultrasound System

With the aging population and lifestyle changes, the number of hemodialysis (HD) patients increases year by year. The arteriovenous fistula (AVF) is the gold standard vascular access used to access the blood for HD treatment. Since the status of stenosis affects HD efficiency, current clinical practices usually use a Doppler ultrasound imaging system to assess the parameters of the stenosis, such as the degree of stenosis (DOS). Unfortunately, this is a very time-consuming task. Furthermore, it is difficult to stably maintain the ultrasound probe for a prolonged period to give doctors clearer or reproducible images. In this study, a robotic ultrasound system (RUS) with ultrasound sequential imaging analysis was designed to evaluate the DOS of the AVF. The sequential imaging analysis was capable of image smoothing and vessel boundary detection. It enabled clinicians to mark the thickness of the plaque for further processing. Finally, the system was used to reconstruct 3D models of fistulas and calculated the DOS for clinical assessment. We also designed a pressure sensing module attached to the ultrasound probe to prevent the probe from coming loose, vibrating, and exerting abnormal pressure on the skin. In the phantom test, the results showed that the error of the DOS that was calculated by RUS was less than 3%. The results of clinical trials obtained from ten patients show that the error between the RUS and clinicians' measurement was about 10% and had a highly linear correlation (R Square > 0.95). In addition, the reproducibility error was about 3% and could effectively save about 46% of the time during clinical examinations.

Note: A highly symmetrical piezoelectric motor with self-matching friction and large output force

We present a new stacked TunaDrive motor with two intact piezoelectric stacks being spring-clamped by a pair of parallel highly polished zirconia ceramic rods. Compared with the earlier version we presented with five stacks, it has a simpler and more compact structure and is much easier to build. Although the usage of piezoelectric material is cut by ∼35%, the motor can still provide a large output force of ∼1.92 N, which is nearly the same as that of the earlier version. The simple and symmetrical structure, high rigidity, and large output force make the new motor an ideal tool for coarse approach with nanometer precision.

Force-assisted ultrasound imaging system through dual force sensing and admittance robot control

PURPOSE

Ultrasound imaging has been a gold standard for clinical diagnoses due to its unique advantages compared to other imaging modalities including: low cost, noninvasiveness, and safeness to the human body. However, the ultrasound scanning process requires applying a large force over extended periods of time, often in uncomfortable postures in order to maintain the desired orientation. This physical requirement over sonographers' careers often leads to musculoskeletal pain and strain injuries.

METHODS

To address this problem, we propose a cooperatively controlled robotic ultrasound system to reduce the force sonographers apply. The proposed system consists of two key components: a six-axis robotic arm that holds and actuates the ultrasound probe, and a dual force sensor setup that enables cooperative control and adaptive force assistance. With the admittance force control, the robotic arm complies with the motion of the operator, while assisting with force during the scanning.

RESULTS

We validated the system through a user study involving expert sonographers and lay people and demonstrated 32-73% reduction in human applied force and 8- 18% improvement in image stability.

CONCLUSION

These results indicate that the system has the potential to not only reduce the burden on the sonographer, but also provide more stable ultrasound scanning.