1
|
Wu Z, Wang X, Cao Y, Zhang W, Xu Q. Robotic Ultrasound Scanning End-Effector with Adjustable Constant Contact Force. CYBORG AND BIONIC SYSTEMS 2025; 6:0251. [PMID: 40321899 PMCID: PMC12046132 DOI: 10.34133/cbsystems.0251] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2024] [Revised: 02/16/2025] [Accepted: 03/11/2025] [Indexed: 05/08/2025] Open
Abstract
In modern medical treatment, ultrasound scanning provides a radiation-free medical imaging method for the diagnosis of soft tissues via skin contact. However, the exerted contact force heavily relies on the skill and experience of the operator, which poses great inspection instability. This article reports on a robotic ultrasound scanning system with a constant-force end-effector. Its uniqueness is the introduction of a hybrid active-passive force control approach to maintaining a constant contact force between the ultrasound probe and the continually changing surface. In particular, the passive constant-force mechanism provides strong buffering to the force variation. The active force control system improves flexibility and provides long-stroke positioning. Experimental tests on both silicone models and human volunteers demonstrate the capability of the proposed robotic ultrasound scanning system for obtaining qualified ultrasound images with high repeatability. Moreover, the ease of operation of the robotic US scanning system is verified. This work provides a promising method to assist doctors in conducting better and cushier ultrasound scanning imaging.
Collapse
Affiliation(s)
- Zehao Wu
- Department of Electromechanical Engineering, Faculty of Science and Technology,
University of Macau, Taipa, Macau, China
| | - Xianli Wang
- Department of Electromechanical Engineering, Faculty of Science and Technology,
University of Macau, Taipa, Macau, China
| | - Yuning Cao
- Department of Electromechanical Engineering, Faculty of Science and Technology,
University of Macau, Taipa, Macau, China
| | - Weijian Zhang
- Department of Electromechanical Engineering, Faculty of Science and Technology,
University of Macau, Taipa, Macau, China
| | - Qingsong Xu
- Department of Electromechanical Engineering, Faculty of Science and Technology,
University of Macau, Taipa, Macau, China
| |
Collapse
|
2
|
Mielke T, Heinrich F, Hansen C. SensARy Substitution: Augmented Reality Techniques to Enhance Force Perception in Touchless Robot Control. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2025; 31:3235-3244. [PMID: 40085451 DOI: 10.1109/tvcg.2025.3549856] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/16/2025]
Abstract
The lack of haptic feedback in touchless human-robot interaction is critical in applications such as robotic ultrasound, where force perception is crucial to ensure image quality. Augmented reality (AR) is a promising tool to address this limitation by providing sensory substitution through visual or vibrotactile feedback. The implementation of visual force feedback requires consideration not only of feedback design but also of positioning. Therefore, we implemented two different visualization types at three different positions and investigated the effects of vibrotactile feedback on these approaches. Furthermore, we examined the effects of multimodal feedback compared to visual or vibrotactile output alone. Our results indicate that sensory substitution eases the interaction in contrast to a feedback-less baseline condition, with the presence of visual support reducing average force errors and being subjectively preferred by the participants. However, the more feedback was provided, the longer users needed to complete their tasks. Regarding visualization design, a 2D bar visualization reduced force errors compared to a 3D arrow concept. Additionally, the visualizations being displayed directly on the ultrasound screen were subjectively preferred. With findings regarding feedback modality and visualization design our work represents an important step toward sensory substitution for touchless human-robot interaction.
Collapse
|
3
|
Bao Y, Zhang J, Yang J, Xia Y, Liang D, Zhao Y, Yu H, Huang S, Guo W, Zhang J. A Novel Solid Hydrogel Sleeve Couplant for Ultrasound Imaging. JOURNAL OF ULTRASOUND IN MEDICINE : OFFICIAL JOURNAL OF THE AMERICAN INSTITUTE OF ULTRASOUND IN MEDICINE 2025. [PMID: 40162515 DOI: 10.1002/jum.16690] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/16/2024] [Revised: 02/26/2025] [Accepted: 03/15/2025] [Indexed: 04/02/2025]
Abstract
OBJECTIVES To address the challenges of ultrasound scanning on curvilinear skin surfaces at joints, this study introduces a novel Shapable and Elastic Couplants sleeve of Hydrogel (SECH) based on a "Curve-to-Smooth" strategy. The aim is to improve acoustic wave transmission, enhance image quality, and enable efficient 3D imaging of high-curvature body parts such as the hand, foot, shoulder, and neck. METHODS The SECH was fabricated using acrylamide (AAm) as the primary monomer, N,N-methylenebisacrylamide (MBAA) as the crosslinking agent, ammonium persulfate (APS) as the initiator, and N,N,N',N'-tetramethylethylenediamine (TEMED) as the accelerator. A dual-mold strategy was employed to shape the hydrogel to specific body parts. Mechanical characterization was performed using tensile tests and manual stretching/compression cycles. Ultrasound imaging was conducted on a healthy adult male volunteer using the Vevo F2 system with an L38 linear probe transducer. Cyclic scans were performed on the hand, foot, shoulder, and neck, and 3D image reconstruction was achieved using Matlab and ImageJ. RESULTS The SECH demonstrated effective mechanical properties, balancing softness and hardness to minimize air gaps and ensure stable acoustic wave transmission. Ultrasound imaging with SECH enabled high-quality 3D reconstructions of high-curvature body parts, including the hand, foot, shoulder, and neck. Multi-planar analysis of the images provided detailed diagnostic information for conditions such as hand fractures, Achilles tendon injuries, shoulder dislocations, and carotid artery stenosis. CONCLUSION The SECH represents a novel ultrasound scanning strategy that overcomes the limitations of conventional rigid probes on curvilinear surfaces. It facilitates large-area 3D imaging of high-curvature body parts, improving diagnostic accuracy and efficiency in clinical ultrasonography. This customizable hydrogel sleeve has the potential to enable convenient and automated ultrasound scanning for irregular anatomical areas.
Collapse
Affiliation(s)
- Yunlong Bao
- College of Engineering, Peking University, Beijing, People's Republic of China
| | - Jiabin Zhang
- College of Future Technology, Peking University, Beijing, People's Republic of China
| | - Jinyu Yang
- College of Engineering, Peking University, Beijing, People's Republic of China
| | - Yu Xia
- College of Engineering, Peking University, Beijing, People's Republic of China
| | - Dongdong Liang
- Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, People's Republic of China
| | - Yunlong Zhao
- Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, People's Republic of China
| | - Hao Yu
- College of Engineering, Peking University, Beijing, People's Republic of China
| | - Shuo Huang
- Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, People's Republic of China
| | - Wenyu Guo
- Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, People's Republic of China
| | - Jue Zhang
- College of Engineering, Peking University, Beijing, People's Republic of China
- Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, People's Republic of China
| |
Collapse
|
4
|
Lin XX, Li MD, Ruan SM, Ke WP, Zhang HR, Huang H, Wu SH, Cheng MQ, Tong WJ, Hu HT, He DN, Lu RF, Lin YD, Kuang M, Lu MD, Chen LD, Huang QH, Wang W. Autonomous robotic ultrasound scanning system: a key to enhancing image analysis reproducibility and observer consistency in ultrasound imaging. Front Robot AI 2025; 12:1527686. [PMID: 39975565 PMCID: PMC11835693 DOI: 10.3389/frobt.2025.1527686] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2024] [Accepted: 01/17/2025] [Indexed: 02/21/2025] Open
Abstract
Purpose This study aims to develop an autonomous robotic ultrasound scanning system (auto-RUSS) pipeline, comparing its reproducibility and observer consistency in image analysis with physicians of varying levels of expertise. Design/methodology/approach An auto-RUSS was engineered using a 7-degree-of-freedom robotic arm, with real-time regulation based on force control and ultrasound visual servoing. Two phantoms were employed for the human-machine comparative experiment, involving three groups: auto-RUSS, non-expert (4 junior physicians), and expert (4 senior physicians). This setup enabled comprehensive assessment of reproducibility in contact force, image acquisition, image measurement and AI-assisted classification. Radiological feature variability was measured using the coefficient of variation (COV), while performance and reproducibility assessments utilized mean and standard deviation (SD). Findings The auto-RUSS had the potential to reduce operator-dependent variability in ultrasound examinations, offering enhanced repeatability and consistency across multiple dimensions including probe contact force, images acquisition, image measurement, and diagnostic model performance. Originality/value In this paper, an autonomous robotic ultrasound scanning system (auto-RUSS) pipeline was proposed. Through comprehensive human-machine comparison experiments, the auto-RUSS was shown to effectively improve the reproducibility of ultrasound images and minimize human-induced variability.
Collapse
Affiliation(s)
- Xin-Xin Lin
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| | - Ming-De Li
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| | - Si-Min Ruan
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| | - Wei-Ping Ke
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| | - Hao-Ruo Zhang
- College of Electronic Information, Guangxi Minzu University, Nanning, China
| | - Hui Huang
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| | - Shao-Hong Wu
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| | - Mei-Qing Cheng
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| | - Wen-Juan Tong
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| | - Hang-Tong Hu
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| | - Dan-Ni He
- Department of Medical Ultrasonics, The Seventh Affiliated Hospital of Sun Yat-Sen University, Shenzhen, China
| | - Rui-Fang Lu
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| | - Ya-Dan Lin
- Department of Medical Ultrasound, The First Affiliated Hospital of Guangxi Medical University, Nanning, China
| | - Ming Kuang
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
- Department of Hepatobiliary Surgery, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| | - Ming-De Lu
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
- Department of Hepatobiliary Surgery, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| | - Li-Da Chen
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| | - Qing-Hua Huang
- College of Electronic Information, Guangxi Minzu University, Nanning, China
- School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University, Xi’an, Shaanxi, China
| | - Wei Wang
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| |
Collapse
|
5
|
Fougeron N, Oddes Z, Ashkenazi A, Solav D. Identification of constitutive materials of bi-layer soft tissues from multimodal indentations. J Mech Behav Biomed Mater 2024; 155:106572. [PMID: 38754153 DOI: 10.1016/j.jmbbm.2024.106572] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2024] [Revised: 04/19/2024] [Accepted: 05/08/2024] [Indexed: 05/18/2024]
Abstract
The personalisation of finite element models is an important problem in the biomechanical fields where subject-specific analyses are fundamental, particularly in studying soft tissue mechanics. The personalisation includes the choice of the constitutive law of the model's material, as well as the choice of the material parameters. In vivo identification of the material properties of soft tissues is challenging considering the complex behaviour of soft tissues that are, among other things, non-linear hyperelastic and heterogeneous. Hybrid experimental-numerical methods combining in vivo indentations and inverse finite element analyses are common to identify these material parameters. Yet, the uniqueness and the uncertainty of the multi-material hyperelastic model have not been evaluated. This study presents a sensitivity analysis performed on synthetic indentation data to investigate the identification uncertainties of the material parameters in a bi-material thigh phantom. Synthetic numerical data, used to replace experimental measurements, considered several measurement modalities: indenter force and displacement, stereo-camera 3D digital image correlation of the indented surface, and ultrasound B-mode images. A finite element model of the indentation was designed with either Ogden-Moerman or Mooney-Rivlin constitutive laws for both materials. The parameters' identifiability (i.e. the possibility of converging to a unique parameter set within an acceptable margin of error) was assessed with various cost functions formulated using the different synthetic data sets. The results underline the need for multiple experimental modalities to reduce the uncertainty of the identified parameters. Additionally, the experimental error can impede the identification of a unique parameter set, and the cost function depends on the constitutive law. This study highlights the need for sensitivity analyses before the design of the experimental protocol. Such studies can also be used to define the acceptable range of errors in the experimental measurement. Eventually, the impact of the evaluated uncertainty of the identified parameters should be further investigated according to the purpose of the finite element modelling.
Collapse
Affiliation(s)
- Nolwenn Fougeron
- Faculty of Mechanical Engineering, Technion Institute of Technology, Haifa, Israel.
| | - Zohar Oddes
- Faculty of Mechanical Engineering, Technion Institute of Technology, Haifa, Israel
| | - Amit Ashkenazi
- Faculty of Mechanical Engineering, Technion Institute of Technology, Haifa, Israel
| | - Dana Solav
- Faculty of Mechanical Engineering, Technion Institute of Technology, Haifa, Israel
| |
Collapse
|
6
|
Sahrmann AS, Handsfield GG, Gizzi L, Gerlach J, Verl A, Besier TF, Rohrle O. A System for Reproducible 3D Ultrasound Measurements of Skeletal Muscles. IEEE Trans Biomed Eng 2024; 71:2022-2032. [PMID: 38285583 DOI: 10.1109/tbme.2024.3359854] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/31/2024]
Abstract
In 3D freehand ultrasound imaging, operator dependent variations in applied forces and movements can lead to errors in the reconstructed images. In this paper, we introduce an automated 3D ultrasound system, which enables acquisitions with controlled movement trajectories by using motors, which electrically move the probe. Due to integrated encoders there is no need of position sensors. An included force control mechanism ensures a constant contact force to the skin. We conducted 8 trials with the automated 3D ultrasound system on 2 different phantoms with 3 force settings and 10 trials on a human tibialis anterior muscle with 2 force settings. For comparison, we also conducted 8 freehand 3D ultrasound scans from 2 operators (4 force settings) on one phantom and 10 with one operator on the tibialis anterior muscle. Both freehand and automated trials showed small errors in volume and length computations of the reconstructions, however the freehand trials showed larger standard deviations. We also computed the thickness of the phantom and the tibialis anterior muscle. We found significant differences in force settings for the operators and higher coefficients of variation for the freehand trials. Overall, the automated 3D ultrasound system shows a high accuracy in reconstruction. Due to the smaller coefficients of variation, the automated 3D ultrasound system enables more reproducible ultrasound examinations than the freehand scanning. Therefore, the automated 3D ultrasound system is a reliable tool for 3D investigations of skeletal muscle.
Collapse
|
7
|
Ochitwa Z, Fotouhi R, Adams SJ, Noguera Cundar AP, Obaid H. MSK-TIM: A Telerobotic Ultrasound System for Assessing the Musculoskeletal System. SENSORS (BASEL, SWITZERLAND) 2024; 24:2368. [PMID: 38610578 PMCID: PMC11013981 DOI: 10.3390/s24072368] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Revised: 03/21/2024] [Accepted: 04/04/2024] [Indexed: 04/14/2024]
Abstract
The aim of this paper is to investigate technological advancements made to a robotic tele-ultrasound system for musculoskeletal imaging, the MSK-TIM (Musculoskeletal Telerobotic Imaging Machine). The hardware was enhanced with a force feedback sensor and a new controller was introduced. Software improvements were developed which allowed the operator to access ultrasound functions such as focus, depth, gain, zoom, color, and power Doppler controls. The device was equipped with Wi-Fi network capability which allowed the master and slave stations to be positioned in different locations. A trial assessing the system to scan the wrist was conducted with twelve participants, for a total of twenty-four arms. Both the participants and radiologist reported their experience. The images obtained were determined to be of satisfactory quality for diagnosis. The system improvements resulted in a better user and patient experience for the radiologist and participants. Latency with the VPN configuration was similar to the WLAN in our experiments. This research explores several technologies in medical telerobotics and provides insight into how they should be used in future. This study provides evidence to support larger-scale trials of the MSK-TIM for musculoskeletal imaging.
Collapse
Affiliation(s)
- Zachary Ochitwa
- Department of Mechanical Engineering, University of Saskatchewan, Saskatoon, SK S7N 5A9, Canada; (Z.O.); (A.P.N.C.)
| | - Reza Fotouhi
- Department of Mechanical Engineering, University of Saskatchewan, Saskatoon, SK S7N 5A9, Canada; (Z.O.); (A.P.N.C.)
| | - Scott J. Adams
- Department of Medical Imaging, University of Saskatchewan, Saskatoon, SK S7N 0W8, Canada; (S.J.A.); (H.O.)
| | - Adriana Paola Noguera Cundar
- Department of Mechanical Engineering, University of Saskatchewan, Saskatoon, SK S7N 5A9, Canada; (Z.O.); (A.P.N.C.)
| | - Haron Obaid
- Department of Medical Imaging, University of Saskatchewan, Saskatoon, SK S7N 0W8, Canada; (S.J.A.); (H.O.)
| |
Collapse
|
8
|
Rosen DP, Nayak R, Wang Y, Gendin D, Larson NB, Fazzio RT, Oberai AA, Hall TJ, Barbone PE, Alizad A, Fatemi M. A Force-Matched Approach to Large-Strain Nonlinearity in Elasticity Imaging for Breast Lesion Characterization. IEEE Trans Biomed Eng 2024; 71:367-374. [PMID: 37590110 PMCID: PMC10843664 DOI: 10.1109/tbme.2023.3305986] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/19/2023]
Abstract
OBJECTIVE Ultrasound elasticity imaging is a class of ultrasound techniques with applications that include the detection of malignancy in breast lesions. Although elasticity imaging traditionally assumes linear elasticity, the large strain elastic response of soft tissue is known to be nonlinear. This study evaluates the nonlinear response of breast lesions for the characterization of malignancy using force measurement and force-controlled compression during ultrasound imaging. METHODS 54 patients were recruited for this study. A custom force-instrumented compression device was used to apply a controlled force during ultrasound imaging. Motion tracking derived strain was averaged over lesion or background ROIs and matched with compression force. The resulting force-matched strain was used for subsequent analysis and curve fitting. RESULTS Greater median differences between malignant and benign lesions were observed at higher compressional forces (p-value < 0.05 for compressional forces of 2-6N). Of three candidate functions, a power law function produced the best fit to the force-matched strain. A statistically significant difference in the scaling parameter of the power function between malignant and benign lesions was observed (p-value = 0.025). CONCLUSIONS We observed a greater separation in average lesion strain between malignant and benign lesions at large compression forces and demonstrated the characterization of this nonlinear effect using a power law model. Using this model, we were able to differentiate between malignant and benign breast lesions. SIGNIFICANCE With further development, the proposed method to utilize the nonlinear elastic response of breast tissue has the potential for improving non-invasive lesion characterization for potential malignancy.
Collapse
|
9
|
Jiang Z, Zhou Y, Cao D, Navab N. DefCor-Net: Physics-aware ultrasound deformation correction. Med Image Anal 2023; 90:102923. [PMID: 37688982 DOI: 10.1016/j.media.2023.102923] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2022] [Revised: 05/22/2023] [Accepted: 08/01/2023] [Indexed: 09/11/2023]
Abstract
The recovery of morphologically accurate anatomical images from deformed ones is challenging in ultrasound (US) image acquisition, but crucial to accurate and consistent diagnosis, particularly in the emerging field of computer-assisted diagnosis. This article presents a novel physics-aware deformation correction approach based on a coarse-to-fine, multi-scale deep neural network (DefCor-Net). To achieve pixel-wise performance, DefCor-Net incorporates biomedical knowledge by estimating pixel-wise stiffness online using a U-shaped feature extractor. The deformation field is then computed using polynomial regression by integrating the measured force applied by the US probe. Based on real-time estimation of pixel-by-pixel tissue properties, the learning-based approach enables the potential for anatomy-aware deformation correction. To demonstrate the effectiveness of the proposed DefCor-Net, images recorded at multiple locations on forearms and upper arms of six volunteers are used to train and validate DefCor-Net. The results demonstrate that DefCor-Net can significantly improve the accuracy of deformation correction to recover the original geometry (Dice Coefficient: from 14.3±20.9 to 82.6±12.1 when the force is 6N). Code:https://github.com/KarolineZhy/DefCorNet.
Collapse
Affiliation(s)
- Zhongliang Jiang
- Computer Aided Medical Procedures, Technical University of Munich, Munich, Germany.
| | - Yue Zhou
- Computer Aided Medical Procedures, Technical University of Munich, Munich, Germany
| | | | - Nassir Navab
- Computer Aided Medical Procedures, Technical University of Munich, Munich, Germany; Computer Aided Medical Procedures, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
10
|
Jiang Z, Salcudean SE, Navab N. Robotic ultrasound imaging: State-of-the-art and future perspectives. Med Image Anal 2023; 89:102878. [PMID: 37541100 DOI: 10.1016/j.media.2023.102878] [Citation(s) in RCA: 21] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Revised: 04/27/2023] [Accepted: 06/22/2023] [Indexed: 08/06/2023]
Abstract
Ultrasound (US) is one of the most widely used modalities for clinical intervention and diagnosis due to the merits of providing non-invasive, radiation-free, and real-time images. However, free-hand US examinations are highly operator-dependent. Robotic US System (RUSS) aims at overcoming this shortcoming by offering reproducibility, while also aiming at improving dexterity, and intelligent anatomy and disease-aware imaging. In addition to enhancing diagnostic outcomes, RUSS also holds the potential to provide medical interventions for populations suffering from the shortage of experienced sonographers. In this paper, we categorize RUSS as teleoperated or autonomous. Regarding teleoperated RUSS, we summarize their technical developments, and clinical evaluations, respectively. This survey then focuses on the review of recent work on autonomous robotic US imaging. We demonstrate that machine learning and artificial intelligence present the key techniques, which enable intelligent patient and process-specific, motion and deformation-aware robotic image acquisition. We also show that the research on artificial intelligence for autonomous RUSS has directed the research community toward understanding and modeling expert sonographers' semantic reasoning and action. Here, we call this process, the recovery of the "language of sonography". This side result of research on autonomous robotic US acquisitions could be considered as valuable and essential as the progress made in the robotic US examination itself. This article will provide both engineers and clinicians with a comprehensive understanding of RUSS by surveying underlying techniques. Additionally, we present the challenges that the scientific community needs to face in the coming years in order to achieve its ultimate goal of developing intelligent robotic sonographer colleagues. These colleagues are expected to be capable of collaborating with human sonographers in dynamic environments to enhance both diagnostic and intraoperative imaging.
Collapse
Affiliation(s)
- Zhongliang Jiang
- Computer Aided Medical Procedures, Technical University of Munich, Munich, Germany.
| | - Septimiu E Salcudean
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC V6T 1Z4, Canada
| | - Nassir Navab
- Computer Aided Medical Procedures, Technical University of Munich, Munich, Germany; Computer Aided Medical Procedures, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
11
|
Tan J, Li B, Leng Y, Li Y, Peng J, Wu J, Luo B, Chen X, Rong Y, Fu C. Fully Automatic Dual-Probe Lung Ultrasound Scanning Robot for Screening Triage. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2023; 70:975-988. [PMID: 36191095 DOI: 10.1109/tuffc.2022.3211532] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Two-dimensional lung ultrasound (LUS) has widely emerged as a rapid and noninvasive imaging tool for the detection and diagnosis of coronavirus disease 2019 (COVID-19). However, image differences will be magnified due to changes in ultrasound (US) imaging experience, such as US probe attitude control and force control, which will directly affect the diagnosis results. In addition, the risk of virus transmission between sonographer and patients is increased due to frequent physical contact. In this study, a fully automatic dual-probe US scanning robot for the acquisition of LUS images is proposed and developed. Furthermore, the trajectory was optimized based on the velocity look-ahead strategy, the stability of contact force of the system and the scanning efficiency were improved by 24.13% and 29.46%, respectively. Also, the control ability of the contact force of robotic automatic scanning was 34.14 times higher than that of traditional manual scanning, which significantly improves the smoothness of scanning. Importantly, there was no significant difference in image quality obtained by robotic automatic scanning and manual scanning. Furthermore, the scanning time for a single person is less than 4 min, which greatly improves the efficiency of screening triage of group COVID-19 diagnosis and suspected patients and reduces the risk of virus exposure and spread.
Collapse
|
12
|
Sai H, Xu Z, Xia C, Wang L, Zhang J. Lightweight Force-Controlled Device for Freehand Ultrasound Acquisition. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2023; 70:944-960. [PMID: 37028093 DOI: 10.1109/tuffc.2023.3252015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
This study investigates a force-controlled auxiliary device for freehand ultrasound (US) examinations. The designed device allows sonographers to maintain a steady target pressure on the US probe, thereby improving the US image quality and reproducibility. The use of a screw motor to power the device and a Raspberry Pi as the system controller results in a lightweight and portable device, while a screen enhances user-interactivity. Using gravity compensation, error compensation, an adaptive proportional-integral-derivative algorithm, and low-pass signal filtering, the designed device provides highly accurate force control. Several experiments using the developed device, including clinical trials relating to the jugular and superficial femoral veins, validate its utility in ensuring the desired pressure in response to varying environments and prolonged US examinations, enabling low or high pressures to be maintained and lowering the threshold of clinical experience. Moreover, the experimental results show that the designed device effectively relieves the stress on the sonographer's hand joints during US examinations and enables rapid assessment of the tissue elasticity characteristics. With automatic pressure tracking between probe and patient, the proposed device offers potentially significant benefits for the reproducibility and stability of US images and the health of sonographers.
Collapse
|
13
|
Sai H, Wang L, Zhang J, Xia C, Xu Z. Portable Device to Assist With Force Control in Ultrasound Acquisition. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2023; 70:930-943. [PMID: 35675230 DOI: 10.1109/tuffc.2022.3181287] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
This study presents a portable device that ensures precise contact force between a subject and a probe to improve the stability and reproducibility of ultrasound (US) acquisition. The mechanical portion of the device includes a servo motor, gears, and a ball screw linear actuator; two photoelectric switches are used to limit the stroke. A combined force and position control system is developed, and a pressure threshold is introduced to reduce the chattering of the system so that it can be applied to US examinations of tissues of different stiffness levels. Force control experiments were conducted on the device, and the results showed that the device can overcome the chattering of a physician's hand and movement caused by a subject's respiration. Additionally, the stability of the US acquisition was substantially improved. Based on clinical trials on humans, this device was observed to improve the consistency of ultrasonic results and the repeatability of images, and it assisted sonographers with maintaining suitable contact force and improving imaging quality. The device can either be handheld by a physician or easily integrated with a manipulator as an autonomous robotic US acquisition device, thereby validating its potential for US applications.
Collapse
|
14
|
Lai DKH, Cheng ESW, Mao YJ, Zheng Y, Yao KY, Ni M, Zhang YQ, Wong DWC, Cheung JCW. Sonoelastography for Testicular Tumor Identification: A Systematic Review and Meta-Analysis of Diagnostic Test Accuracy. Cancers (Basel) 2023; 15:3770. [PMID: 37568585 PMCID: PMC10417060 DOI: 10.3390/cancers15153770] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2023] [Revised: 07/18/2023] [Accepted: 07/21/2023] [Indexed: 08/13/2023] Open
Abstract
The objective of this review was to summarize the applications of sonoelastography in testicular tumor identification and inquire about their test performances. Two authors independently searched English journal articles and full conference papers from CINAHL, Embase, IEEE Xplore®, PubMed, Scopus, and Web of Science from inception and organized them into a PIRO (patient, index test, reference test, outcome) framework. Eleven studies (n = 11) were eligible for data synthesis, nine of which (n = 9) utilized strain elastography and two (n = 2) employed shear-wave elastography. Meta-analyses were performed on the distinction between neoplasm (tumor) and non-neoplasm (non-tumor) from four study arms and between malignancy and benignity from seven study arms. The pooled sensitivity of classifying malignancy and benignity was 86.0% (95%CI, 79.7% to 90.6%). There was substantial heterogeneity in the classification of neoplasm and non-neoplasm and in the specificity of classifying malignancy and benignity, which could not be addressed by the subgroup analysis of sonoelastography techniques. Heterogeneity might be associated with the high risk of bias and applicability concern, including a wide spectrum of testicular pathologies and verification bias in the reference tests. Key technical obstacles in the index test were manual compression in strain elastography, qualitative observation of non-standardized color codes, and locating the Regions of Interest (ROI), in addition to decisions in feature extractions. Future research may focus on multiparametric sonoelastography using deep learning models and ensemble learning. A decision model on the benefits-risks of surgical exploration (reference test) could also be developed to direct the test-and-treat strategy for testicular tumors.
Collapse
Affiliation(s)
- Derek Ka-Hei Lai
- Department of Biomedical Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong, China
| | - Ethan Shiu-Wang Cheng
- Department of Electronic and Information Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong, China
| | - Ye-Jiao Mao
- Department of Biomedical Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong, China
| | - Yi Zheng
- Department of Biomedical Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong, China
| | - Ke-Yu Yao
- Department of Materials, Imperial College, London SW7 2AZ, UK
| | - Ming Ni
- Department of Orthopaedics, Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai 200240, China
- Laboratory of Prevention and Treatment of Bone and Joint Diseases, Shanghai Institute of Traumatology and Orthopaedics, Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Ying-Qi Zhang
- Department of Orthopaedics, Tongji Hospital, School of Medicine, Tongji University, Shanghai 200065, China
| | - Duo Wai-Chi Wong
- Department of Biomedical Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong, China
| | - James Chung-Wai Cheung
- Department of Biomedical Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong, China
- Research Institute of Smart Ageing, The Hong Kong Polytechnic University, Hong Kong, China
| |
Collapse
|
15
|
Tsumura R, Koseki Y, Nitta N, Yoshinaka K. Towards fully automated robotic platform for remote auscultation. Int J Med Robot 2023; 19:e2461. [PMID: 36097703 DOI: 10.1002/rcs.2461] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2021] [Revised: 02/22/2022] [Accepted: 02/24/2022] [Indexed: 01/05/2023]
Abstract
BACKGROUND Since most developed countries are facing an increase in the number of patients per healthcare worker due to a declining birth rate and an ageing population, relatively simple and safe diagnosis tasks may need to be performed using robotics and automation technologies, without specialists and hospitals. This study presents an automated robotic platform for remote auscultation, which is a highly cost-effective screening tool for detecting abnormal clinical signs. METHOD The developed robotic platform is composed of a 6-degree-of-freedom cooperative robotic arm, LiDAR camera, and a spring-based mechanism holding an electric stethoscope. The platform enables autonomous stethoscope positioning based on external body information acquired using the LiDAR camera-based multi-way registration; the platform also ensures safe and flexible contact, maintaining the contact force within a certain range through the passive-actuated mechanism. RESULTS Our preliminary results confirm that the robotic platform enables estimation of the landing positions required for cardiac examinations based on the depth and landmark information of the body surface. It also handles the stethoscope while maintaining the contact force without relying on the push-in displacement by the robotic arm. CONCLUSION The developed robotic platform enables the estimation of the landing positions and handling the stethoscope while maintaining the contact force, which promises the potential of automatic remote auscultation.
Collapse
Affiliation(s)
- Ryosuke Tsumura
- Health and Medical Research Institute, National Institute of Advanced Industrial Science and Technology, Tsukuba, Ibaraki, Japan
| | - Yoshihiko Koseki
- Health and Medical Research Institute, National Institute of Advanced Industrial Science and Technology, Tsukuba, Ibaraki, Japan
| | - Naotaka Nitta
- Health and Medical Research Institute, National Institute of Advanced Industrial Science and Technology, Tsukuba, Ibaraki, Japan
| | - Kiyoshi Yoshinaka
- Health and Medical Research Institute, National Institute of Advanced Industrial Science and Technology, Tsukuba, Ibaraki, Japan
| |
Collapse
|
16
|
Li Q, Zhang F, Xi Q, Jiao Z, Ni X. Nondeformed Ultrasound Image Production Method for Ultrasound-Guided Radiotherapy. Technol Cancer Res Treat 2023; 22:15330338231194546. [PMID: 37700675 PMCID: PMC10501062 DOI: 10.1177/15330338231194546] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2023] [Revised: 07/23/2023] [Accepted: 07/26/2023] [Indexed: 09/14/2023] Open
Abstract
Purpose: During ultrasound (US)-guided radiotherapy, the tissue is deformed by probe pressure, and the US image is limited by changes in tissue and organ position and geometry when the US image is aligned with computed tomography (CT) image, leading to poor alignment. Accordingly, a pixel displacement-based nondeformed US image production method is proposed. Methods: The correction of US image deformation is achieved by calculating the pixel displacement of an image. The positioning CT image (CTstd) is used as the gold standard. The deformed US image (USdef) is inputted into the Harris algorithm to extract corner points for selecting feature points, and the displacement of adjacent pixels of feature points in the US video stream is calculated using the Lucas-Kanade optical flow algorithm. The moving least squares algorithm is used to correct USdef globally and locally in accordance with image pixel displacement to generate a nondeformed US image (USrev). In addition, USdef and USrev were separately aligned with CTstd to evaluate the improvement of alignment accuracy through deformation correction. Results: In the phantom experiment, the overall and local average correction errors of the US image under the optimal probe pressure were 1.0944 and 0.7388 mm, respectively, and the registration accuracy of USdef and USrev with CTstd was 0.6764 and 0.9016, respectively. During the volunteer experiment, the correction error of all 12 patients' data ranged from -1.7525 to 1.5685 mm, with a mean absolute error of 0.8612 mm. The improvement range of US and CT registration accuracy, before and after image deformation correction in the 12 patients evaluated by a normalized correlation coefficient, was 0.1232 to 0.2476. Conclusion: The pixel displacement-based deformation correction method can solve the limitation imposed by image deformation on image alignment in US-guided radiotherapy. Compared with USdef, the alignment results of USrev with CT were better.
Collapse
Affiliation(s)
- Qixuan Li
- School of Microelectronics and Control Engineering, Changzhou University, Changzhou, China
- Department of Radiotherapy, The Affiliated Changzhou No.2 People's Hospital of Nanjing Medical University, Changzhou, China
- Jiangsu Province Engineering Research Center of Medical Physics, Changzhou, China
- Center for Medical Physics, Nanjing Medical University, Changzhou, China
| | - Fan Zhang
- School of Microelectronics and Control Engineering, Changzhou University, Changzhou, China
- Department of Radiotherapy, The Affiliated Changzhou No.2 People's Hospital of Nanjing Medical University, Changzhou, China
- Jiangsu Province Engineering Research Center of Medical Physics, Changzhou, China
- Center for Medical Physics, Nanjing Medical University, Changzhou, China
| | - Qianyi Xi
- School of Microelectronics and Control Engineering, Changzhou University, Changzhou, China
- Department of Radiotherapy, The Affiliated Changzhou No.2 People's Hospital of Nanjing Medical University, Changzhou, China
- Jiangsu Province Engineering Research Center of Medical Physics, Changzhou, China
- Center for Medical Physics, Nanjing Medical University, Changzhou, China
| | - Zhuqing Jiao
- School of Microelectronics and Control Engineering, Changzhou University, Changzhou, China
- School of Computer and Artificial Intelligence, Changzhou University, Changzhou, China
| | - Xinye Ni
- Department of Radiotherapy, The Affiliated Changzhou No.2 People's Hospital of Nanjing Medical University, Changzhou, China
- Jiangsu Province Engineering Research Center of Medical Physics, Changzhou, China
- Center for Medical Physics, Nanjing Medical University, Changzhou, China
| |
Collapse
|
17
|
Jiang Z, Gao Y, Xie L, Navab N. Towards Autonomous Atlas-Based Ultrasound Acquisitions in Presence of Articulated Motion. IEEE Robot Autom Lett 2022. [DOI: 10.1109/lra.2022.3180440] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Affiliation(s)
- Zhongliang Jiang
- Chair for Computer Aided Medical Procedures and Augmented Reality (CAMP), Technical University of Munich, Garching, Germany
| | - Yuan Gao
- Chair for Computer Aided Medical Procedures and Augmented Reality (CAMP), Technical University of Munich, Garching, Germany
| | - Le Xie
- Institute of Forming Technology and Equipment and Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China
| | - Nassir Navab
- Chair for Computer Aided Medical Procedures and Augmented Reality (CAMP), Technical University of Munich, Garching, Germany
| |
Collapse
|
18
|
Remote Ultrasound Scan Procedures with Medical Robots: Towards New Perspectives between Medicine and Engineering. Appl Bionics Biomech 2022; 2022:1072642. [PMID: 35154375 PMCID: PMC8832154 DOI: 10.1155/2022/1072642] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2021] [Revised: 12/10/2021] [Accepted: 02/01/2022] [Indexed: 12/13/2022] Open
Abstract
Background This review explores state-of-the-art teleoperated robots for medical ultrasound scan procedures, providing a comprehensive look including the recent trends arising from the COVID-19 pandemic. Methods Physicians' experience is included to indicate the importance of their role in the design of improved medical robots. From this perspective, novel classes of equipment for remote diagnostics based on medical robotics are discussed in terms of innovative engineering technologies. Results Relevant literature is reviewed under the system engineering point of view, organizing the discussion on the basis of the main technological focus of each contribution. Conclusions This contribution is aimed at stimulating new research to obtain faster results on teleoperated robotics for ultrasound diagnostics in response to the high demand raised by the ongoing pandemic.
Collapse
|
19
|
Jiang Z, Zhou Y, Bi Y, Zhou M, Wendler T, Navab N. Deformation-Aware Robotic 3D Ultrasound. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3099080] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
20
|
Tsumura R, Hardin JW, Bimbraw K, Grossestreuer AV, Odusanya OS, Zheng Y, Hill JC, Hoffmann B, Soboyejo W, Zhang HK. Tele-Operative Low-Cost Robotic Lung Ultrasound Scanning Platform for Triage of COVID-19 Patients. IEEE Robot Autom Lett 2021; 6:4664-4671. [PMID: 34532570 PMCID: PMC8442628 DOI: 10.1109/lra.2021.3068702] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Accepted: 02/23/2021] [Indexed: 12/21/2022]
Abstract
Novel severe acute respiratory syndrome coronavirus 2 (COVID-19) has become a pandemic of epic proportions, and global response to prepare health systems worldwide is of utmost importance. 2-dimensional (2D) lung ultrasound (LUS) has emerged as a rapid, noninvasive imaging tool for diagnosing COVID-19 infected patients. Concerns surrounding LUS include the disparity of infected patients and healthcare providers, and importantly, the requirement for substantial physical contact between the patient and operator, increasing the risk of transmission. New variants of COVID-19 will continue to emerge; therefore, mitigation of the virus's spread is of paramount importance. A tele-operative robotic ultrasound platform capable of performing LUS in COVID-19 infected patients may be of significant benefit, especially in low- and middle-income countries. The authors address the issues mentioned above surrounding the use of LUS in COVID-19 infected patients and the potential for extension of this technology in a resource-limited environment. Additionally, first-time application, feasibility, and safety were validated in healthy subjects. Preliminary results demonstrate that our platform allows for the successful acquisition and application of robotic LUS in humans.
Collapse
Affiliation(s)
- Ryosuke Tsumura
- Department of Biomedical EngineeringWorcester Polytechnic InstituteWorcesterMA01609USA
| | - John W. Hardin
- Department of Emergency MedicineBeth Israel Deaconess Medical CenterBostonMA02215USA
| | - Keshav Bimbraw
- Department of Biomedical EngineeringWorcester Polytechnic InstituteWorcesterMA01609USA
| | - Anne V. Grossestreuer
- Department of Emergency MedicineBeth Israel Deaconess Medical CenterBostonMA02215USA
| | | | - Yihao Zheng
- Department of Mechanical EngineeringWorcester Polytechnic InstituteWorcesterMA01609USA
| | - Jeffrey C. Hill
- Department of Diagnostic Medical Sonography, School of Medical Imaging and TherapeuticsMCPHS UniversityWorcesterMA01608USA
| | - Beatrice Hoffmann
- Department of Emergency MedicineBeth Israel Deaconess Medical CenterBostonMA02215USA
| | - Winston Soboyejo
- Department of Mechanical EngineeringWorcester Polytechnic InstituteWorcesterMA01609USA
| | - Haichong K. Zhang
- Department of Biomedical EngineeringWorcester Polytechnic InstituteWorcesterMA01609USA
| |
Collapse
|
21
|
Chen S, Li Z, Lin Y, Wang F, Cao Q. Automatic ultrasound scanning robotic system with optical waveguide-based force measurement. Int J Comput Assist Radiol Surg 2021; 16:1015-1025. [PMID: 33939078 DOI: 10.1007/s11548-021-02385-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/25/2020] [Accepted: 04/19/2021] [Indexed: 12/18/2022]
Abstract
PURPOSE The three-dimensional (3D) ultrasound (US) imaging realized by continuous scanning of a region is of great value for medical diagnosis and robot-assisted needle insertion. During scanning, the contact force and posture between the probe and skin of the patient are crucial factors that determine the quality of US imaging. We propose a robotic system for automatic scanning of curved surfaces with a stable contact force and vertical contact posture (the probe is parallel to the normal of the surface at the contact point). METHODS A 6-DOF robotic arm is used to hold and drive a two-dimensional (2D) US probe to complete automatic scanning. Further, a path-planning strategy is proposed to generate the scan path covering the selected area automatically. We also developed a novel force-measuring device based on optical waveguides to measure the distributed contact force and contact posture. Based on the measured force and posture, the robotic arm automatically adjusts the position and orientation of the probe and maintains a stable contact force and vertical contact posture at each scan point. RESULTS The novel force-measuring device is easy to fabricate, integrates with the probe and has the capacity of measuring the force distributed on the contact surface and estimating the contact posture. The experimental results of automatic scanning of a US phantom and parts of the human body demonstrate that the proposed system performs well in automatically scanning curved surfaces, maintaining a stable contact force and vertical contact posture and producing a good quality 3D US volume. CONCLUSION An automatic US scanning robotic system with an optical waveguide-based force-measuring device was developed and tested successfully. Experimental results demonstrated the feasibility of the proposed system to scan the human body.
Collapse
Affiliation(s)
- Shihang Chen
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Zhaojun Li
- Shanghai General Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Yanping Lin
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China. .,Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China.
| | - Fang Wang
- Shanghai General Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Qixin Cao
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China.,Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| |
Collapse
|
22
|
Chen L, Zeng G, Guo D, Liu J, Zhang X, Lin S, Zhang K. Soft elastic hydrogel couplants for ultrasonography. MATERIALS SCIENCE & ENGINEERING. C, MATERIALS FOR BIOLOGICAL APPLICATIONS 2020; 119:111609. [PMID: 33321653 DOI: 10.1016/j.msec.2020.111609] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/13/2020] [Revised: 09/11/2020] [Accepted: 10/05/2020] [Indexed: 12/19/2022]
Abstract
Couplants play significant roles in ultrasonography. To ensure imaging quality, it is critical to maintain conformal contact of the couplant with both the skin surface and the ultrasound probe in clinical applications. In addition, either the probe or the couplant should not deform the skin surface significantly, which will result in an overestimated modulus of the tissue for elastography imaging. However, existing liquid gel couplants cannot bear external compressive force, while existing solid gel couplants cannot maintain a conformal contact with skin surface. Especially, the nonconformal contacts and deformation become more severe on body parts of locally high curvatures such as skin tumors, fingers, and elbows. Here we report a bilayer design of couplant for ultrasonography, composing of a stiff layer and a compliant layer of hydrogels. The bilayer hydrogel pad enables it to bear external compression, allowing the probe to move smoothly, conforming high curvature parts and releasing stress concentration. Our clinical experiments further show high quality imaging of thyroid nodules, skin tumors in elbows and fingers using the bilayer hydrogel pad, which represents a promising alternative for a range of applications in ultrasonic diagnosis.
Collapse
Affiliation(s)
- Luzeng Chen
- Department of Ultrasonography, Peking University First Hospital, Beijing 100081, China
| | - Guangzhi Zeng
- School of Aerospace Engineering, Beijing Institute of Technology, Beijing 100081, China
| | - Dengke Guo
- School of Aerospace Engineering, Beijing Institute of Technology, Beijing 100081, China
| | - Jinghua Liu
- Department of Ultrasonography, Peking University First Hospital, Beijing 100081, China
| | - Xiang Zhang
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
| | - Shaoting Lin
- Soft Active Materials Laboratory, Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
| | - Kai Zhang
- School of Aerospace Engineering, Beijing Institute of Technology, Beijing 100081, China.
| |
Collapse
|
23
|
Skerl K, Eichhorn B, Poltorjanoks R, Cochran S, Evans A. Introduction of a Measurement Setup to Monitor the Pressure Applied During Handheld Ultrasound Elastography. ULTRASOUND IN MEDICINE & BIOLOGY 2020; 46:2556-2559. [PMID: 32553692 DOI: 10.1016/j.ultrasmedbio.2020.04.017] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/25/2019] [Revised: 04/09/2020] [Accepted: 04/16/2020] [Indexed: 06/11/2023]
Abstract
Shear-wave elastography may produce misleadingly high values if too much pressure is applied during the imaging process. However, in clinical routine there is presently no way to monitor the pressure applied during the measurements. In this work we introduce a novel measurement setup which can directly be attached to an ultrasonic imaging transducer and allows observation of the applied pressure in real time. The setup supports free-hand imaging according to the clinical standard. We tested the setup by carrying out shear-wave elastography under varying pressures on ex vivo animal tissue. The values increased with pressure, as was expected. Thus, the setup is a possible solution for measuring applied pressure in real time.
Collapse
Affiliation(s)
- Katrin Skerl
- University of Dundee, Dundee, Scotland, UK; Furtwangen University, Villingen-Schwenningen, Germany.
| | | | | | | | | |
Collapse
|
24
|
Sarcopenia Detection System Using RGB-D Camera and Ultrasound Probe: System Development and Preclinical In-Vitro Test. SENSORS 2020; 20:s20164447. [PMID: 32784914 PMCID: PMC7472485 DOI: 10.3390/s20164447] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/23/2020] [Revised: 08/05/2020] [Accepted: 08/05/2020] [Indexed: 12/17/2022]
Abstract
Sarcopenia is defined as muscle mass and strength loss with aging. As places, such as South Korea, Japan, and Europe have entered an aged society, sarcopenia is attracting global attention with elderly health. However, only few developed devices can quantify sarcopenia diagnosis modalities. Thus, the authors developed a sarcopenia detection system with 4 degrees of freedom to scan the human thigh with ultrasound probe and determine whether he/she has sarcopenia by inspecting the length of muscle thickness in the thigh by ultrasound image. To accurately measure the muscle thickness, the ultrasound probe attached to the sarcopenia detection system, must be moved angularly along the convex surface of the thigh with predefined pressure maintained. Therefore, the authors proposed an angular thigh scanning method for the aforementioned reason. The method first curve-fits the angular surface of the subject’s thigh with piecewise arcs using D information from a fixed RGB-D camera. Then, it incorporates a Jacobian-based ultrasound probe moving method to move the ultrasound probe along the curve-fitted arc and maintains radial interface force between the probe and the surface by force feedback control. The proposed method was validated by in-vitro test with a human thigh mimicked ham-gelatin phantom. The result showed the ham tissue thickness was maintained within approximately 26.01 ± 1.0 mm during 82° scanning with a 2.5 N radial force setting and the radial force between probe and surface of the phantom was maintained within 2.50 ± 0.1 N.
Collapse
|
25
|
Chen X, Chen H, Peng Y, Tao D. Probe Sector Matching for Freehand 3D Ultrasound Reconstruction. SENSORS (BASEL, SWITZERLAND) 2020; 20:E3146. [PMID: 32498321 PMCID: PMC7308927 DOI: 10.3390/s20113146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/14/2020] [Revised: 05/26/2020] [Accepted: 05/28/2020] [Indexed: 11/16/2022]
Abstract
A 3D ultrasound image reconstruction technique, named probe sector matching (PSM), is proposed in this paper for a freehand linear array ultrasound probe equipped with multiple sensors, providing the position and attitude of the transducer and the pressure between the transducer and the target surface. The proposed PSM method includes three main steps. First, the imaging target and the working range of the probe are set to be the center and the radius of the imaging field of view, respectively. To reconstruct a 3D volume, the positions of all necessary probe sectors are pre-calculated inversely to form a sector database. Second, 2D cross-section probe sectors with the corresponding optical positioning, attitude and pressure information are collected when the ultrasound probe is moving around the imaging target. Last, an improved 3D Hough transform is used to match the plane of the current probe sector to the existing sector images in the sector database. After all pre-calculated probe sectors are acquired and matched into the 3D space defined by the sector database, a 3D ultrasound reconstruction is completed. The PSM is validated through two experiments: a virtual simulation using a numerical model and a lab experiment using a real physical model. The experimental results show that the PSM effectively reduces the errors caused by changes in the target position due to the uneven surface pressure or the inhomogeneity of the transmission media. We conclude that the PSM proposed in this study may help to design a lightweight, inexpensive and flexible ultrasound device with accurate 3D imaging capacity.
Collapse
Affiliation(s)
- Xin Chen
- School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China; (H.C.); (Y.P.); (D.T.)
| | | | | | | |
Collapse
|
26
|
Jiang Z, Grimm M, Zhou M, Esteban J, Simson W, Zahnd G, Navab N. Automatic Normal Positioning of Robotic Ultrasound Probe Based Only on Confidence Map Optimization and Force Measurement. IEEE Robot Autom Lett 2020. [DOI: 10.1109/lra.2020.2967682] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
27
|
A Control Scheme for Physical Human-Robot Interaction Coupled with an Environment of Unknown Stiffness. J INTELL ROBOT SYST 2020. [DOI: 10.1007/s10846-020-01176-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
28
|
Rosen D, Jiang J. Modeling Uncertainty of Strain Ratio Measurements in Ultrasound Breast Strain Elastography: A Factorial Experiment. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2020; 67:258-268. [PMID: 31545719 PMCID: PMC8011866 DOI: 10.1109/tuffc.2019.2942821] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/06/2023]
Abstract
Strain elastography (SE) is a technique in which images of localized tissue strains are used to detect the relative stiffness of tissues. The application of SE in differentiating malignant breast lesions from benign ones has been broadly investigated. The strain ratio (SR) between the background and the breast tumor has been used and its results have been mixed. Due to the complex nature of tissue elasticity and how it relates to the strain fields measured in SE, the exact reason is not known. In this study, we apply a novel design-of-experiments-based metamodeling approach to mechanical simulation of SE in the human breast. To our knowledge, such a study has not been reported in the ultrasound SE literature. More specifically, we first conduct a screening study to identify the biomechanical factors/simulation inputs that most strongly determine SR. We then apply a response surface experimental design to these factors to produce a metamodel of SR as a function of said factors. Results from the screening study suggest that the SR measurements are primarily influenced by three factors: the initial shear modulus of the lesion, the elastic nonlinearity of the lesion, and the precompression applied during acquisition. In order to investigate the implications of these results, stochastic inputs for these three factors associated with the malignant and benign cases were applied to the resulting response surface. The resulting optimal cutoffs, sensitivity, and specificity were generally in line with a majority (>60%) of 19 clinical trials in the literature.
Collapse
|
29
|
Frontin JB, Anthony BW. Quantifying Dermatology: Method and Device for User-Independent Ultrasound Measurement of Skin Thickness. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2019; 2019:5743-5748. [PMID: 31947157 DOI: 10.1109/embc.2019.8857813] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
A device and technique to acquire and construct 3D ultrasound volumes of the skin of the hand and arm were developed. The Repeated Skin Thickness Measurement (RSTM) Device moves a high frequency ultrasound probe linearly in 3 axes in a water tank and images a submerged arm. These images are combined into an ultrasound volume, the skin layer segmented, and the thickness extracted. One particular application is measuring progression of scleroderma, a skin thickening disease. The current ultrasound-based scleroderma diagnostic processes assess skin thickness based on a single ultrasound image taken by a clinician holding the ultrasound probe, resulting in low measurement repeatability. The imagery that results from the instrumentation and analysis presented here can be used to create quantitative maps of skin thickness, to monitor the progression of skin-thickening diseases, and to observe the structures of tendons, ligaments, and the other soft tissue of the hand.
Collapse
|
30
|
Norton JC, Slawinski PR, Lay HS, Martin JW, Cox BF, Cummins G, Desmulliez MP, Clutton RE, Obstein KL, Cochran S, Valdastri P. Intelligent magnetic manipulation for gastrointestinal ultrasound. Sci Robot 2019; 4:eaav7725. [PMID: 31380501 PMCID: PMC6677276 DOI: 10.1126/scirobotics.aav7725] [Citation(s) in RCA: 33] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
Diagnostic endoscopy in the gastrointestinal tract has remained largely unchanged for decades and is limited to the visualization of the tissue surface, the collection of biopsy samples for diagnoses, and minor interventions such as clipping or tissue removal. In this work, we present the autonomous servoing of a magnetic capsule robot for in-situ, subsurface diagnostics of microanatomy. We investigated and showed the feasibility of closed-loop magnetic control using digitized microultrasound (μUS) feedback; this is crucial for obtaining robust imaging in an unknown and unconstrained environment. We demonstrated the functionality of an autonomous servoing algorithm that uses μUS feedback, both on benchtop trials as well as in-vivo in a porcine model. We have validated this magnetic-μUS servoing in instances of autonomous linear probe motion and were able to locate markers in an agar phantom with 1.0 ± 0.9 mm position accuracy using a fusion of robot localization and μUS image information. This work demonstrates the feasibility of closed-loop robotic μUS imaging in the bowel without the need for either a rigid physical link between the transducer and extracorporeal tools or complex manual manipulation.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | | | - Keith L. Obstein
- STORM Lab USA, Vanderbilt University, Nashville, USA
- Vanderbilt University Medical Center, Nashville, USA
| | - Sandy Cochran
- University of Glasgow, School of Mechanical Engineering, Glasgow, UK
| | | |
Collapse
|
31
|
Fincke JR, Wynn CM, Haupt R, Zhang X, Rivera D, Anthony B. Characterization of laser ultrasound source signals in biological tissues for imaging applications. JOURNAL OF BIOMEDICAL OPTICS 2018; 24:1-11. [PMID: 30550046 PMCID: PMC6987635 DOI: 10.1117/1.jbo.24.2.021206] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/25/2018] [Accepted: 11/07/2018] [Indexed: 05/30/2023]
Abstract
Short optical pulses emitted from a tunable Q-switched laser (800 to 2000 nm) generate laser ultrasound (LUS) signals at the surface of biological tissue. The LUS signal's acoustic frequency content, dependence on sample type, and optical wavelength are observed in the far field. The experiments yield a reference dataset for the design of noncontact LUS imaging systems. Measurements show that the majority of LUS signal energy in biological tissues is within the 0.5 and 3 MHz frequency bands and the total acoustic energy generated increases with the optical absorption coefficient of water, which governs tissue optical absorption in the infrared range. The experimental results also link tissue surface roughness and acoustic attenuation with limited LUS signal bandwidth in biological tissue. Images constructed using 810-, 1064-, 1550-, and 2000-nm generation laser wavelengths and a contact piezoelectric receiver demonstrates the impact of the generation laser wavelength on image quality. A noncontact LUS-based medical imaging system has the potential to be an effective medical imaging device. Such a system may mitigate interoperator variability associated with current medical ultrasound imaging techniques and expand the scope of imaging applications for ultrasound.
Collapse
Affiliation(s)
- Jonathan R. Fincke
- Massachusetts Institute of Technology, Department of Mechanical Engineering, Cambridge, Massachusetts, United States
- Institute for Medical Engineering and Science (IMES), Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
| | - Charles M. Wynn
- Massachusetts Institute of Technology, Lincoln Laboratory, Lexington, Massachusetts, United States
| | - Rob Haupt
- Massachusetts Institute of Technology, Lincoln Laboratory, Lexington, Massachusetts, United States
| | - Xiang Zhang
- Massachusetts Institute of Technology, Department of Mechanical Engineering, Cambridge, Massachusetts, United States
- Institute for Medical Engineering and Science (IMES), Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
| | - Diego Rivera
- Massachusetts Institute of Technology, Lincoln Laboratory, Lexington, Massachusetts, United States
| | - Brian Anthony
- Massachusetts Institute of Technology, Department of Mechanical Engineering, Cambridge, Massachusetts, United States
- Institute for Medical Engineering and Science (IMES), Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
| |
Collapse
|
32
|
Li HY, Paranawithana I, Yang L, Lim TSK, Foong S, Ng FC, Tan UX. Stable and Compliant Motion of Physical Human–Robot Interaction Coupled With a Moving Environment Using Variable Admittance and Adaptive Control. IEEE Robot Autom Lett 2018. [DOI: 10.1109/lra.2018.2812916] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
33
|
Huang Q, Wu B, Lan J, Li X. Fully Automatic Three-Dimensional Ultrasound Imaging Based on Conventional B-Scan. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2018; 12:426-436. [PMID: 29570068 DOI: 10.1109/tbcas.2017.2782815] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Robotic ultrasound systems have turned into clinical use over the past few decades, increasing precision and quality of medical operations. In this paper, we propose a fully automatic scanning system for three-dimensional (3-D) ultrasound imaging. A depth camera was first used to obtain the depth data and color data of the tissue surface. Based on the depth image, the 3-D contour of the tissue was rendered and the scan path of ultrasound probe was automatically planned. Following the scan path, a 3-D translating device drove the probe to move on the tissue surface. Simultaneously, the B-scans and their positional information were recorded for subsequent volume reconstruction. In order to stop the scanning process when the pressure on the skin exceeded a preset threshold, two force sensors were attached to the front side of the probe for force measurement. In vitro and in vivo experiments were conducted for assessing the performance of the proposed system. Quantitative results show that the error of volume measurement was less than 1%, indicating that the system is capable of automatic ultrasound scanning and 3-D imaging. It is expected that the proposed system can be well used in clinical practices.
Collapse
|
34
|
|
35
|
Ikhsan M, Tan KK, Putra AS. Assistive technology for ultrasound-guided central venous catheter placement. J Med Ultrason (2001) 2017; 45:41-57. [DOI: 10.1007/s10396-017-0789-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2016] [Accepted: 03/30/2017] [Indexed: 11/28/2022]
|
36
|
Pigula AJ, Wu JS, Gilbertson MW, Darras BT, Rutkove SB, Anthony BW. Force-controlled ultrasound to measure passive mechanical properties of muscle in Duchenne muscular dystrophy. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2016; 2016:2865-2868. [PMID: 28324973 DOI: 10.1109/embc.2016.7591327] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
The purpose of this study is to assess differences in skeletal muscle compressibility between patients with Duchenne muscular dystrophy (DMD) and normal subjects. The transverse passive mechanical properties of muscle, particularly those related to stiffness and elasticity, can be measured using force-controlled ultrasound. We acquired ultrasound videos of muscle compression under known pressures in the biceps and quadriceps in 23 boys with DMD and 20 age-matched healthy controls. We calculated the bulk linear spring constant, nonlinear stress-strain response, and average Young's modulus for each. Young's modulus was found to be significantly higher in the DMD population in both the biceps (normal: 33 ± 6 kPa, DMD: 45 ± 14, p <; .01) and quadriceps (normal: 42 ± 6, DMD: 58 ± 14, p <; .0001). Muscle compressibility measured by force-controlled ultrasound is an objective and robust technique to quantitatively monitor the effects of DMD and distinguish from normal subjects.
Collapse
|