1
|
Rauby B, Xing P, Gasse M, Provost J. Deep Learning in Ultrasound Localization Microscopy: Applications and Perspectives. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2024; 71:1765-1784. [PMID: 39288061 DOI: 10.1109/tuffc.2024.3462299] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/19/2024]
Abstract
Ultrasound localization microscopy (ULM) is a novel super-resolution imaging technique that can image the vasculature in vivo at depth with resolution far beyond the conventional limit of diffraction. By relying on the localization and tracking of clinically approved microbubbles injected in the blood stream, ULM can provide not only anatomical visualization but also hemodynamic quantification of the microvasculature. Several deep learning approaches have been proposed to address challenges in ULM including denoising, improving microbubble localization, estimating blood flow velocity, or performing aberration correction. Proposed deep learning methods often outperform their conventional counterparts by improving image quality and reducing processing time. In addition, their robustness to high concentrations of microbubbles can lead to reduced acquisition times in ULM, addressing a major hindrance to ULM clinical application. Herein, we propose a comprehensive review of the diversity of deep learning applications in ULM focusing on approaches assuming a sparse microbubble distribution. We first provide an overview of how existing studies vary in the constitution of their datasets or in the tasks targeted by the deep learning model. We also take a deeper look into the numerous approaches that have been proposed to improve the localization of microbubbles since they differ highly in their formulation of the optimization problem, their evaluation, or their network architectures. We finally discuss the current limitations and challenges of these methods, as well as the promises and potential of deep learning for ULM in the future.
Collapse
|
2
|
Ashikuzzaman M, Sharma A, Venkatayogi N, Oluyemi E, Myers K, Ambinder E, Rivaz H, Lediju Bell MA. MixTURE: L1-Norm-Based Mixed Second-Order Continuity in Strain Tensor Ultrasound Elastography. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2024; 71:1389-1405. [PMID: 39186421 PMCID: PMC11861389 DOI: 10.1109/tuffc.2024.3449815] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/28/2024]
Abstract
Energy-based displacement tracking of ultrasound images can be implemented by optimizing a cost function consisting of a data term, a mechanical congruency term, and first- and second-order continuity terms. This approach recently provided a promising solution to 2-D axial and lateral displacement tracking in ultrasound strain elastography. However, the associated second-order regularizer only considers the unmixed second derivatives and disregards the mixed derivatives, thereby providing suboptimal noise suppression and limiting possibilities for total strain tensor imaging. We propose to improve axial, lateral, axial shear, and lateral shear strain estimation quality by formulating and optimizing a novel -norm-based second-order regularizer that penalizes both mixed and unmixed displacement derivatives. We name the proposed technique -MixTURE, which stands for -norm Mixed derivative for Total UltRasound Elastography. When compared with simulated ground-truth results, the mean structural similarity (MSSIM) obtained with -MixTURE ranged 0.53-0.86 and the mean absolute error (MAE) ranged 0.00053-0.005. In addition, the mean elastographic signal-to-noise ratio (SNR) achieved with simulated, experimental phantom, and in vivo breast datasets ranged 1.87-52.98, and the mean elastographic contrast-to-noise ratio (CNR) ranged 7.40-24.53. When compared with a closely related existing technique that does not consider the mixed derivatives, -MixTURE generally outperformed the MSSIM, MAE, SNR, and CNR by up to 37.96%, 67.82%, and 25.53% in the simulated, experimental phantom, and in vivo datasets, respectively. These results collectively highlight the ability of -MixTURE to deliver highly accurate axial, lateral, axial shear, and lateral shear strain estimates and advance the state-of-the-art in elastography-guided diagnostic and interventional decisions.
Collapse
|
3
|
Asgariandehkordi H, Goudarzi S, Sharifzadeh M, Basarab A, Rivaz H. Denoising Plane Wave Ultrasound Images Using Diffusion Probabilistic Models. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2024; 71:1526-1539. [PMID: 39186422 DOI: 10.1109/tuffc.2024.3448209] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/28/2024]
Abstract
Ultrasound plane wave (PW) imaging is a cutting-edge technique that enables high frame-rate imaging. However, one challenge associated with high frame-rate ultrasound imaging is the high noise associated with them, hindering their wider adoption. Therefore, the development of a denoising method becomes imperative to augment the quality of PW images. Drawing inspiration from denoising diffusion probabilistic models (DDPMs), our proposed solution aims to enhance PW image quality. Specifically, the method considers the distinction between low-angle and high-angle compounding PWs as noise and effectively eliminates it by adapting a DDPM to beamformed radio frequency (RF) data. The method underwent training using only 400 simulated images. In addition, our approach employs natural image segmentation masks as intensity maps for the generated images, resulting in accurate denoising for various anatomy shapes. The proposed method was assessed across simulation, phantom, and in vivo images. The results of the evaluations indicate that our approach not only enhances the image quality on simulated data but also demonstrates effectiveness on phantom and in vivo data in terms of image quality. Comparative analysis with other methods underscores the superiority of our proposed method across various evaluation metrics. The source code and trained model will be released along with the dataset at: http://code.sonography.ai.
Collapse
|
4
|
Xiang T, Li Y, Deng H, Tian C, Peng B, Jiang J. Teacher-student guided knowledge distillation for unsupervised convolutional neural network-based speckle tracking in ultrasound strain elastography. Med Biol Eng Comput 2024; 62:2265-2279. [PMID: 38627356 DOI: 10.1007/s11517-024-03078-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2023] [Accepted: 03/22/2024] [Indexed: 07/31/2024]
Abstract
Accurate and efficient motion estimation is a crucial component of real-time ultrasound elastography (USE). However, obtaining radiofrequency ultrasound (RF) data in clinical practice can be challenging. In contrast, although B-mode (BM) data is readily available, elastographic data derived from BM data results in sub-optimal elastographic images. Furthermore, existing conventional ultrasound devices (e.g., portable devices) cannot provide elastography modes, which has become a significant obstacle to the widespread use of traditional ultrasound devices. To address the challenges above, we developed a teacher-student guided knowledge distillation for an unsupervised convolutional neural network (TSGUPWC-Net) to improve the accuracy of BM motion estimation by employing a well-established convolutional neural network (CNN) named modified pyramid warping and cost volume network (MPWC-Net). A pre-trained teacher model based on RF is utilized to guide the training of a student model using BM data. Innovations outlined below include employing spatial attention transfer at intermediate layers to enhance the guidance effect of the model. The loss function consists of smoothness of the displacement field, knowledge distillation loss, and intermediate layer loss. We evaluated our method on simulated data, phantoms, and in vivo ultrasound data. The results indicate that our method has higher signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) values in axial strain estimation than the model trained on BM. The model is unsupervised and requires no ground truth labels during training, making it highly promising for motion estimation applications.
Collapse
Affiliation(s)
- Tianqiang Xiang
- School of Computer Science and Software Engineering, Southwest Petroleum University, Sichuan, China
| | - Yan Li
- School of Computer Science and Software Engineering, Southwest Petroleum University, Sichuan, China
| | - Hui Deng
- School of Computer Science and Software Engineering, Southwest Petroleum University, Sichuan, China
| | - Chao Tian
- Operation and Development Department, Sichuan Clinical Research Center for Cancer, Sichuan Cancer Hospital & Institute, Chengdu, China
| | - Bo Peng
- School of Computer Science and Software Engineering, Southwest Petroleum University, Sichuan, China.
| | - Jingfeng Jiang
- Department of Biomedical Engineering, Michigan Technological University, Houghton, MI, 49931, USA.
- Department of Medical Physics, University of Wisconsin, Madison, WI, 53705, USA.
| |
Collapse
|
5
|
Ashikuzzaman M, Peng B, Jiang J, Rivaz H. Alternating direction method of multipliers for displacement estimation in ultrasound strain elastography. Med Phys 2024; 51:3521-3540. [PMID: 38159299 DOI: 10.1002/mp.16921] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 12/11/2023] [Accepted: 12/13/2023] [Indexed: 01/03/2024] Open
Abstract
BACKGROUND Ultrasound strain imaging, which delineates mechanical properties to detect tissue abnormalities, involves estimating the time delay between two radio-frequency (RF) frames collected before and after tissue deformation. The existing regularized optimization-based time-delay estimation (TDE) techniques suffer from at least one of the following drawbacks: (1) The regularizer is not aligned with the tissue deformation physics due to taking only the first-order displacement derivative into account; (2) TheL 2 $L2$ -norm of the displacement derivatives, which oversmooths the estimated time-delay, is utilized as the regularizer; (3) The modulus function defined mathematically should be approximated by a smooth function to facilitate the optimization ofL 1 $L1$ -norm. PURPOSE Our purpose is to develop a novel TDE technique that resolves the aforementioned shortcomings of the existing algorithms. METHODS Herein, we propose employing the alternating direction method of multipliers (ADMM) for optimizing a novel cost function consisting ofL 2 $L2$ -norm data fidelity term andL 1 $L1$ -norm first- and second-order spatial continuity terms. ADMM empowers the proposed algorithm to use different techniques for optimizing different parts of the cost function and obtain high-contrast strain images with smooth backgrounds and sharp boundaries. We name our technique ADMM for totaL variaTion RegUlarIzation in ultrasound STrain imaging (ALTRUIST). ALTRUIST's efficacy is quantified using absolute error (AE), Structural SIMilarity (SSIM), signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), and strain ratio (SR) with respect to GLUE, OVERWIND, andL 1 $L1$ -SOUL, three recently published energy-based techniques, and UMEN-Net, a state-of-the-art deep learning-based algorithm. Analysis of variance (ANOVA)-led multiple comparison tests and paired t $t$ -tests at5 % $5\%$ overall significance level were conducted to assess the statistical significance of our findings. The Bonferroni correction was taken into account in all statistical tests. Two simulated layer phantoms, three simulated resolution phantoms, one hard-inclusion simulated phantom, one multi-inclusion simulated phantom, one experimental breast phantom, and three in vivo liver cancer datasets have been used for validation experiments. We have published the ALTRUIST code at http://code.sonography.ai. RESULTS ALTRUIST substantially outperforms the four state-of-the-art benchmarks in all validation experiments, both qualitatively and quantitatively. ALTRUIST yields up to573 % ∗ ${573\%}^{*}$ ,41 % ∗ ${41\%}^{*}$ , and51 % ∗ ${51\%}^{*}$ SNR improvements and443 % ∗ ${443\%}^{*}$ ,53 % ∗ ${53\%}^{*}$ , and15 % ∗ ${15\%}^{*}$ CNR improvements overL 1 $L1$ -SOUL, its closest competitor, for simulated, phantom, and in vivo liver cancer datasets, respectively, where the asterisk (*) indicates statistical significance. In addition, ANOVA-led multiple comparison tests and paired t $t$ -tests indicate that ALTRUIST generally achieves statistically significant improvements over GLUE, UMEN-Net, OVERWIND, andL 1 $L1$ -SOUL in terms of AE, SSIM map, SNR, and CNR. CONCLUSIONS A novel ultrasonic displacement tracking algorithm named ALTRUIST has been developed. The principal novelty of ALTRUIST is incorporating ADMM for optimizing anL 1 $L1$ -norm regularization-based cost function. ALTRUIST exhibits promising performance in simulation, phantom, and in vivo experiments.
Collapse
Affiliation(s)
- Md Ashikuzzaman
- Department of Electrical and Computer Engineering, Concordia University, Montreal, Québec, Canada
| | - Bo Peng
- School of Computer Science and Software Engineering, Southwest Petroleum University, Chengdu, China
| | - Jingfeng Jiang
- Department of Biomedical Engineering, Michigan Technological University, Houghton, Michigan, USA
| | - Hassan Rivaz
- Department of Electrical and Computer Engineering, Concordia University, Montreal, Québec, Canada
| |
Collapse
|
6
|
Chen X, Li X, Turco S, van Sloun RJG, Mischi M. Ultrasound Viscoelastography by Acoustic Radiation Force: A State-of-the-Art Review. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2024; 71:536-557. [PMID: 38526897 DOI: 10.1109/tuffc.2024.3381529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/27/2024]
Abstract
Ultrasound elastography (USE) is a promising tool for tissue characterization as several diseases result in alterations of tissue structure and composition, which manifest as changes in tissue mechanical properties. By imaging the tissue response to an applied mechanical excitation, USE mimics the manual palpation performed by clinicians to sense the tissue elasticity for diagnostic purposes. Next to elasticity, viscosity has recently been investigated as an additional, relevant, diagnostic biomarker. Moreover, since biological tissues are inherently viscoelastic, accounting for viscosity in the tissue characterization process enhances the accuracy of the elasticity estimation. Recently, methods exploiting different acquisition and processing techniques have been proposed to perform ultrasound viscoelastography. After introducing the physics describing viscoelasticity, a comprehensive overview of the currently available USE acquisition techniques is provided, followed by a structured review of the existing viscoelasticity estimators classified according to the employed processing technique. These estimators are further reviewed from a clinical usage perspective, and current outstanding challenges are discussed.
Collapse
|
7
|
Lu J, Millioz F, Varray F, Poree J, Provost J, Bernard O, Garcia D, Friboulet D. Ultrafast Cardiac Imaging Using Deep Learning for Speckle-Tracking Echocardiography. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2023; 70:1761-1772. [PMID: 37862280 DOI: 10.1109/tuffc.2023.3326377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/22/2023]
Abstract
High-quality ultrafast ultrasound imaging is based on coherent compounding from multiple transmissions of plane waves (PW) or diverging waves (DW). However, compounding results in reduced frame rate, as well as destructive interferences from high-velocity tissue motion if motion compensation (MoCo) is not considered. While many studies have recently shown the interest of deep learning for the reconstruction of high-quality static images from PW or DW, its ability to achieve such performance while maintaining the capability of tracking cardiac motion has yet to be assessed. In this article, we addressed such issue by deploying a complex-weighted convolutional neural network (CNN) for image reconstruction and a state-of-the-art speckle-tracking method. The evaluation of this approach was first performed by designing an adapted simulation framework, which provides specific reference data, i.e., high-quality, motion artifact-free cardiac images. The obtained results showed that, while using only three DWs as input, the CNN-based approach yielded an image quality and a motion accuracy equivalent to those obtained by compounding 31 DWs free of motion artifacts. The performance was then further evaluated on nonsimulated, experimental in vitro data, using a spinning disk phantom. This experiment demonstrated that our approach yielded high-quality image reconstruction and motion estimation, under a large range of velocities and outperforms a state-of-the-art MoCo-based approach at high velocities. Our method was finally assessed on in vivo datasets and showed consistent improvement in image quality and motion estimation compared to standard compounding. This demonstrates the feasibility and effectiveness of deep learning reconstruction for ultrafast speckle-tracking echocardiography.
Collapse
|
8
|
Tehrani AKZ, Rosado-Mendez IM, Rivaz H. Deep Model Projected Statistical Features for Homodyned K-Distribution Parameters Estimation. 2023 IEEE INTERNATIONAL ULTRASONICS SYMPOSIUM (IUS) 2023:1-4. [DOI: 10.1109/ius51837.2023.10306362] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/04/2025]
Affiliation(s)
- Ali K. Z. Tehrani
- Concordia University,Electrical & Computer Eng. Dept.,Montreal,Canada
| | - Ivan M. Rosado-Mendez
- University of Wisconsin,Departments of Medical Physics and Radiology,Madison,United States
| | - Hassan Rivaz
- Concordia University,Electrical & Computer Eng. Dept.,Montreal,Canada
| |
Collapse
|
9
|
Karageorgos GM, Liang P, Mobadersany N, Gami P, Konofagou EE. Unsupervised deep learning-based displacement estimation for vascular elasticity imaging applications. Phys Med Biol 2023; 68:10.1088/1361-6560/ace0f0. [PMID: 37348487 PMCID: PMC10528442 DOI: 10.1088/1361-6560/ace0f0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2023] [Accepted: 06/22/2023] [Indexed: 06/24/2023]
Abstract
Objective. Arterial wall stiffness can provide valuable information on the proper function of the cardiovascular system. Ultrasound elasticity imaging techniques have shown great promise as a low-cost and non-invasive tool to enable localized maps of arterial wall stiffness. Such techniques rely upon motion detection algorithms that provide arterial wall displacement estimation.Approach. In this study, we propose an unsupervised deep learning-based approach, originally proposed for image registration, in order to enable improved quality arterial wall displacement estimation at high temporal and spatial resolutions. The performance of the proposed network was assessed through phantom experiments, where various models were trained by using ultrasound RF signals, or B-mode images, as well as different loss functions.Main results. Using the mean square error (MSE) for the training process provided the highest signal-to-noise ratio when training on the B-modes images (30.36 ± 1.14 dB) and highest contrast-to-noise ratio when training on the RF signals (32.84 ± 1.89 dB). In addition, training the model on RF signals demonstrated the capability of providing accurate localized pulse wave velocity (PWV) maps, with a mean relative error (MREPWV) of 3.32 ± 1.80% and anR2 of 0.97 ± 0.03. Finally, the developed model was tested in human common carotid arteriesin vivo, providing accurate tracking of the distension pulse wave propagation, with an MREPWV= 3.86 ± 2.69% andR2 = 0.95 ± 0.03.Significance. In conclusion, a novel displacement estimation approach was presented, showing promise in improving vascular elasticity imaging techniques.
Collapse
Affiliation(s)
- Grigorios M Karageorgos
- Biomedical Engineering Department, Columbia University, New York, NY, United States of America
| | - Pengcheng Liang
- Biomedical Engineering Department, Columbia University, New York, NY, United States of America
| | - Nima Mobadersany
- Department of Radiology, Columbia University, New York, NY, United States of America
| | - Parth Gami
- Biomedical Engineering Department, Columbia University, New York, NY, United States of America
| | - Elisa E Konofagou
- Biomedical Engineering Department, Columbia University, New York, NY, United States of America
- Department of Radiology, Columbia University, New York, NY, United States of America
| |
Collapse
|
10
|
Tehrani AKZ, Ashikuzzaman M, Rivaz H. Lateral Strain Imaging Using Self-Supervised and Physically Inspired Constraints in Unsupervised Regularized Elastography. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:1462-1471. [PMID: 37015465 DOI: 10.1109/tmi.2022.3230635] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Convolutional Neural Networks (CNN) have shown promising results for displacement estimation in UltraSound Elastography (USE). Many modifications have been proposed to improve the displacement estimation of CNNs for USE in the axial direction. However, the lateral strain, which is essential in several downstream tasks such as the inverse problem of elasticity imaging, remains a challenge. The lateral strain estimation is complicated since the motion and the sampling frequency in this direction are substantially lower than the axial one, and a lack of carrier signal in this direction. In computer vision applications, the axial and the lateral motions are independent. In contrast, the tissue motion pattern in USE is governed by laws of physics which link the axial and lateral displacements. In this paper, inspired by Hooke's law, we, first propose Physically Inspired ConsTraint for Unsupervised Regularized Elastography (PICTURE), where we impose a constraint on the Effective Poisson's ratio (EPR) to improve the lateral strain estimation. In the next step, we propose self-supervised PICTURE (sPICTURE) to further enhance the strain image estimation. Extensive experiments on simulation, experimental phantom and in vivo data demonstrate that the proposed methods estimate accurate axial and lateral strain maps.
Collapse
|
11
|
Wen S, Peng B, Wei X, Luo J, Jiang J. Convolutional Neural Network-Based Speckle Tracking for Ultrasound Strain Elastography: An Unsupervised Learning Approach. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2023; 70:354-367. [PMID: 37022912 DOI: 10.1109/tuffc.2023.3243539] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/16/2023]
Abstract
Accurate and computationally efficient motion estimation is a critical component of real-time ultrasound strain elastography (USE). With the advent of deep-learning neural network models, a growing body of work has explored supervised convolutional neural network (CNN)-based optical flow in the framework of USE. However, the above-said supervised learning was often done using simulated ultrasound data. The research community has questioned whether simulated ultrasound data containing simple motion can train deep-learning CNN models that can reliably track complex in vivo speckle motion. In parallel with other research groups' efforts, this study developed an unsupervised motion estimation neural network (UMEN-Net) for USE by adapting a well-established CNN model named PWC-Net. Our network's input is a pair of predeformation and postdeformation radio frequency (RF) echo signals. The proposed network outputs both axial and lateral displacement fields. The loss function consists of a correlation between the predeformation signal and the motion-compensated postcompression signal, smoothness of the displacement fields, and tissue incompressibility. Notably, an innovative correlation method known as the globally optimized correspondence (GOCor) volumes module developed by Truong et al. was used to replace the original Corr module to enhance our evaluation of signal correlation. The proposed CNN model was tested using simulated, phantom, and in vivo ultrasound data containing biologically confirmed breast lesions. Its performance was compared against other state-of-the-art methods, including two deep-learning-based tracking methods (MPWC-Net++ and ReUSENet) and two conventional tracking methods (GLUE and BRGMT-LPF). In summary, compared with the four known methods mentioned above, our unsupervised CNN model not only obtained higher signal-to-noise ratios (SNRs) and contrast-to-noise ratios (CNRs) for axial strain estimates but also improved the quality of the lateral strain estimates.
Collapse
|
12
|
Chen X, Chennakeshava N, Wildeboer R, Mischi M, van Sloun RJG. Shear-Wave Particle-Velocity Estimation and Enhancement Using a Multi-Resolution Convolutional Neural Network. ULTRASOUND IN MEDICINE & BIOLOGY 2023; 49:1518-1526. [PMID: 37088606 DOI: 10.1016/j.ultrasmedbio.2023.02.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Revised: 02/01/2023] [Accepted: 02/07/2023] [Indexed: 05/03/2023]
Abstract
OBJECTIVE Tissue mechanical properties are valuable markers for tissue characterization, aiding in the detection and staging of pathologies. Shear wave elastography (SWE) offers a quantitative assessment of tissue mechanical characteristics based on the SW propagation profile, which is derived from the SW particle motion. Improving the signal-to-noise ratio (SNR) of the SW particle motion would directly enhance the accuracy of the material property estimates such as elasticity or viscosity. METHODS In this paper, we present a 3-D multi-resolution convolutional neural network (MRCNN) to perform improved estimation of the SW particle velocity Vz. Additionally, we propose a novel approach to generate training data from real acquisitions, providing high SNR ground truth target data, one-to-one paired to inputs that are corrupted with real-world noise and disturbances. DISCUSSION By testing the network on in vitro data acquired from a commercial breast elastography phantom, we show that the MRCNN outperforms Loupas' autocorrelation algorithm with an improved SNR of 4.47 dB for the Vz signals, a two-fold decrease in the standard deviation of the downstream elasticity estimates, and a two-fold increase in the contrast-to-noise ratio of the elasticity maps. The generalizability of the network was further demonstrated with a set of ex vivo porcine liver data. CONCLUSION The proposed MRCNN outperforms the standard autocorrelation method, in particular in low SNR regimes.
Collapse
Affiliation(s)
- Xufei Chen
- Lab of Biomedical Diagnostics, Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands.
| | - Nishith Chennakeshava
- Lab of Biomedical Diagnostics, Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands
| | | | - Massimo Mischi
- Lab of Biomedical Diagnostics, Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands
| | - Ruud J G van Sloun
- Lab of Biomedical Diagnostics, Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands
| |
Collapse
|
13
|
Tripura T, Awasthi A, Roy S, Chakraborty S. A wavelet neural operator based elastography for localization and quantification of tumors. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 232:107436. [PMID: 36870167 DOI: 10.1016/j.cmpb.2023.107436] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/07/2022] [Revised: 01/19/2023] [Accepted: 02/18/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND AND OBJECTIVES The application of intelligent imaging techniques and deep learning in the field of computer-aided diagnosis and medical imaging have improved and accelerated the early diagnosis of many diseases. Elastography is an imaging modality where an inverse problem is solved to extract the elastic properties of tissues and subsequently mapped to anatomical images for diagnostic purposes. In the present work, we propose a wavelet neural operator-based approach for correctly learning the non-linear mapping of elastic properties directly from measured displacement field data. METHODS The proposed framework learns the underlying operator behind the elastic mapping and thus can map any displacement data from a family to the elastic properties. The displacement fields are first uplifted to a high-dimensional space using a fully connected neural network. On the lifted data, certain iterations are performed using wavelet neural blocks. In each wavelet neural block, the lifted data are decomposed into low, and high-frequency components using wavelet decomposition. To learn the most relevant patterns and structural information from the input, the neural network kernels are directly convoluted with the outputs of the wavelet decomposition. Thereafter the elasticity field is reconstructed from the outputs from convolution. The mapping between the displacement and the elasticity using wavelets is unique and remains stable during training. RESULTS The proposed framework is tested on several artificially fabricated numerical examples, including a benign-cum-malignant tumor prediction problem. The trained model was also tested on real Ultrasound-based elastography data to demonstrate the applicability of the proposed scheme in clinical usage. The proposed framework reproduces the highly accurate elasticity field directly from the displacement inputs. CONCLUSIONS The proposed framework circumvents different data pre-processing and intermediate steps utilized in traditional methods, hence providing an accurate elasticity map. The computationally efficient framework requires fewer epochs for training, which bodes well for its clinical usability for real-time predictions. The weights and biases from pre-trained models can also be employed for transfer learning, which reduces the effective training time with random initialization.
Collapse
Affiliation(s)
- Tapas Tripura
- Department of Applied Mechanics, Indian Institute of Technology Delhi, Hauz Khas, Delhi, 110016, India.
| | - Abhilash Awasthi
- Department of Applied Mechanics, Indian Institute of Technology Delhi, Hauz Khas, Delhi, 110016, India.
| | - Sitikantha Roy
- Department of Applied Mechanics, Indian Institute of Technology Delhi, Hauz Khas, Delhi, 110016, India; Yardi School of Artificial Intelligence (ScAI), Indian Institute of Technology Delhi, Hauz Khas, Delhi, 110016, India.
| | - Souvik Chakraborty
- Department of Applied Mechanics, Indian Institute of Technology Delhi, Hauz Khas, Delhi, 110016, India; Yardi School of Artificial Intelligence (ScAI), Indian Institute of Technology Delhi, Hauz Khas, Delhi, 110016, India.
| |
Collapse
|
14
|
A new three-dimensional elastography using phase based shifted Fourier transform. MEDICINE IN NOVEL TECHNOLOGY AND DEVICES 2022. [DOI: 10.1016/j.medntd.2022.100186] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022] Open
|
15
|
Evain E, Sun Y, Faraz K, Garcia D, Saloux E, Gerber BL, De Craene M, Bernard O. Motion Estimation by Deep Learning in 2D Echocardiography: Synthetic Dataset and Validation. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:1911-1924. [PMID: 35157582 DOI: 10.1109/tmi.2022.3151606] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Motion estimation in echocardiography plays an important role in the characterization of cardiac function, allowing the computation of myocardial deformation indices. However, there exist limitations in clinical practice, particularly with regard to the accuracy and robustness of measurements extracted from images. We therefore propose a novel deep learning solution for motion estimation in echocardiography. Our network corresponds to a modified version of PWC-Net which achieves high performance on ultrasound sequences. In parallel, we designed a novel simulation pipeline allowing the generation of a large amount of realistic B-mode sequences. These synthetic data, together with strategies during training and inference, were used to improve the performance of our deep learning solution, which achieved an average endpoint error of 0.07 ± 0.06 mm per frame and 1.20 ± 0.67 mm between ED and ES on our simulated dataset. The performance of our method was further investigated on 30 patients from a publicly available clinical dataset acquired from a GE system. The method showed promise by achieving a mean absolute error of the global longitudinal strain of 2.5 ± 2.1% and a correlation of 0.77 compared to GLS derived from manual segmentation, much better than one of the most efficient methods in the state-of-the-art (namely the FFT-Xcorr block-matching method). We finally evaluated our method on an auxiliary dataset including 30 patients from another center and acquired with a different system. Comparable results were achieved, illustrating the ability of our method to maintain high performance regardless of the echocardiographic data processed.
Collapse
|
16
|
Li H, Bhatt M, Qu Z, Zhang S, Hartel MC, Khademhosseini A, Cloutier G. Deep learning in ultrasound elastography imaging: A review. Med Phys 2022; 49:5993-6018. [PMID: 35842833 DOI: 10.1002/mp.15856] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2021] [Revised: 02/04/2022] [Accepted: 07/06/2022] [Indexed: 11/11/2022] Open
Abstract
It is known that changes in the mechanical properties of tissues are associated with the onset and progression of certain diseases. Ultrasound elastography is a technique to characterize tissue stiffness using ultrasound imaging either by measuring tissue strain using quasi-static elastography or natural organ pulsation elastography, or by tracing a propagated shear wave induced by a source or a natural vibration using dynamic elastography. In recent years, deep learning has begun to emerge in ultrasound elastography research. In this review, several common deep learning frameworks in the computer vision community, such as multilayer perceptron, convolutional neural network, and recurrent neural network are described. Then, recent advances in ultrasound elastography using such deep learning techniques are revisited in terms of algorithm development and clinical diagnosis. Finally, the current challenges and future developments of deep learning in ultrasound elastography are prospected. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Hongliang Li
- Laboratory of Biorheology and Medical Ultrasonics, University of Montreal Hospital Research Center, Montréal, Québec, Canada.,Institute of Biomedical Engineering, University of Montreal, Montréal, Québec, Canada
| | - Manish Bhatt
- Laboratory of Biorheology and Medical Ultrasonics, University of Montreal Hospital Research Center, Montréal, Québec, Canada
| | - Zhen Qu
- Laboratory of Biorheology and Medical Ultrasonics, University of Montreal Hospital Research Center, Montréal, Québec, Canada
| | - Shiming Zhang
- California Nanosystems Institute, University of California, Los Angeles, California, USA
| | - Martin C Hartel
- California Nanosystems Institute, University of California, Los Angeles, California, USA
| | - Ali Khademhosseini
- California Nanosystems Institute, University of California, Los Angeles, California, USA
| | - Guy Cloutier
- Laboratory of Biorheology and Medical Ultrasonics, University of Montreal Hospital Research Center, Montréal, Québec, Canada.,Institute of Biomedical Engineering, University of Montreal, Montréal, Québec, Canada.,Department of Radiology, Radio-Oncology and Nuclear Medicine, University of Montreal, Montréal, Québec, Canada
| |
Collapse
|
17
|
Wei X, Wang Y, Ge L, Peng B, He Q, Wang R, Huang L, Xu Y, Luo J. Unsupervised Convolutional Neural Network for Motion Estimation in Ultrasound Elastography. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2022; 69:2236-2247. [PMID: 35500076 DOI: 10.1109/tuffc.2022.3171676] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
High-quality motion estimation is essential for ultrasound elastography (USE). Traditional motion estimation algorithms based on speckle tracking such as normalized cross correlation (NCC) or regularization such as global ultrasound elastography (GLUE) are time-consuming. In order to reduce the computational cost and ensure the accuracy of motion estimation, many convolutional neural networks have been introduced into USE. Most of these networks such as radio-frequency modified pyramid, warping and cost volume network (RFMPWC-Net) are supervised and need many ground truths as labels in network training. However, the ground truths are laborious to collect for USE. In this study, we proposed a MaskFlownet-based unsupervised convolutional neural network (MF-UCNN) for fast and high-quality motion estimation in USE. The inputs to MF-UCNN are the concatenation of RF, envelope, and B-mode images before and after deformation, while the outputs are the axial and lateral displacement fields. The similarity between the predeformed image and the warped image (i.e., the postdeformed image compensated by the estimated displacement fields) and the smoothness of the estimated displacement fields were incorporated in the loss function. The network was compared with modified pyramid, warping and cost volume network (MPWC-Net)++, RFMPWC-Net, GLUE, and NCC. Results of simulations, breast phantom, and in vivo experiments show that MF-UCNN obtains higher signal-to-noise ratio (SNR) and higher contrast-to-noise ratio (CNR). MF-UCNN achieves high-quality motion estimation with significantly reduced computation time. It is unsupervised and does not need any ground truths as labels in the training, and, thus, has great potential for motion estimation in USE.
Collapse
|
18
|
Ashikuzzaman M, Hall TJ, Rivaz H. Incorporating Gradient Similarity for Robust Time Delay Estimation in Ultrasound Elastography. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2022; 69:1738-1750. [PMID: 35363613 DOI: 10.1109/tuffc.2022.3164287] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Energy-based ultrasound elastography techniques minimize a regularized cost function consisting of data and continuity terms to obtain local displacement estimates based on the local time-delay estimation (TDE) between radio frequency (RF) frames. The data term associated with the existing techniques takes only the amplitude similarity into account and hence is not sufficiently robust to the outlier samples present in the RF frames under consideration. This drawback creates noticeable artifacts in the strain image. To resolve this issue, we propose to formulate the data function as a linear combination of the amplitude and gradient similarity constraints. We estimate the adaptive weight concerning each similarity term following an iterative scheme. Finally, we optimize the nonlinear cost function in an efficient manner to convert the problem to a sparse system of linear equations which are solved for millions of variables. We call our technique rGLUE: robust data term in GLobal Ultrasound Elastography. rGLUE has been validated using simulation, phantom, in vivo liver, and breast datasets. In all our experiments, rGLUE substantially outperforms the recent elastography methods both visually and quantitatively. For simulated, phantom, and in vivo datasets, respectively, rGLUE achieves 107%, 18%, and 23% improvements of signal-to-noise ratio (SNR) and 61%, 19%, and 25% improvements of contrast-to-noise ratio (CNR) over global ultrasound elastography (GLUE), a recently published elastography algorithm.
Collapse
|
19
|
Tehrani AKZ, Sharifzadeh M, Boctor E, Rivaz H. Bi-Directional Semi-Supervised Training of Convolutional Neural Networks for Ultrasound Elastography Displacement Estimation. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2022; 69:1181-1190. [PMID: 35085077 DOI: 10.1109/tuffc.2022.3147097] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
The performance of ultrasound elastography (USE) heavily depends on the accuracy of displacement estimation. Recently, convolutional neural networks (CNNs) have shown promising performance in optical flow estimation and have been adopted for USE displacement estimation. Networks trained on computer vision images are not optimized for USE displacement estimation since there is a large gap between the computer vision images and the high-frequency radio frequency (RF) ultrasound data. Many researchers tried to adopt the optical flow CNNs to USE by applying transfer learning to improve the performance of CNNs for USE. However, the ground-truth displacement in real ultrasound data is unknown, and simulated data exhibit a domain shift compared to the real data and are also computationally expensive to generate. To resolve this issue, semisupervised methods have been proposed in which the networks pretrained on computer vision images are fine-tuned using real ultrasound data. In this article, we employ a semisupervised method by exploiting the first- and second-order derivatives of the displacement field for regularization. We also modify the network structure to estimate both forward and backward displacements and propose to use consistency between the forward and backward strains as an additional regularizer to further enhance the performance. We validate our method using several experimental phantom and in vivo data. We also show that the network fine-tuned by our proposed method using experimental phantom data performs well on in vivo data similar to the network fine-tuned on in vivo data. Our results also show that the proposed method outperforms current deep learning methods and is comparable to computationally expensive optimization-based algorithms.
Collapse
|
20
|
Tehrani AKZ, Rosado-Mendez IM, Rivaz H. Robust Scatterer Number Density Segmentation of Ultrasound Images. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2022; 69:1169-1180. [PMID: 35044911 PMCID: PMC11500800 DOI: 10.1109/tuffc.2022.3144685] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Quantitative ultrasound (QUS) aims to reveal information about the tissue microstructure using backscattered echo signals from clinical scanners. Among different QUS parameters, scatterer number density is an important property that can affect the estimation of other QUS parameters. Scatterer number density can be classified into high or low scatterer densities. If there are more than ten scatterers inside the resolution cell, the envelope data are considered as fully developed speckle (FDS) and, otherwise, as underdeveloped speckle (UDS). In conventional methods, the envelope data are divided into small overlapping windows (a strategy here we refer to as patching), and statistical parameters, such as SNR and skewness, are employed to classify each patch of envelope data. However, these parameters are system-dependent, meaning that their distribution can change by the imaging settings and patch size. Therefore, reference phantoms that have known scatterer number density are imaged with the same imaging settings to mitigate system dependency. In this article, we aim to segment regions of ultrasound data without any patching. A large dataset is generated, which has different shapes of scatterer number density and mean scatterer amplitude using a fast simulation method. We employ a convolutional neural network (CNN) for the segmentation task and investigate the effect of domain shift when the network is tested on different datasets with different imaging settings. Nakagami parametric image is employed for multitask learning to improve performance. Furthermore, inspired by the reference phantom methods in QUS, a domain adaptation stage is proposed, which requires only two frames of data from FDS and UDS classes. We evaluate our method for different experimental phantoms and in vivo data.
Collapse
|
21
|
Byra M, Jarosik P, Dobruch-Sobczak K, Klimonda Z, Piotrzkowska-Wroblewska H, Litniewski J, Nowicki A. Joint segmentation and classification of breast masses based on ultrasound radio-frequency data and convolutional neural networks. ULTRASONICS 2022; 121:106682. [PMID: 35065458 DOI: 10.1016/j.ultras.2021.106682] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/01/2021] [Revised: 12/08/2021] [Accepted: 12/30/2021] [Indexed: 06/14/2023]
Abstract
In this paper, we propose a novel deep learning method for joint classification and segmentation of breast masses based on radio-frequency (RF) ultrasound (US) data. In comparison to commonly used classification and segmentation techniques, utilizing B-mode US images, we train the network with RF data (data before envelope detection and dynamic compression), which are considered to include more information on tissue's physical properties than standard B-mode US images. Our multi-task network, based on the Y-Net architecture, can effectively process large matrices of RF data by mixing 1D and 2D convolutional filters. We use data collected from 273 breast masses to compare the performance of networks trained with RF data and US images. The multi-task model developed based on the RF data achieved good classification performance, with area under the receiver operating characteristic curve (AUC) of 0.90. The network based on the US images achieved AUC of 0.87. In the case of the segmentation, we obtained mean Dice scores of 0.64 and 0.60 for the approaches utilizing US images and RF data, respectively. Moreover, the interpretability of the networks was studied using class activation mapping technique and by filter weights visualizations.
Collapse
Affiliation(s)
- Michal Byra
- Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland.
| | - Piotr Jarosik
- Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland
| | - Katarzyna Dobruch-Sobczak
- Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland; Maria Sklodowska-Curie Memorial Cancer Centre and Institute of Oncology, Warsaw, Poland
| | - Ziemowit Klimonda
- Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland
| | | | - Jerzy Litniewski
- Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland
| | - Andrzej Nowicki
- Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland
| |
Collapse
|
22
|
Wang Y, Xie X, He Q, Liao H, Zhang H, Luo J. Hadamard-Encoded Synthetic Transmit Aperture Imaging for Improved Lateral Motion Estimation in Ultrasound Elastography. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2022; 69:1204-1218. [PMID: 35100113 DOI: 10.1109/tuffc.2022.3148332] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Lateral motion estimation has been a challenge in ultrasound elastography mainly due to the low resolution, low sampling frequency, and lack of phase information in the lateral direction. Synthetic transmit aperture (STA) can achieve high resolution due to two-way focusing and can beamform high-density image lines for improved lateral motion estimation with the disadvantages of low signal-to-noise ratio (SNR) and limited penetration depth. In this study, Hadamard-encoded STA (Hadamard-STA) is proposed for the improvement of lateral motion estimation in elastography, and it is compared with STA and conventional focused wave (CFW) imaging. Simulations, phantom, and in vivo experiments were conducted to make the comparison. The normalized root mean square error (NRMSE) and the contrast-to-noise ratio (CNR) were calculated as the evaluation criteria in the simulations. The results show that, at a noise level of -10 dB and an applied strain of -1% (compression), Hadamard-STA decreases the NRMSEs of lateral displacements by 46.92% and 35.35%, decreases the NRMSEs of lateral strains by 52.34% and 39.75%, and increases the CNRs by 9.70 and 9.75 dB compared with STA and CFW, respectively. In the phantom experiments performed on a heterogeneous tissue-mimicking phantom, the sum of squared differences (SSD) between the reference and the motion-compensated RF data, and the CNR were calculated as the evaluation criteria. At an applied strain of -1.80%, Hadamard-STA is found to decrease the SSDs by 20.91% and 30.99% and increase the CNRs by 14.15 and 24.66 dB compared with STA and CFW, respectively. In the experiments performed on a breast phantom, Hadamard-STA achieves better visualization of the breast inclusion with a clearer boundary between the inclusion and the background than STA and CFW. The in vivo experiments were performed on a patient with a breast tumor, and the tumor could also be better visualized with a more homogeneous background in the strain image obtained by Hadamard-STA than by STA and CFW. These results demonstrate that Hadamard-STA achieves a substantial improvement in lateral motion estimation and maybe a competitive method for quasi-static elastography.
Collapse
|
23
|
Ashikuzzaman M, Rivaz H. Second-Order Ultrasound Elastography With L1-Norm Spatial Regularization. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2022; 69:1008-1019. [PMID: 34995188 DOI: 10.1109/tuffc.2022.3141686] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Time delay estimation (TDE) between two radio-frequency (RF) frames is one of the major steps of quasi-static ultrasound elastography, which detects tissue pathology by estimating its mechanical properties. Regularized optimization-based techniques, a prominent class of TDE algorithms, optimize a nonlinear energy functional consisting of data constancy and spatial continuity constraints to obtain the displacement and strain maps between the time-series frames under consideration. The existing optimization-based TDE methods often consider the L2 -norm of displacement derivatives to construct the regularizer. However, such a formulation over-penalizes the displacement irregularity and poses two major issues to the estimated strain field. First, the boundaries between different tissues are blurred. Second, the visual contrast between the target and the background is suboptimal. To resolve these issues, herein, we propose a novel TDE algorithm where instead of L2 -, L1 -norms of both first- and second-order displacement derivatives are taken into account to devise the continuity functional. We handle the non-differentiability of L1 -norm by smoothing the absolute value function's sharp corner and optimize the resulting cost function in an iterative manner. We call our technique Second-Order Ultrasound eLastography (SOUL) with the L1 -norm spatial regularization ( L1 -SOUL). In terms of both sharpness and visual contrast, L1 -SOUL substantially outperforms GLobal Ultrasound Elastography (GLUE), tOtal Variation rEgulaRization and WINDow-based time delay estimation (OVERWIND), and SOUL, three recently published TDE algorithms in all validation experiments performed in this study. In cases of simulated, phantom, and in vivo datasets, respectively, L1 -SOUL achieves 67.8%, 46.81%, and 117.35% improvements of contrast-to-noise ratio (CNR) over SOUL. The L1 -SOUL code can be downloaded from http://code.sonography.ai.
Collapse
|
24
|
Mukaddim RA, Meshram NH, Weichmann AM, Mitchell CC, Varghese T. Spatiotemporal Bayesian Regularization for Cardiac Strain Imaging: Simulation and In Vivo Results. IEEE OPEN JOURNAL OF ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2022; 1:21-36. [PMID: 35174360 PMCID: PMC8846604 DOI: 10.1109/ojuffc.2021.3130021] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
Cardiac strain imaging (CSI) plays a critical role in the detection of myocardial motion abnormalities. Displacement estimation is an important processing step to ensure the accuracy and precision of derived strain tensors. In this paper, we propose and implement Spatiotemporal Bayesian regularization (STBR) algorithms for two-dimensional (2-D) normalized cross-correlation (NCC) based multi-level block matching along with incorporation into a Lagrangian cardiac strain estimation framework. Assuming smooth temporal variation over a short span of time, the proposed STBR algorithm performs displacement estimation using at least four consecutive ultrasound radio-frequency (RF) frames by iteratively regularizing 2-D NCC matrices using information from a local spatiotemporal neighborhood in a Bayesian sense. Two STBR schemes are proposed to construct Bayesian likelihood functions termed as Spatial then Temporal Bayesian (STBR-1) and simultaneous Spatiotemporal Bayesian (STBR-2). Radial and longitudinal strain estimated from a finite-element-analysis (FEA) model of realistic canine myocardial deformation were utilized to quantify strain bias, normalized strain error and total temporal relative error (TTR). Statistical analysis with one-way analysis of variance (ANOVA) showed that all Bayesian regularization methods significantly outperform NCC with lower bias and errors (p < 0.001). However, there was no significant difference among Bayesian methods. For example, mean longitudinal TTR for NCC, SBR, STBR-1 and STBR-2 were 25.41%, 9.27%, 10.38% and 10.13% respectively An in vivo feasibility study using RF data from ten healthy mice hearts were used to compare the elastographic signal-to-noise ratio (SNRe) calculated using stochastic analysis. STBR-2 had the highest expected SNRe both for radial and longitudinal strain. The mean expected SNRe values for accumulated radial strain for NCC, SBR, STBR-1 and STBR-2 were 5.03, 9.43, 9.42 and 10.58, respectively. Overall results suggest that STBR improves CSI in vivo.
Collapse
Affiliation(s)
- Rashid Al Mukaddim
- Department of Medical Physics, University of Wisconsin School of Medicine and Public Health, Madison, WI 53706 USA.,Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison, WI 53706 USA
| | - Nirvedh H Meshram
- Department of Medical Physics, University of Wisconsin School of Medicine and Public Health, Madison, WI 53706 USA.,Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison, WI 53706 USA
| | - Ashley M Weichmann
- Small Animal Imaging and Radiotherapy Facility, UW Carbone Cancer Center, Madison, WI 53705 USA
| | - Carol C Mitchell
- Department of Medicine/Division of Cardiovascular Medicine, University of Wisconsin School of Medicine and Public Health, Madison, WI 53792 USA
| | - Tomy Varghese
- Department of Medical Physics, University of Wisconsin School of Medicine and Public Health, Madison, WI 53706 USA.,Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison, WI 53706 USA
| |
Collapse
|
25
|
Yamamoto M, Yoshizawa S. Displacement detection with sub-pixel accuracy and high spatial resolution using deep learning. J Med Ultrason (2001) 2022; 49:3-15. [PMID: 34837159 DOI: 10.1007/s10396-021-01162-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2021] [Accepted: 08/25/2021] [Indexed: 01/25/2023]
Abstract
PURPOSE The purpose of this study was to detect two dimensional and sub-pixel displacement with high spatial resolution using an ultrasonic diagnostic apparatus. Conventional displacement detection methods assume neighborhood uniformity and cannot achieve both high spatial resolution and sub-pixel displacement detection. METHODS A deep-learning network that utilizes ultrasound images and output displacement distribution was developed. The network structure was constructed by modifying FlowNet2, a widely used network for optical flow estimation, and a training dataset was developed using ultrasound image simulation. Detection accuracy and spatial resolution were evaluated via simulated ultrasound images, and the clinical usefulness was evaluated with ultrasound images of the liver exposed to high-intensity-focused ultrasound (HIFU). These results were compared to the Lucas-Kanade method, a conventional sub-pixel displacement detection method. RESULTS For a displacement within ± 40 µm (± 0.6 pixels), a pixel size of 67 µm, and signal noise of 1%, the accuracy was above 0.5 µm and 0.2 µm, the precision was above 0.4 µm and 0.3 µm, and the spatial resolution was 1.1 mm and 0.8 mm for the lateral and axial displacements, respectively. These improvements were also observed in the experimental data. Visualization of the lateral displacement distribution, which determines the edge of the treated lesion using HIFU, was also realized. CONCLUSION Two-dimensional and sub-pixel displacement detection with high spatial resolution was realized using a deep-learning methodology. The proposed method enabled the monitoring of small and local tissue deformations induced by HIFU exposure.
Collapse
Affiliation(s)
- Mariko Yamamoto
- Graduate School of Engineering, Tohoku University, 6-6-05 Aoba, Aramaki, Aoba-ku, Sendai, 980-8579, Japan.
| | - Shin Yoshizawa
- Graduate School of Engineering, Tohoku University, 6-6-05 Aoba, Aramaki, Aoba-ku, Sendai, 980-8579, Japan
| |
Collapse
|
26
|
Bharadwaj S, Prasad S, Almekkawy M. An Upgraded Siamese Neural Network for Motion Tracking in Ultrasound Image Sequences. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2021; 68:3515-3527. [PMID: 34232873 DOI: 10.1109/tuffc.2021.3095299] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/28/2023]
Abstract
Deep learning is heavily being borrowed to solve problems in medical imaging applications, and Siamese neural networks are the front runners of motion tracking. In this article, we propose to upgrade one such Siamese architecture-based neural network for robust and accurate landmark tracking in ultrasound images to improve the quality of image-guided radiation therapy. Although several researchers have improved the Siamese architecture-based networks with sophisticated detection modules and by incorporating transfer learning, the inherent assumptions of the constant position model and the missing motion model remain unaddressed limitations. In our proposed model, we overcome these limitations by introducing two modules into the original architecture. We employ a reference template update to resolve the constant position model and a linear Kalman filter (LKF) to address the missing motion model. Moreover, we demonstrate that the proposed architecture provides promising results without transfer learning. The proposed model was submitted to an open challenge organized by MICCAI and was evaluated exhaustively on the Liver US Tracking (CLUST) 2D dataset. Experimental results proved that the proposed model tracked the landmarks with promising accuracy. Furthermore, we also induced synthetic occlusions to perform a qualitative analysis of the proposed approach. The evaluations were performed on the training set of the CLUST 2D dataset. The proposed method outperformed the original Siamese architecture by a significant margin.
Collapse
|
27
|
Xie Y, Liao H, Zhang D, Zhou L, Chen F. Image-Based 3D Ultrasound Reconstruction with Optical Flow via Pyramid Warping Network. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:3539-3542. [PMID: 34892003 DOI: 10.1109/embc46164.2021.9630853] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
3D Ultrasound (US) contains rich spatial information which is helpful for medical diagnosis. However, current reconstruction methods with tracking devices are not suitable for clinical application. The sensorless freehand methods reconstruct based on US images which is less accuracy. In this paper, we proposed a network which reconstructs the US volume based on US images features and optical flow features. We proposed the pyramid warping layer which merges the image features and optical flow features with warping operation. To fuse the warped features of different scales in different pyramid levels, we adopted the fusion module using the attention mechanism. Meanwhile, we adopted the channel attention and spatial attention to our network. Our method was evaluated in 100 freehand US sweeps of human forearms which exhibits the efficient performance on volume reconstruction compared with other methods.
Collapse
|
28
|
Delaunay R, Hu Y, Vercauteren T. An unsupervised learning approach to ultrasound strain elastography with spatio-temporal consistency. Phys Med Biol 2021; 66. [PMID: 34298531 PMCID: PMC8417818 DOI: 10.1088/1361-6560/ac176a] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2021] [Accepted: 07/23/2021] [Indexed: 12/19/2022]
Abstract
Quasi-static ultrasound elastography (USE) is an imaging modality that measures deformation (i.e. strain) of tissue in response to an applied mechanical force. In USE, the strain modulus is traditionally obtained by deriving the displacement field estimated between a pair of radio-frequency data. In this work we propose a recurrent network architecture with convolutional long-short-term memory decoder blocks to improve displacement estimation and spatio-temporal continuity between time series ultrasound frames. The network is trained in an unsupervised way, by optimising a similarity metric between the reference and compressed image. Our training loss is also composed of a regularisation term that preserves displacement continuity by directly optimising the strain smoothness, and a temporal continuity term that enforces consistency between successive strain predictions. In addition, we propose an open-access in vivo database for quasi-static USE, which consists of radio-frequency data sequences captured on the arm of a human volunteer. Our results from numerical simulation and in vivo data suggest that our recurrent neural network can account for larger deformations, as compared with two other feed-forward neural networks. In all experiments, our recurrent network outperformed the state-of-the-art for both learning-based and optimisation-based methods, in terms of elastographic signal-to-noise ratio, strain consistency, and image similarity. Finally, our open-source code provides a 3D-slicer visualisation module that can be used to process ultrasound RF frames in real-time, at a rate of up to 20 frames per second, using a standard GPU.
Collapse
Affiliation(s)
- Rémi Delaunay
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, Gower Street, London WC1E 6BT, United Kingdom.,School of Biomedical Engineering & Imaging Sciences, King's College London, Strand, London WC2R 2LS, United Kingdom
| | - Yipeng Hu
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, Gower Street, London WC1E 6BT, United Kingdom
| | - Tom Vercauteren
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, Gower Street, London WC1E 6BT, United Kingdom.,School of Biomedical Engineering & Imaging Sciences, King's College London, Strand, London WC2R 2LS, United Kingdom
| |
Collapse
|
29
|
Qu X, Yan G, Zheng D, Fan S, Rao Q, Jiang J. A Deep Learning-Based Automatic First-Arrival Picking Method for Ultrasound Sound-Speed Tomography. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2021; 68:2675-2686. [PMID: 33886467 DOI: 10.1109/tuffc.2021.3074983] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Ultrasound sound-speed tomography (USST) has shown great prospects for breast cancer diagnosis due to its advantages of nonradiation, low cost, 3-D breast images, and quantitative indicators. However, the reconstruction quality of USST is highly dependent on the first-arrival picking of the transmission wave. Traditional first-arrival picking methods have low accuracy and noise robustness. To improve the accuracy and robustness, we introduced a self-attention mechanism into the bidirectional long short-term memory (BLSTM) network and proposed the self-attention BLSTM (SAT-BLSTM) network. The proposed method predicts the probability of the first-arrival time and selects the time with maximum probability. A numerical simulation and prototype experiment were conducted. In the numerical simulation, the proposed SAT-BLSTM showed the best results. For signal-to-noise ratios (SNRs) of 50, 30, and 15 dB, the mean absolute errors (MAEs) were 48, 49, and 76 ns, respectively. The BLSTM had the second-best results, with MAEs of 55, 56, and 85 ns, respectively. The MAEs of the Akaike information criterion (AIC) method were 57, 296, and 489 ns, respectively. In the prototype experiment, the MAEs of the SAT-BLSTM, the BLSTM, and the AIC were 94, 111, and 410 ns, respectively.
Collapse
|
30
|
Tehrani AKZ, Amiri M, Rosado-Mendez IM, Hall TJ, Rivaz H. Ultrasound Scatterer Density Classification Using Convolutional Neural Networks and Patch Statistics. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2021; 68:2697-2706. [PMID: 33900913 DOI: 10.1109/tuffc.2021.3075912] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Quantitative ultrasound (QUS) can reveal crucial information on tissue properties, such as scatterer density. If the scatterer density per resolution cell is above or below 10, the tissue is considered as fully developed speckle (FDS) or underdeveloped speckle (UDS), respectively. Conventionally, the scatterer density has been classified using estimated statistical parameters of the amplitude of backscattered echoes. However, if the patch size is small, the estimation is not accurate. These parameters are also highly dependent on imaging settings. In this article, we adapt convolutional neural network (CNN) architectures for QUS and train them using simulation data. We further improve the network's performance by utilizing patch statistics as additional input channels. Inspired by deep supervision and multitask learning, we propose a second method to exploit patch statistics. We evaluate the networks using simulation data and experimental phantoms. We also compare our proposed methods with different classic and deep learning models and demonstrate their superior performance in the classification of tissues with different scatterer density values. The results also show that we are able to classify scatterer density in different imaging parameters with no need for a reference phantom. This work demonstrates the potential of CNNs in classifying scatterer density in ultrasound images.
Collapse
|
31
|
Ashikuzzaman M, Sadeghi-Naini A, Samani A, Rivaz H. Combining First- and Second-Order Continuity Constraints in Ultrasound Elastography. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2021; 68:2407-2418. [PMID: 33710956 DOI: 10.1109/tuffc.2021.3065884] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Ultrasound elastography is a prominent noninvasive medical imaging technique that estimates tissue elastic properties to detect abnormalities in an organ. A common approximation to tissue elastic modulus is tissue strain induced after mechanical stimulation. To compute tissue strain, ultrasound radio frequency (RF) data can be processed using energy-based algorithms. These algorithms suffer from ill-posedness to tackle. A continuity constraint along with the data amplitude similarity is imposed to obtain a unique solution to the time-delay estimation (TDE) problem. Existing energy-based methods exploit the first-order spatial derivative of the displacement field to construct a regularizer. This first-order regularization scheme alone is not fully consistent with the mechanics of tissue deformation while perturbed with an external force. As a consequence, state-of-the-art techniques suffer from two crucial drawbacks. First, the strain map is not sufficiently smooth in uniform tissue regions. Second, the edges of the hard or soft inclusions are not well-defined in the image. Herein, we address these issues by formulating a novel regularizer taking both first- and second-order derivatives of the displacement field into account. The second-order constraint, which is the principal novelty of this work, contributes both to background continuity and edge sharpness by suppressing spurious noisy edges and enhancing strong boundaries. We name the proposed technique: Second-Order Ultrasound eLastography (SOUL). Comparative assessment of qualitative and quantitative results shows that SOUL substantially outperforms three recently developed TDE algorithms called Hybrid, GLUE, and MPWC-Net++. SOUL yields 27.72%, 62.56%, and 81.37% improvements of the signal-to-noise ratio (SNR) and 72.35%, 54.03%, and 65.17% improvements of the contrast-to-noise ratio (CNR) over GLUE with data pertaining to simulation, phantom, and in vivo tissue, respectively. The SOUL code can be downloaded from code.sonography.ai.
Collapse
|
32
|
Chan DY, Morris DC, Polascik TJ, Palmeri ML, Nightingale KR. Deep Convolutional Neural Networks for Displacement Estimation in ARFI Imaging. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2021; 68:2472-2481. [PMID: 33760733 PMCID: PMC8363049 DOI: 10.1109/tuffc.2021.3068377] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/09/2023]
Abstract
Ultrasound elasticity imaging in soft tissue with acoustic radiation force requires the estimation of displacements, typically on the order of several microns, from serially acquired raw data A-lines. In this work, we implement a fully convolutional neural network (CNN) for ultrasound displacement estimation. We present a novel method for generating ultrasound training data, in which synthetic 3-D displacement volumes with a combination of randomly seeded ellipsoids are created and used to displace scatterers, from which simulated ultrasonic imaging is performed using Field II. Network performance was tested on these virtual displacement volumes, as well as an experimental ARFI phantom data set and a human in vivo prostate ARFI data set. In the simulated data, the proposed neural network performed comparably to Loupas's algorithm, a conventional phase-based displacement estimation algorithm; the rms error was [Formula: see text] for the CNN and 0.73 [Formula: see text] for Loupas. Similarly, in the phantom data, the contrast-to-noise ratio (CNR) of a stiff inclusion was 2.27 for the CNN-estimated image and 2.21 for the Loupas-estimated image. Applying the trained network to in vivo data enabled the visualization of prostate cancer and prostate anatomy. The proposed training method provided 26 000 training cases, which allowed robust network training. The CNN had a computation time that was comparable to Loupas's algorithm; further refinements to the network architecture may provide an improvement in the computation time. We conclude that deep neural network-based displacement estimation from ultrasonic data is feasible, providing comparable performance with respect to both accuracy and speed compared to current standard time-delay estimation approaches.
Collapse
|
33
|
Mirzaei M, Asif A, Rivaz H. Virtual Source Synthetic Aperture for Accurate Lateral Displacement Estimation in Ultrasound Elastography. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2021; 68:1687-1695. [PMID: 33351760 DOI: 10.1109/tuffc.2020.3046445] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Ultrasound elastography (USE) is an emerging noninvasive imaging technique in which pathological alterations can be visualized by revealing the mechanical properties of the tissue. Estimating tissue displacement in all directions is required to accurately estimate the mechanical properties. Despite capabilities of elastography techniques in estimating displacement in both axial and lateral directions, estimation of axial displacement is more accurate than lateral direction due to higher sampling frequency, higher resolution, and having a carrier signal propagating in the axial direction. Among different ultrasound imaging techniques, synthetic aperture (SA) has better lateral resolution than others, but it is not commonly used for USE due to its limitation in imaging depth of field. Virtual source synthetic aperture (VSSA) imaging is a technique to implement SA beamforming on the focused transmitted data to overcome the limitation of SA in depth of field while maintaining the same lateral resolution as SA. Besides lateral resolution, VSSA has the capability of increasing sampling frequency in the lateral direction without interpolation. In this article, we utilize VSSA to perform beamforming to enable higher resolution and sampling frequency in the lateral direction. The beamformed data are then processed using our recently published elastography technique, OVERWIND. Simulation and experimental results show substantial improvement in the estimation of lateral displacements.
Collapse
|
34
|
Zayed A, Rivaz H. Fast Strain Estimation and Frame Selection in Ultrasound Elastography Using Machine Learning. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2021; 68:406-415. [PMID: 32406831 DOI: 10.1109/tuffc.2020.2994028] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Ultrasound elastography aims to determine the mechanical properties of the tissue by monitoring tissue deformation due to internal or external forces. Tissue deformations are estimated from ultrasound radio frequency (RF) signals and are often referred to as time delay estimation (TDE). Given two RF frames I1 and I2 , we can compute a displacement image, which shows the change in the position of each sample in I1 to a new position in I2 . Two important challenges in TDE include high computational complexity and the difficulty in choosing suitable RF frames. Selecting suitable frames is of high importance because many pairs of RF frames either do not have acceptable deformation for extracting informative strain images or are decorrelated and deformation cannot be reliably estimated. Herein, we introduce a method that learns 12 displacement modes in quasi-static elastography by performing principal component analysis (PCA) on displacement fields of a large training database. In the inference stage, we use dynamic programming (DP) to compute an initial displacement estimate of around 1% of the samples and then decompose this sparse displacement into a linear combination of the 12 displacement modes. Our method assumes that the displacement of the whole image could also be described by this linear combination of principal components. We then use the GLobal Ultrasound Elastography (GLUE) method to fine-tune the result yielding the exact displacement image. Our method, which we call PCA-GLUE, is more than 10× faster than DP in calculating the initial displacement map while giving the same result. This is due to converting the problem of estimating millions of variables in DP into a much simpler problem of only 12 unknown weights of the principal components. Our second contribution in this article is determining the suitability of the frame pair I1 and I2 for strain estimation, which we achieve by using the weight vector that we calculated for PCA-GLUE as an input to a multilayer perceptron (MLP) classifier. We validate PCA-GLUE using simulation, phantom, and in vivo data. Our classifier takes only 1.5 ms during the testing phase and has an F1-measure of more than 92% when tested on 1430 instances collected from both phantom and in vivo data sets.
Collapse
|
35
|
Amiri M, Tehrani AKZ, Rivaz H. Segmentation of Ultrasound Images based on Scatterer Density using U-Net. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:2063-2066. [PMID: 33018411 DOI: 10.1109/embc44109.2020.9175353] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Quantitative ultrasound can provide an objective estimation of different tissue properties, which may be used for tissue characterization and detection of abnormal tissue. The effective number of scatterers in different parts of a tissue is one of the important tissue properties that can be estimated by quantitative ultrasound techniques. The envelope echo is the signal which is usually used to estimate the scatterer density. In this study, we proposed using deep learning to estimate the effective number of scatterers. We generated 2000 simulated phantom data containing randomly distributed inclusions with three different values for number of scatterers per resolution cell. We used U-Net to segment the envelope data and to distinguish three different values of scatterer densities. We show that U-Net can discriminate different scattering regimes, particularly, when the difference between the number of scatterers is substantial. The overall accuracy of the network is 83.9%, and the average sensitivity and specificity among the three classes are 83.1% and 92.3% respectively. This study confirms the potential of deep learning framework in quantitative ultrasound and estimation of tissue properties using ultrasound images.
Collapse
|
36
|
Tehrani AKZ, Amiri M, Rivaz H. Real-time and High Quality Ultrasound Elastography Using Convolutional Neural Network by Incorporating Analytic Signal. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:2075-2078. [PMID: 33018414 DOI: 10.1109/embc44109.2020.9176025] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Convolutional Neural Networks (CNN) have been extensively used for many computer vision applications including optical flow estimation. Although CNNs have been very successful in optical flow problem, they have been rarely used for displacement estimation in Ultrasound Elastography (USE) due to vast differences between ultrasound data and computer vision images. In USE, a main goal is to obtain the strain image which is the derivative of the axial displacement in axial direction; therefore, a very accurate displacement estimation is required. Radio Frequency (RF) data is needed to obtain accurate displacement estimation. RF data contains high frequency contents which cannot be downsampled without significant loss of information, in contrast to computer vision images. We propose a novel technique to utilize LiteFlowNet for USE. For the first time, we incorporate analytic signal to improve the quality of the displacement estimation. We show that this network with the designed inputs is more suitable for USE compared to more complex networks such as FlowNet2. The network is adopted to our application and it is compared with FlowNet2 and a state-of-the-art elastography method (GLUE). The results show that this network performs well and comparable to GLUE. Furthermore, not only this network is faster and has lower memory footprint compared to FlowNet2, but also it obtains higher quality strain images which makes it suitable for portable and real-time elastography devices.
Collapse
|
37
|
Mirzaei M, Asif A, Rivaz H. Synthetic aperture with high lateral sampling frequency for ultrasound elastography. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:2071-2074. [PMID: 33018413 DOI: 10.1109/embc44109.2020.9175426] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Ultrasound elastography is a non-invasive technique for detecting pathological alterations in tissue. It is known that pathological alteration of tissue often has a direct impact on its elastic modulus, which can be revealed using elastography. For estimating elastic modulus, we need to estimate both axial and lateral displacement accurately. Current state of the art elastography techniques provide a substantially less accurate lateral displacement field as compared to the axial displacement field. One of the most important factors in poor lateral estimation is a low sampling frequency in the lateral direction. In this paper, we use synthetic aperture beamforming to benefit from its capability of high sampling frequency in the lateral direction. We compare highly sampled data and focused line per line beam formed data by feeding them to our recently published elastography method, OVERWIND [1]. According to simulation and phantom experiments, not only the lateral displacement estimation is substantially improved, but also the axial displacement estimation is improved.
Collapse
|
38
|
Mirzaei M, Asif A, Rivaz H. Accurate and Precise Time-Delay Estimation for Ultrasound Elastography With Prebeamformed Channel Data. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2020; 67:1752-1763. [PMID: 32248101 DOI: 10.1109/tuffc.2020.2985060] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Free-hand palpation ultrasound elastography is a noninvasive approach for detecting pathological alteration in tissue. In this method, the tissue is compressed by a handheld probe and displacement of each sample is estimated, a process which is also known as time-delay estimation (TDE). Even with the simplifying assumption that ignores out of plane motion, TDE is an ill-posed problem requiring estimation of axial and lateral displacements for each sample from its intensity. A well-known class of methods for making elastography a well-posed problem is regularized optimization-based methods, which imposes smoothness regularization in the associated cost function. In this article, we propose to utilize channel data that have been compensated for time gain and time delay (introduced by transmission) instead of postbeamformed radio frequency (RF) data in the optimization problem. We name our proposed method Channel data for GLobal Ultrasound Elastography (CGLUE). We analytically derive bias and variances of TDE as functions of data noise for CGLUE and Global Ultrasound Elastography (GLUE) and use the Cauchy-Schwarz inequality to prove that CGLUE provides a TDE with lower bias and variance error. To further illustrate the improved performance of CGLUE, the results of simulation, experimental phantom, and ex-vivo experiments are presented.
Collapse
|