Data-Driven Elasticity Imaging Using Cartesian Neural Network Constitutive Models and the Autoprogressive Method

Elastic parameter identification of three-dimensional soft tissue based on deep neural network

In the field of virtual surgery and deformation simulation, the identification of elastic parameters of human soft tissues is a critical technology that directly affects the accuracy of deformation simulation. Current research on soft tissue deformation simulation predominantly assumes that the elasticity of tissues is fixed and already known, leading to the difficulty in populating with the elasticity measured or identified from specific tissues of real patients. Existing elasticity modeling efforts struggle to be implemented on irregularly structured soft tissues, failing to adapt to clinical surgical practices. Therefore, this paper proposes a new method for identifying human soft tissue elastic parameters based on the finite element method and the deep neural network, UNet. This method requires only the full-field displacement data of soft tissues under external loads to predict their elastic distribution. The performance and validity of the algorithm are assessed using test data and clinical data from rhinoplasty surgeries. Experiments demonstrate that the method proposed in this paper can achieve an accuracy of over 99% in predicting elastic parameters. Clinical data validation shows that the predicted elastic distribution can reduce the error in finite element deformation simulations by more than 80% at the maximum compared to the error with traditional uniform elastic parameters, effectively enhancing the computational accuracy in virtual surgery simulations and soft tissue deformation modeling.

DefCor-Net: Physics-aware ultrasound deformation correction

The recovery of morphologically accurate anatomical images from deformed ones is challenging in ultrasound (US) image acquisition, but crucial to accurate and consistent diagnosis, particularly in the emerging field of computer-assisted diagnosis. This article presents a novel physics-aware deformation correction approach based on a coarse-to-fine, multi-scale deep neural network (DefCor-Net). To achieve pixel-wise performance, DefCor-Net incorporates biomedical knowledge by estimating pixel-wise stiffness online using a U-shaped feature extractor. The deformation field is then computed using polynomial regression by integrating the measured force applied by the US probe. Based on real-time estimation of pixel-by-pixel tissue properties, the learning-based approach enables the potential for anatomy-aware deformation correction. To demonstrate the effectiveness of the proposed DefCor-Net, images recorded at multiple locations on forearms and upper arms of six volunteers are used to train and validate DefCor-Net. The results demonstrate that DefCor-Net can significantly improve the accuracy of deformation correction to recover the original geometry (Dice Coefficient: from 14.3±20.9 to 82.6±12.1 when the force is 6N). Code:https://github.com/KarolineZhy/DefCorNet.

Reconstructing the Spatial Distribution of the Relative Shear Modulus in Quasi-static Ultrasound Elastography: Plane Stress Analysis

Quasi-static ultrasound elastography (QSUE) is an imaging technique that mainly provides axial strain maps of tissues when the latter are subjected to compression. In this article, a method for reconstructing the relative shear modulus distribution within a linear elastic and isotropic medium, in QSUE, is introduced. More specifically, the plane stress inverse problem is considered. The proposed method is based on the variational formulation of the equilibrium equations and on the choice of adapted discretization spaces, and only requires displacement fields in the analyzed media to be determined. Results from plane stress and 3-D numerical simulations, as well as from phantom experiments, showed that the method is able to reconstruct the different regions within a medium, with shear modulus contrasts that unambiguously reveal whether inclusions are stiffer or softer than the surrounding material. More specifically, for the plane stress simulations, inclusion-to-background modulus ratios were found to be very accurately estimated, with an error lower than 3%. For the 3-D simulations, for which the plane stress conditions are no longer satisfied, these ratios were, as expected, less accurate, with an error that remained lower than 10% for two of the three cases analyzed but was around 34% for the last case. Concerning the phantom experiments, a comparison with a shear wave elastography technique from a clinical ultrasound scanner was also made. Overall, the inclusion-to-background shear modulus ratios obtained with our approach were found to be closer to those given by the phantom manufacturer than the ratios provided by the clinical system.

Fusing electrical and elasticity imaging

Electrical and elasticity imaging are promising modalities for a suite of different applications, including medical tomography, non-destructive testing and structural health monitoring. These emerging modalities are capable of providing remote, non-invasive and low-cost opportunities. Unfortunately, both modalities are severely ill-posed nonlinear inverse problems, susceptive to noise and modelling errors. Nevertheless, the ability to incorporate complimentary datasets obtained simultaneously offers mutually beneficial information. By fusing electrical and elastic modalities as a joint problem, we are afforded the possibility to stabilize the inversion process via the utilization of auxiliary information from both modalities as well as joint structural operators. In this study, we will discuss a possible approach to combine electrical and elasticity imaging in a joint reconstruction problem giving rise to novel multi-modality applications for use in both medical and structural engineering. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'.

Fast Strain Estimation and Frame Selection in Ultrasound Elastography Using Machine Learning

Ultrasound elastography aims to determine the mechanical properties of the tissue by monitoring tissue deformation due to internal or external forces. Tissue deformations are estimated from ultrasound radio frequency (RF) signals and are often referred to as time delay estimation (TDE). Given two RF frames I₁ and I₂ , we can compute a displacement image, which shows the change in the position of each sample in I₁ to a new position in I₂ . Two important challenges in TDE include high computational complexity and the difficulty in choosing suitable RF frames. Selecting suitable frames is of high importance because many pairs of RF frames either do not have acceptable deformation for extracting informative strain images or are decorrelated and deformation cannot be reliably estimated. Herein, we introduce a method that learns 12 displacement modes in quasi-static elastography by performing principal component analysis (PCA) on displacement fields of a large training database. In the inference stage, we use dynamic programming (DP) to compute an initial displacement estimate of around 1% of the samples and then decompose this sparse displacement into a linear combination of the 12 displacement modes. Our method assumes that the displacement of the whole image could also be described by this linear combination of principal components. We then use the GLobal Ultrasound Elastography (GLUE) method to fine-tune the result yielding the exact displacement image. Our method, which we call PCA-GLUE, is more than 10× faster than DP in calculating the initial displacement map while giving the same result. This is due to converting the problem of estimating millions of variables in DP into a much simpler problem of only 12 unknown weights of the principal components. Our second contribution in this article is determining the suitability of the frame pair I₁ and I₂ for strain estimation, which we achieve by using the weight vector that we calculated for PCA-GLUE as an input to a multilayer perceptron (MLP) classifier. We validate PCA-GLUE using simulation, phantom, and in vivo data. Our classifier takes only 1.5 ms during the testing phase and has an F1-measure of more than 92% when tested on 1430 instances collected from both phantom and in vivo data sets.

Shape-Driven EIT Reconstruction Using Fourier Representations

Shape-driven approaches have been proposed as an effective strategy for the electrical impedance tomography (EIT) reconstruction problem in recent years. In order to augment the shape-driven approaches, we propose a new method that transforms the shape to be reconstructed as basic primitives directly modeled by using Fourier representations. To allow automatic topological changes between the basic primitives and surrounding objects simultaneously, Boolean operations are employed. The Boolean operations with direct representation of primitives can be utilized for dimensionality and ill-posedness reduction, enabling feasible shape and topology optimization with shape-driven approaches. As a proof of principle, we leverage the proposed method for two dimensional shape reconstruction in EIT with various conductivity distributions. We demonstrate that our method is able to improve EIT reconstructions by enabling accurate shape and topology optimization.

Breast Tumor Diagnosis Using Finite-Element Modeling Based on Clinical in vivo Elastographic Data

OBJECTIVES

This study exploited finite-element modeling (FEM) to simulate breast tissue multicompression during ultrasound elastography to classify breast tumors based on their nonlinear biomechanical properties.

METHODS

Numeric simulations were first calculated by using 3-dimensional (3D) virtual models with an assumed tumor's geometric dimensions but with actual material properties to test and validate the FEM. Further numeric simulations were used to construct 3D models based on in vivo experimental data to verify our models. The models were designed for each individual in vivo case, emphasizing the geometry, position, and biomechanical properties of the breast tissue. At different compression levels, tissue strains were analyzed between the tumors and the background normal tissues to explore their nonlinearity and classify the tumor type. Tumor classification parameters were deduced by using a power-law relationship between the applied compressive forces and strain differences.

RESULTS

Classification parameters were compared between benign and malignant tumors, for which they were found to be statistically significant in classifying the tumor types (P < .05) by both the validation and verification of FEM. We compared the classification parameters between the in vivo and FEM classifications, for which they were found to be strongly correlated (R = 0.875; P < .001), with no statistical differences between their outcomes (P = .909).

CONCLUSIONS

Good agreement between the model outcomes and the in vivo diagnostics was reported. The implemented models were validated and verified. The introduced 3D modeling method may augment elastographic methods to preliminary classify breast tumors at an early stage.

Displacement Estimation in Ultrasound Elastography Using Pyramidal Convolutional Neural Network

In this article, two novel deep learning methods are proposed for displacement estimation in ultrasound elastography (USE). Although convolutional neural networks (CNNs) have been very successful for displacement estimation in computer vision, they have been rarely used for USE. One of the main limitations is that the radio frequency (RF) ultrasound data, which is crucial for precise displacement estimation, has vastly different frequency characteristics compared with images in computer vision. Top-rank CNN methods used in computer vision applications are mostly based on a multilevel strategy, which estimates finer resolution based on coarser ones. This strategy does not work well for RF data due to its large high-frequency content. To mitigate the problem, we propose modified pyramid warping and cost volume network (MPWC-Net) and RFMPWC-Net, both based on PWC-Net, to exploit information in RF data by employing two different strategies. We obtained promising results using networks trained only on computer vision images. In the next step, we constructed a large ultrasound simulation database and proposed a new loss function to fine-tune the network to improve its performance. The proposed networks and well-known optical flow networks as well as state-of-the-art elastography methods are evaluated using simulation, phantom, and in vivo data. Our two proposed networks substantially outperform current deep learning methods in terms of contrast-to-noise ratio (CNR) and strain ratio (SR). Also, the proposed methods perform similar to the state-of-the-art elastography methods in terms of CNR and have better SR by substantially reducing the underestimation bias.

A Pilot Study on Scatterer Density Classification of Ultrasound Images Using Deep Neural Networks

Quantitative ultrasound estimates different intrinsic tissue properties, which can be used for tissue characterization. Among different tissue properties, the effective number of scatterers per resolution cell is an important parameter, which can be estimated by the echo envelope. Assuming the signal is stationary and coherent, if the number of scatterers per resolution cell is above approximately 10, envelope signal is considered to be fully developed speckle (FDS) and otherwise they are from low scatterer number density (LSND). Two statistical parameters named R and S are often calculated from envelope intensity to classify FDS from LSND. The main problem is that limited data from small patches often renders this classification inaccurate. Herein, we propose two techniques based on neural networks to estimate the effective number of scatterers. The first network is a multi-layer perceptron (MLP) that uses the hand-crafted features of R and S for classification. The second network is a convolutional neural network (CNN) that does not need hand-crafted features and instead utilizes spectrum and the envelope intensity directly. We show that the proposed MLP works very well for large patches wherein a reliable estimation of R and S can be made. However, its classification becomes inaccurate for small patches, where the proposed CNN provides accurate classifications.

Physics-guided machine learning for 3-D quantitative quasi-static elasticity imaging

We present a 3D extension of the Autoprogressive Method (AutoP) for quantitative quasi-static ultrasonic elastography (QUSE) based on sparse sampling of force-displacement measurements. Compared to current model-based inverse methods, our approach requires neither geometric nor constitutive model assumptions. We build upon our previous report for 2D QUSE and demonstrate the feasibility of recovering the 3D linear-elastic material property distribution of gelatin phantoms under compressive loads. Measurements of boundary geometry, applied surface forces, and axial displacements enter into AutoP where a Cartesian neural network constitutive model (CaNNCM) interacts with finite element analyses to learn physically consistent material properties with no prior constitutive model assumption. We introduce a new regularization term uniquely suited to AutoP that improves the ability of CaNNCMs to extract information about spatial stress distributions from measurement data. Results of our study demonstrate that acquiring multiple sets of force-displacement measurements by moving the US probe to different locations on the phantom surface not only provides AutoP with the necessary information for a CaNNCM to learn the 3D material property distribution, but may significantly improve the accuracy of the Young's modulus estimates. Furthermore, we investigate the trade-offs of decreasing the contact area between the US transducer and phantom surface in an effort to increase sensitivity to surface force variations without additional instrumentation. Each of these modifications improves the ability of CaNNCMs trained in AutoP to learn the spatial distribution of Young's modulus from force-displacement measurements.

Data-Driven Elasticity Imaging Using Cartesian Neural Network Constitutive Models and the Autoprogressive Method

Quasi-static elasticity imaging techniques rely on model-based mathematical inverse methods to estimate mechanical parameters from force-displacement measurements. These techniques introduce simplifying assumptions that preclude exploration of unknown mechanical properties with potential diagnostic value. We previously reported a data-driven approach to elasticity imaging using artificial neural networks (NNs) that circumvents limitations associated with model-based inverse methods. NN constitutive models can learn stress-strain behavior from force-displacement measurements using the autoprogressive (AutoP) method without prior assumptions of the underlying constitutive model. However, information about internal structure was required. We invented Cartesian NN constitutive models (CaNNCMs) that learn the spatial variations of material properties. We are presenting the first implementation of CaNNCMs trained with AutoP to develop data-driven models of 2-D linear-elastic materials. Both simulated and experimental force-displacement data were used as input to AutoP to show that CaNNCMs are able to model both continuous and discrete material property distributions with no prior information of internal object structure. Furthermore, we demonstrate that CaNNCMs are robust to measurement noise and can reconstruct reasonably accurate Young's modulus images from a sparse sampling of measurement data. CaNNCMs are an important step toward clinical use of data-driven elasticity imaging using AutoP.