A Pipeline for the Generation of Realistic 3D Synthetic Echocardiographic Sequences: Methodology and Open-Access Database

Robust unsupervised texture segmentation for motion analysis in ultrasound images

PURPOSE

Ultrasound imaging has emerged as a promising cost-effective and portable non-irradiant modality for the diagnosis and follow-up of diseases. Motion analysis can be performed by segmenting anatomical structures of interest before tracking them over time. However, doing so in a robust way is challenging as ultrasound images often display a low contrast and blurry boundaries.

METHODS

In this paper, a robust descriptor inspired from the fractal dimension is presented to locally characterize the gray-level variations of an image. This descriptor is an adaptive grid pattern whose scale locally varies as the gray-level variations of the image. Robust features are then located based on the gray-level variations, which are more likely to be consistently tracked over time despite the presence of noise.

RESULTS

The method was validated on three datasets: segmentation of the left ventricle on simulated echocardiography (Dice coefficient, DC), accuracy of diaphragm motion tracking for healthy subjects (mean sum of distances, MSD) and for a scoliosis patient (root mean square error, RMSE). Results show that the method segments the left ventricle accurately ( DC = 0.84 ) and robustly tracks the diaphragm motion for healthy subjects ( MSD = 1.10 mm) and for the scoliosis patient ( RMSE = 1.22 mm).

CONCLUSIONS

This method has the potential to segment structures of interest according to their texture in an unsupervised fashion, as well as to help analyze the deformation of tissues. Possible applications are not limited to US image. The same principle could also be applied to other medical imaging modalities such as MRI or CT scans.

Continuous 3D Myocardial Motion Tracking via Echocardiography

Myocardial motion tracking stands as an essential clinical tool in the prevention and detection of cardiovascular diseases (CVDs), the foremost cause of death globally. However, current techniques suffer from incomplete and inaccurate motion estimation of the myocardium in both spatial and temporal dimensions, hindering the early identification of myocardial dysfunction. To address these challenges, this paper introduces the Neural Cardiac Motion Field (NeuralCMF). NeuralCMF leverages implicit neural representation (INR) to model the 3D structure and the comprehensive 6D forward/backward motion of the heart. This method surpasses pixel-wise limitations by offering the capability to continuously query the precise shape and motion of the myocardium at any specific point throughout the cardiac cycle, enhancing the detailed analysis of cardiac dynamics beyond traditional speckle tracking. Notably, NeuralCMF operates without the need for paired datasets, and its optimization is self-supervised through the physics knowledge priors in both space and time dimensions, ensuring compatibility with both 2D and 3D echocardiogram video inputs. Experimental validations across three representative datasets support the robustness and innovative nature of the NeuralCMF, marking significant advantages over existing state-of-the-art methods in cardiac imaging and motion tracking. Code is available at: https://njuvision.github.io/NeuralCMF.

Coordinate-Independent 3-D Ultrasound Principal Stretch and Direction Imaging

OBJECTIVE

In clinical ultrasound, current 2-D strain imaging faces challenges in quantifying three orthogonal normal strain components. This requires separate image acquisitions based on the pixel-dependent cardiac coordinate system, leading to additional computations and estimation discrepancies due to probe orientation. Most systems lack shear strain information, as displaying all components is challenging to interpret.

METHODS

This paper presents a 3-D high-spatial-resolution, coordinate-independent strain imaging approach based on principal stretch and axis estimation. All strain components are transformed into three principal stretches along three normal principal axes, enabling direct visualization of the primary deformation. We devised an efficient 3-D speckle tracking method with tilt filtering, incorporating randomized searching in a two-pass tracking framework and rotating the phase of the 3-D correlation function for robust filtering. The proposed speckle tracking approach significantly improves estimates of displacement gradients related to the axial displacement component. Non-axial displacement gradient estimates are enhanced using a correlation-weighted least-squares method constrained by tissue incompressibility.

RESULTS

Simulated and in vivo canine cardiac datasets were evaluated to estimate Lagrangian strains from end-diastole to end-systole. The proposed speckle tracking method improves displacement estimation by a factor of 4.3 to 10.5 over conventional 1-pass processing. Maximum principal axis/direction imaging enables better detection of local disease regions than conventional strain imaging.

CONCLUSION

Coordinate-independent tracking can identify myocardial abnormalities with high accuracy.

SIGNIFICANCE

This study offers enhanced accuracy and robustness in strain imaging compared to current methods.

Synthesizing Real-Time Ultrasound Images of Muscle Based on Biomechanical Simulation and Conditional Diffusion Network

Quantitative muscle function analysis based on ultrasound imaging, has been used for various applications, particularly with recent development of deep learning methods. The nature of speckle noises in ultrasound images poses challenges to accurate and reliable data annotation for supervised learning algorithms. To obtain a large and reliable dataset without manual scanning and labeling, we proposed a synthesizing pipeline to provide synthetic ultrasound datasets of muscle movement with an accurate ground truth, allowing augmenting, training, and evaluating models for different tasks. Our pipeline contained biomechanical simulation using a finite-element method (FEM), an algorithm for reconstructing sparse fascicles, and a diffusion network for ultrasound image generation. With the adjustment of a few parameters, the proposed pipeline can generate a large dataset of real-time ultrasound images with diversity in morphology and pattern. With 3030 ultrasound images generated, we qualitatively and quantitatively verified that the synthetic images closely matched with the in vivo images. In addition, we applied the synthetic dataset to different tasks of muscle analysis. Compared to trained on an unaugmented dataset, a model trained on synthetic one had better cross-dataset performance, which demonstrates the feasibility of synthesizing pipeline to augment model training and avoid overfitting. The results of the regression task show potentials under the conditions that the number of datasets or the accurate label is limited. The proposed synthesizing pipeline can not only be used for muscle-related study but also for other similar study and model development, where sequential images are needed for training.

Left Ventricular Segmentation, Warping, and Myocardial Registration for Automated Strain Measurement

The left ventricular global longitudinal strain (LVGLS) is a crucial prognostic indicator. However, inconsistencies in measurements due to the speckle tracking algorithm and manual adjustments have hindered its standardization and democratization. To solve this issue, we proposed a fully automated strain measurement by artificial intelligence-assisted LV segmentation contours. The LV segmentation model was trained from echocardiograms of 368 adults (11,125 frames). We compared the registration-like effects of dynamic time warping (DTW) with speckle tracking on a synthetic echocardiographic dataset in experiment-1. In experiment-2, we enrolled 80 patients to compare the DTW method with commercially available software. In experiment-3, we combined the segmentation model and DTW method to create the artificial intelligence (AI)-DTW method, which was then tested on 40 patients with general LV morphology, 20 with dilated cardiomyopathy (DCMP), and 20 with transthyretin-associated cardiac amyloidosis (ATTR-CA), 20 with severe aortic stenosis (AS), and 20 with severe mitral regurgitation (MR). Experiments-1 and -2 revealed that the DTW method is consistent with dedicated software. In experiment-3, the AI-DTW strain method showed comparable results for general LV morphology (bias - 0.137 ± 0.398%), DCMP (- 0.397 ± 0.607%), ATTR-CA (0.095 ± 0.581%), AS (0.334 ± 0.358%), and MR (0.237 ± 0.490%). Moreover, the strain curves showed a high correlation in their characteristics, with R-squared values of 0.8879-0.9452 for those LV morphology in experiment-3. Measuring LVGLS through dynamic warping of segmentation contour is a feasible method compared to traditional tracking techniques. This approach has the potential to decrease the need for manual demarcation and make LVGLS measurements more efficient and user-friendly for daily practice.

Cardiac ultrasound simulation for autonomous ultrasound navigation

Introduction

Ultrasound is well-established as an imaging modality for diagnostic and interventional purposes. However, the image quality varies with operator skills as acquiring and interpreting ultrasound images requires extensive training due to the imaging artefacts, the range of acquisition parameters and the variability of patient anatomies. Automating the image acquisition task could improve acquisition reproducibility and quality but training such an algorithm requires large amounts of navigation data, not saved in routine examinations.

Methods

We propose a method to generate large amounts of ultrasound images from other modalities and from arbitrary positions, such that this pipeline can later be used by learning algorithms for navigation. We present a novel simulation pipeline which uses segmentations from other modalities, an optimized volumetric data representation and GPU-accelerated Monte Carlo path tracing to generate view-dependent and patient-specific ultrasound images.

Results

We extensively validate the correctness of our pipeline with a phantom experiment, where structures' sizes, contrast and speckle noise properties are assessed. Furthermore, we demonstrate its usability to train neural networks for navigation in an echocardiography view classification experiment by generating synthetic images from more than 1,000 patients. Networks pre-trained with our simulations achieve significantly superior performance in settings where large real datasets are not available, especially for under-represented classes.

Discussion

The proposed approach allows for fast and accurate patient-specific ultrasound image generation, and its usability for training networks for navigation-related tasks is demonstrated.

Multi-Task Learning for Motion Analysis and Segmentation in 3D Echocardiography

Characterizing left ventricular deformation and strain using 3D+time echocardiography provides useful insights into cardiac function and can be used to detect and localize myocardial injury. To achieve this, it is imperative to obtain accurate motion estimates of the left ventricle. In many strain analysis pipelines, this step is often accompanied by a separate segmentation step; however, recent works have shown both tasks to be highly related and can be complementary when optimized jointly. In this work, we present a multi-task learning network that can simultaneously segment the left ventricle and track its motion between multiple time frames. Two task-specific networks are trained using a composite loss function. Cross-stitch units combine the activations of these networks by learning shared representations between the tasks at different levels. We also propose a novel shape-consistency unit that encourages motion propagated segmentations to match directly predicted segmentations. Using a combined synthetic and in-vivo 3D echocardiography dataset, we demonstrate that our proposed model can achieve excellent estimates of left ventricular motion displacement and myocardial segmentation. Additionally, we observe strong correlation of our image-based strain measurements with crystal-based strain measurements as well as good correspondence with SPECT perfusion mappings. Finally, we demonstrate the clinical utility of the segmentation masks in estimating ejection fraction and sphericity indices that correspond well with benchmark measurements.

Radiomics Harmonization in Ultrasound Images for Cervical Cancer Lymph Node Metastasis Prediction Using Cycle-GAN

Background: Ultrasound (US) based radiomics is susceptible to variations in scanners, sonographers. Objective: To retrospectively investigate the feasibility of an adapted cycle generative adversarial networks (CycleGAN) in the style transfer to improve US based radiomics in the prediction of lymph node metastasis (LNM) with images from multiple scanners for patients with early cervical cancer (ECC). Methods: The CycleGAN was firstly trained to transfer paired US phantom images from one US device to another one; the model was then further trained and tested with clinical US images of ECC by transferring images from four US devices to one specific device; finally, the adapted model was tested with its effects on the radiomics feature harmonization and accuracy of LNM prediction in US based radiomics for ECC patients. Results: Phantom study demonstrated an increased radiomics harmonization using CycleGAN with an average Pearson correlation coefficient of 0.60 and 0.81 for radiomics features extracted from original and generated images in correlation with the target phantom images, respectively. Additionally, the image quality metric Peak Signal-to-Noise Ratio (PSNR) was increased from 11.18 for the original images to 15.45 for the generated image. Clinical US images of 169 ECC patients were enrolled for style transfer model training and validation. The area under curve (AUC) of LNM prediction radiomics models with features extracted from generated images of different style transfer models ranged from 0.73 to 0.85. The AUC was improved from 0.78 with features extracted from original images to 0.85 with style transferred images. Conclusions: The adapted CycleGAN network is able to increase the radiomics feature harmonization for images from different ultrasound equipment based on image domain and improve the LNM prediction accuracy for ECC.

Ultrafast Cardiac Imaging Using Deep Learning for Speckle-Tracking Echocardiography

High-quality ultrafast ultrasound imaging is based on coherent compounding from multiple transmissions of plane waves (PW) or diverging waves (DW). However, compounding results in reduced frame rate, as well as destructive interferences from high-velocity tissue motion if motion compensation (MoCo) is not considered. While many studies have recently shown the interest of deep learning for the reconstruction of high-quality static images from PW or DW, its ability to achieve such performance while maintaining the capability of tracking cardiac motion has yet to be assessed. In this article, we addressed such issue by deploying a complex-weighted convolutional neural network (CNN) for image reconstruction and a state-of-the-art speckle-tracking method. The evaluation of this approach was first performed by designing an adapted simulation framework, which provides specific reference data, i.e., high-quality, motion artifact-free cardiac images. The obtained results showed that, while using only three DWs as input, the CNN-based approach yielded an image quality and a motion accuracy equivalent to those obtained by compounding 31 DWs free of motion artifacts. The performance was then further evaluated on nonsimulated, experimental in vitro data, using a spinning disk phantom. This experiment demonstrated that our approach yielded high-quality image reconstruction and motion estimation, under a large range of velocities and outperforms a state-of-the-art MoCo-based approach at high velocities. Our method was finally assessed on in vivo datasets and showed consistent improvement in image quality and motion estimation compared to standard compounding. This demonstrates the feasibility and effectiveness of deep learning reconstruction for ultrafast speckle-tracking echocardiography.

Fast strain mapping in abdominal aortic aneurysm wall reveals heterogeneous patterns

Abdominal aortic aneurysm patients are regularly monitored to assess aneurysm development and risk of rupture. A preventive surgical procedure is recommended when the maximum aortic antero-posterior diameter, periodically assessed on two-dimensional abdominal ultrasound scans, reaches 5.5 mm. Although the maximum diameter criterion has limited ability to predict aneurysm rupture, no clinically relevant tool that could complement the current guidelines has emerged so far. In vivo cyclic strains in the aneurysm wall are related to the wall response to blood pressure pulse, and therefore, they can be linked to wall mechanical properties, which in turn contribute to determining the risk of rupture. This work aimed to enable biomechanical estimations in the aneurysm wall by providing a fast and semi-automatic method to post-process dynamic clinical ultrasound sequences and by mapping the cross-sectional strains on the B-mode image. Specifically, the Sparse Demons algorithm was employed to track the wall motion throughout multiple cardiac cycles. Then, the cyclic strains were mapped by means of radial basis function interpolation and differentiation. We applied our method to two-dimensional sequences from eight patients. The automatic part of the analysis took under 1.5 min per cardiac cycle. The tracking method was validated against simulated ultrasound sequences, and a maximum root mean square error of 0.22 mm was found. The strain was calculated both with our method and with the established finite-element method, and a very good agreement was found, with mean differences of one order of magnitude smaller than the image spatial resolution. Most patients exhibited a strain pattern that suggests interaction with the spine. To conclude, our method is a promising tool for investigating abdominal aortic aneurysm wall biomechanics as it can provide a fast and accurate measurement of the cyclic wall strains from clinical ultrasound sequences.

Increasing frame rate of echocardiography based on a novel 2D spatio-temporal meshless interpolation

BACKGROUND

Increasing temporal resolution through numerical methods aids clinicians to evaluate fast moving structures of the heart with more confidence.

METHODOLOGY

In this study, a spatio-temporal numerical method is proposed to increase the frame rate based on two-dimensional (2D) interpolation. More specifically, we propose a novel intensity variation time surface (IVTS) strategy to incorporate both temporal and spatial information in the reconstruction. In this regard, we exploit radial basis functions (RBFs) for 2D interpolation. The reason for choosing RBFs for this task is manifold. First, RBFs are able to interpolate on large-scale datasets. Moreover, their mathematical implementation is simple. Another important property of this interpolation technique, which is addressed in this study, is its meshless nature. The meshless property enables higher up-sampling (UpS) rates for echocardiography to improve temporal resolution without noticeably degrading image quality. To evaluate the proposed approach, we tested the RBF interpolation on 2D/3D echocardiography datasets. The reconstructed frames were analyzed using different image quality metrics, and the results were compared with two popular techniques from the literature.

RESULTS

The findings demonstrated that, with a down-sampling rate of 3, the proposed technique outperformed the best existing method by 42%, 87%, 8%, and 11%, respectively, in terms of mean square error (MSE), contrast to noise ratio (CNR), peak signal-to-noise ratio (PSNR), and figure of merit (FOM). It should be noted that the proposed method is comparable to the best available method in terms of structural similarity (SSIM) index. Furthermore, when compared to the original images, the results of employing our technique on radio-frequency (RF) level analysis demonstrated that the reconstruction accuracy is satisfactory in terms of image quality criterion.

CONCLUSION

Finally, it is worthwhile noting that the proposed method is better than (or comparable to) the other methods in terms of reconstruction performance and processing time. Therefore, the RBF interpolation can be a promising alternative to the existing methods.

MRXCAT2.0: Synthesis of realistic numerical phantoms by combining left-ventricular shape learning, biophysical simulations and tissue texture generation

BACKGROUND

Standardised performance assessment of image acquisition, reconstruction and processing methods is limited by the absence of images paired with ground truth reference values. To this end, we propose MRXCAT2.0 to generate synthetic data, covering healthy and pathological function, using a biophysical model. We exemplify the approach by generating cardiovascular magnetic resonance (CMR) images of healthy, infarcted, dilated and hypertrophic left-ventricular (LV) function.

METHOD

In MRXCAT2.0, the XCAT torso phantom is coupled with a statistical shape model, describing population (patho)physiological variability, and a biophysical model, providing known and detailed functional ground truth of LV morphology and function. CMR balanced steady-state free precession images are generated using MRXCAT2.0 while realistic image appearance is ensured by assigning texturized tissue properties to the phantom labels.

FINDING

Paired CMR image and ground truth data of LV function were generated with a range of LV masses (85-140 g), ejection fractions (34-51%) and peak radial and circumferential strains (0.45 to 0.95 and - 0.18 to - 0.13, respectively). These ranges cover healthy and pathological cases, including infarction, dilated and hypertrophic cardiomyopathy. The generation of the anatomy takes a few seconds and it improves on current state-of-the-art models where the pathological representation is not explicitly addressed. For the full simulation framework, the biophysical models require approximately two hours, while image generation requires a few minutes per slice.

CONCLUSION

MRXCAT2.0 offers synthesis of realistic images embedding population-based anatomical and functional variability and associated ground truth parameters to facilitate a standardized assessment of CMR acquisition, reconstruction and processing methods.

Co-attention spatial transformer network for unsupervised motion tracking and cardiac strain analysis in 3D echocardiography

Myocardial ischemia/infarction causes wall-motion abnormalities in the left ventricle. Therefore, reliable motion estimation and strain analysis using 3D+time echocardiography for localization and characterization of myocardial injury is valuable for early detection and targeted interventions. Previous unsupervised cardiac motion tracking methods rely on heavily-weighted regularization functions to smooth out the noisy displacement fields in echocardiography. In this work, we present a Co-Attention Spatial Transformer Network (STN) for improved motion tracking and strain analysis in 3D echocardiography. Co-Attention STN aims to extract inter-frame dependent features between frames to improve the motion tracking in otherwise noisy 3D echocardiography images. We also propose a novel temporal constraint to further regularize the motion field to produce smooth and realistic cardiac displacement paths over time without prior assumptions on cardiac motion. Our experimental results on both synthetic and in vivo 3D echocardiography datasets demonstrate that our Co-Attention STN provides superior performance compared to existing methods. Strain analysis from Co-Attention STNs also correspond well with the matched SPECT perfusion maps, demonstrating the clinical utility for using 3D echocardiography for infarct localization.

MITEA: A dataset for machine learning segmentation of the left ventricle in 3D echocardiography using subject-specific labels from cardiac magnetic resonance imaging

Segmentation of the left ventricle (LV) in echocardiography is an important task for the quantification of volume and mass in heart disease. Continuing advances in echocardiography have extended imaging capabilities into the 3D domain, subsequently overcoming the geometric assumptions associated with conventional 2D acquisitions. Nevertheless, the analysis of 3D echocardiography (3DE) poses several challenges associated with limited spatial resolution, poor contrast-to-noise ratio, complex noise characteristics, and image anisotropy. To develop automated methods for 3DE analysis, a sufficiently large, labeled dataset is typically required. However, ground truth segmentations have historically been difficult to obtain due to the high inter-observer variability associated with manual analysis. We address this lack of expert consensus by registering labels derived from higher-resolution subject-specific cardiac magnetic resonance (CMR) images, producing 536 annotated 3DE images from 143 human subjects (10 of which were excluded). This heterogeneous population consists of healthy controls and patients with cardiac disease, across a range of demographics. To demonstrate the utility of such a dataset, a state-of-the-art, self-configuring deep learning network for semantic segmentation was employed for automated 3DE analysis. Using the proposed dataset for training, the network produced measurement biases of -9 ± 16 ml, -1 ± 10 ml, -2 ± 5 %, and 5 ± 23 g, for end-diastolic volume, end-systolic volume, ejection fraction, and mass, respectively, outperforming an expert human observer in terms of accuracy as well as scan-rescan reproducibility. As part of the Cardiac Atlas Project, we present here a large, publicly available 3DE dataset with ground truth labels that leverage the higher resolution and contrast of CMR, to provide a new benchmark for automated 3DE analysis. Such an approach not only reduces the effect of observer-specific bias present in manual 3DE annotations, but also enables the development of analysis techniques which exhibit better agreement with CMR compared to conventional methods. This represents an important step for enabling more efficient and accurate diagnostic and prognostic information to be obtained from echocardiography.

Motion Estimation by Deep Learning in 2D Echocardiography: Synthetic Dataset and Validation

Motion estimation in echocardiography plays an important role in the characterization of cardiac function, allowing the computation of myocardial deformation indices. However, there exist limitations in clinical practice, particularly with regard to the accuracy and robustness of measurements extracted from images. We therefore propose a novel deep learning solution for motion estimation in echocardiography. Our network corresponds to a modified version of PWC-Net which achieves high performance on ultrasound sequences. In parallel, we designed a novel simulation pipeline allowing the generation of a large amount of realistic B-mode sequences. These synthetic data, together with strategies during training and inference, were used to improve the performance of our deep learning solution, which achieved an average endpoint error of 0.07 ± 0.06 mm per frame and 1.20 ± 0.67 mm between ED and ES on our simulated dataset. The performance of our method was further investigated on 30 patients from a publicly available clinical dataset acquired from a GE system. The method showed promise by achieving a mean absolute error of the global longitudinal strain of 2.5 ± 2.1% and a correlation of 0.77 compared to GLS derived from manual segmentation, much better than one of the most efficient methods in the state-of-the-art (namely the FFT-Xcorr block-matching method). We finally evaluated our method on an auxiliary dataset including 30 patients from another center and acquired with a different system. Comparable results were achieved, illustrating the ability of our method to maintain high performance regardless of the echocardiographic data processed.

A Pipeline for the Generation of Synthetic Cardiac Color Doppler

Color Doppler imaging (CDI) is the modality of choice for simultaneous visualization of myocardium and intracavitary flow over a wide scan area. This visualization modality is subject to several sources of error, the main ones being aliasing and clutter. Mitigation of these artifacts is a major concern for better analysis of intracardiac flow. One option to address these issues is through simulations. In this article, we present a numerical framework for generating clinical-like CDI. Synthetic blood vector fields were obtained from a patient-specific computational fluid dynamics CFD model. Realistic texture and clutter artifacts were simulated from real clinical ultrasound cineloops. We simulated several scenarios highlighting the effects of 1) flow acceleration; 2) wall clutter; and 3) transmit wavefronts, on Doppler velocities. As a comparison, an "ideal" color Doppler was also simulated, without these harmful effects. This synthetic dataset is made publicly available and can be used to evaluate the quality of Doppler estimation techniques. Besides, this approach can be seen as a first step toward the generation of comprehensive datasets for training neural networks to improve the quality of Doppler imaging.

Assessing mechanical vibration-altered wall shear stress in digital arteries

The aim of this study is to implement and validate a method for assessing acute vibration-altered Wall Shear Stress (WSS) in the proper volar digital artery of the non-exposed left forefinger when subjecting the right hand to mechanical vibration. These changes of WSS may be involved in Vibration White Finger. Hence, an experimental device was set-up to link a vibration shaker and an ultra-high frequency ultrasound scanner. The Womersley-based WSS was computed by picking up the maximum velocity from pulse Wave Doppler measurements and extracting the artery diameter from B-mode images through an in-house image processing technique. The parameters of the former method were optimised on numerical ultrasound phantoms of cylindrical and lifelike arteries. These phantoms were computed with the FIELD II and FOCUS platforms which mimicked our true ultrasound device. The Womersley-based WSS were compared to full Fluid Structure Interaction (FSI) and rigid wall models built from resonance magnetic images of a volunteer-specific forefinger artery. Our FSI model took into account the artery's surrounding tissues. The diameter computing procedure led to a bias of 4%. The Womersley-based WSS resulted in misestimating the FSI model by roughly 10% to 20%. No difference was found between the rigid wall computational model and FSI simulations. Regarding the WSS measured on a group of 20 volunteers, the group-averaged basal value was 3 Pa, while the vibration-altered WSS was reduced to 1 Pa, possibly triggering intimal hyperplasia mechanisms and leading to the arterial stenoses encountered in patients suffering from vibration-induced Raynaud's syndrome.

Generating Synthetic Labeled Data From Existing Anatomical Models: An Example With Echocardiography Segmentation

Deep learning can bring time savings and increased reproducibility to medical image analysis. However, acquiring training data is challenging due to the time-intensive nature of labeling and high inter-observer variability in annotations. Rather than labeling images, in this work we propose an alternative pipeline where images are generated from existing high-quality annotations using generative adversarial networks (GANs). Annotations are derived automatically from previously built anatomical models and are transformed into realistic synthetic ultrasound images with paired labels using a CycleGAN. We demonstrate the pipeline by generating synthetic 2D echocardiography images to compare with existing deep learning ultrasound segmentation datasets. A convolutional neural network is trained to segment the left ventricle and left atrium using only synthetic images. Networks trained with synthetic images were extensively tested on four different unseen datasets of real images with median Dice scores of 91, 90, 88, and 87 for left ventricle segmentation. These results match or are better than inter-observer results measured on real ultrasound datasets and are comparable to a network trained on a separate set of real images. Results demonstrate the images produced can effectively be used in place of real data for training. The proposed pipeline opens the door for automatic generation of training data for many tasks in medical imaging as the same process can be applied to other segmentation or landmark detection tasks in any modality. The source code and anatomical models are available to other researchers.1 1https://adgilbert.github.io/data-generation/.

Learning-Based Regularization for Cardiac Strain Analysis via Domain Adaptation

Reliable motion estimation and strain analysis using 3D+ time echocardiography (4DE) for localization and characterization of myocardial injury is valuable for early detection and targeted interventions. However, motion estimation is difficult due to the low-SNR that stems from the inherent image properties of 4DE, and intelligent regularization is critical for producing reliable motion estimates. In this work, we incorporated the notion of domain adaptation into a supervised neural network regularization framework. We first propose a semi-supervised Multi-Layered Perceptron (MLP) network with biomechanical constraints for learning a latent representation that is shown to have more physiologically plausible displacements. We extended this framework to include a supervised loss term on synthetic data and showed the effects of biomechanical constraints on the network's ability for domain adaptation. We validated the semi-supervised regularization method on in vivo data with implanted sonomicrometers. Finally, we showed the ability of our semi-supervised learning regularization approach to identify infarct regions using estimated regional strain maps with good agreement to manually traced infarct regions from postmortem excised hearts.

Main Uncertainties in the RF Ultrasound Scanning Simulation of the Standard Ultrasound Phantoms

Ultrasound echoscopy technologies are continuously evolving towards new modalities including quantitative parameter imaging, elastography, 3D scanning, and others. The development and analysis of new methods and algorithms require an adequate digital simulation of radiofrequency (RF) signal transformations. The purpose of this paper is the quantitative evaluation of RF signal simulation uncertainties in resolution and contrast reproduction with the model of a phased array transducer. The method is based on three types of standard physical phantoms. Digital 3D models of those phantoms are composed of point scatterers representing the weak backscattering of the background material and stronger backscattering from inclusions. The simulation results of echoscopy with sector scanning transducer by Field II software are compared with the RF output of the Ultrasonix scanner after scanning standard phantoms with 2.5 MHz phased array. The quantitative comparison of axial, lateral, and elevation resolutions have shown uncertainties from 9 to 22% correspondingly. The echoscopy simulation with two densities of scatterers is compared with contrast phantom imaging on the backscattered RF signals and B-scan reconstructed image, showing that the main sources of uncertainties limiting the echoscopy RF signal simulation adequacy are an insufficient knowledge of the scanner and phantom’s parameters. The attempt made for the quantitative evaluation of simulation uncertainties shows both problems and the potential of echoscopy simulation in imaging technology developments. The analysis presented could be interesting for researchers developing quantitative ultrasound imaging and elastography technologies looking for simulated raw RF signals comparable to those obtained from real ultrasonic scanning.

Non-rigid image registration using a modified fuzzy feature-based inference system for 3D cardiac motion estimation

BACKGROUND AND OBJECTIVE

Non-rigid image registration is a well-established method for estimating cardiac motion on 3D echocardiographic images. However, such images have relatively poor spatio-temporal resolution making registration challenging. Some of the main challenges are extracting features relevant to the registration problem and defining a suitable geometrical transformation to be applied. The latter can be tackled using a fuzzy inference system considering its potential in transformation modeling. From this point of view, feature-based image registration can be considered an identification problem in which the transformation parameters are computed through an optimization process. This study, thus, aims to estimate cardiac motion on 3D echocardiographic images based on feature-based non-rigid image registration through sets of modified fuzzy rules.

METHODS

The 3D volume features were extracted with the popular scale-invariant feature transform (SIFT) descriptors in 3D space. Sets of fuzzy rules were generated according to the extracted features to register every two consecutive frames. Finally, some supplementary rules modified the registration rule for estimating cardiac motion.

RESULTS

Applying the fuzzy feature-based inference system on the STRAUS synthetic database showed the proposed method to be competitive with other well-established registration algorithms in terms of tracking error and accuracy of strain estimates. The proposed algorithm yielded a tracking error of 1 mm and a relative circumferential strain error of 0.82±4.69%. In addition, the potential of the proposed algorithm for clinical applications was confirmed by evaluating its performance on an in-vivo database called CETUS.

CONCLUSION

This paper proposes a novel registration method based on fuzzy logic which was shown to enable tracking complex cardiac deformations in 3D echocardiographic images with high accuracy.

Direct 3D ultrasound fusion for transesophageal echocardiography

BACKGROUND

Real-time three-dimensional transesophageal echocardiography (3D TEE) has been increasingly used in clinic for fast 3D analysis of cardiac anatomy and function. However, 3D TEE still suffers from the limited field of view (FoV). It is challenging to adopt conventional multi-view fusion methods to 3D TEE images because feature-based registration methods tend to fail in the ultrasound scenario, and conventional intensity-based methods have poor convergence properties and require an iterative coarse-to-fine strategy.

METHODS

A novel multi-view registration and fusion method is proposed to enlarge the FoV of 3D TEE images efficiently. A direct method is proposed to solve the registration problem in the Lie algebra space. Fast implementation is realized by searching voxels on three orthogonal planes between two volumes. Besides, a weighted-average 3D fusion method is proposed to fuse the aligned images seamlessly. For a sequence of 3D TEE images, they are fused incrementally.

RESULTS

Qualitative and quantitative results of in-vivo experiments indicate that the proposed registration algorithm outperforms a state-of-the-art PCA-based registration method in terms of accuracy and efficiency. Image registration and fusion performed on 76 in-vivo 3D TEE volumes from nine patients show apparent enlargement of FoV (enlarged around two times) in the obtained fused images.

CONCLUSIONS

The proposed methods can fuse 3D TEE images efficiently and accurately so that the whole Region of Interest (ROI) can be seen in a single frame. This research shows good potential to assist clinical diagnosis, preoperative planning, and future intraoperative guidance with 3D TEE.

Estimating Arterial Wall Deformations from Automatic Key-Point Detection and Matching

Assessing arterial-wall motion and deformations may reveal pathologic alterations in biomechanical properties of the parietal tissues and, thus, contribute to the detection of vascular disease onset. Ultrasound image sequences allow the observation of this motion and many methods have been developed to estimate temporal changes in artery diameter and wall thickness and to track 2-D displacements of selected points. Some methods enable the assessment of shearing or stretching within the wall, but none of them can estimate all these deformations simultaneously. The method herein proposed was devised to simultaneously estimate translation, compression, stretching and shearing of the arterial wall in ultrasound B-mode image sequences representing the carotid artery longitudinal section. Salient blob-like patterns, called key points, are automatically detected in each frame and matched between successive frames. A robust estimator based on an affine transformation model is then used to assess frame-to-frame motion explaining at best the key-point matches and to reject outliers. Realistic simulated image sequences were used to evaluate the accuracy and robustness of the method against ground truth. The method was also visually assessed on clinical image sequences, for which true deformations are unknown.

Deep Network for Scatterer Distribution Estimation for Ultrasound Image Simulation

Simulation-based ultrasound (US) training can be an essential educational tool. Realistic US image appearance with typical speckle texture can be modeled as convolution of a point spread function with point scatterers representing tissue microstructure. Such scatterer distribution, however, is in general not known and its estimation for a given tissue type is fundamentally an ill-posed inverse problem. In this article, we demonstrate a convolutional neural network approach for probabilistic scatterer estimation from observed US data. We herein propose to impose a known statistical distribution on scatterers and learn the mapping between US image and distribution parameter map by training a convolutional neural network on synthetic images. In comparison with several existing approaches, we demonstrate in numerical simulations and with in vivo images that the synthesized images from scatterer representations estimated with our approach closely match the observations with varying acquisition parameters such as compression and rotation of the imaged domain.

Myocardial strain imaging: review of general principles, validation, and sources of discrepancies

Myocardial tissue tracking imaging techniques have been developed for a more accurate evaluation of myocardial deformation (i.e. strain), with the potential to overcome the limitations of ejection fraction (EF) and to contribute, incremental to EF, to the diagnosis and prognosis in cardiac diseases. While most of the deformation imaging techniques are based on the similar principles of detecting and tracking specific patterns within an image, there are intra- and inter-imaging modality inconsistencies limiting the wide clinical applicability of strain. In this review, we aimed to describe the particularities of the echocardiographic and cardiac magnetic resonance deformation techniques, in order to understand the discrepancies in strain measurement, focusing on the potential sources of variation: related to the software used to analyse the data, to the different physics of image acquisition and the different principles of 2D vs. 3D approaches. As strain measurements are not interchangeable, it is highly desirable to work with validated strain assessment tools, in order to derive information from evidence-based data. There is, however, a lack of solid validation of the current tissue tracking techniques, as only a few of the commercial deformation imaging softwares have been properly investigated. We have, therefore, addressed in this review the neglected issue of suboptimal validation of tissue tracking techniques, in order to advocate for this matter.

A Semi-supervised Joint Network for Simultaneous Left Ventricular Motion Tracking and Segmentation in 4D Echocardiography

This work presents a novel deep learning method to combine segmentation and motion tracking in 4D echocardiography. The network iteratively trains a motion branch and a segmentation branch. The motion branch is initially trained entirely unsupervised and learns to roughly map the displacements between a source and a target frame. The estimated displacement maps are then used to generate pseudo-ground truth labels to train the segmentation branch. The labels predicted by the trained segmentation branch are fed back into the motion branch and act as landmarks to help retrain the branch to produce smoother displacement estimations. These smoothed out displacements are then used to obtain smoother pseudo-labels to retrain the segmentation branch. Additionally, a biomechanically-inspired incompressibility constraint is implemented in order to encourage more realistic cardiac motion. The proposed method is evaluated against other approaches using synthetic and in-vivo canine studies. Both the segmentation and motion tracking results of our model perform favorably against competing methods.

A Novel 2-D Speckle Tracking Method for High-Frame-Rate Echocardiography

Speckle tracking echocardiography (STE) is a clinical tool to noninvasively assess regional myocardial function through the quantification of regional motion and deformation. Even if the time resolution of STE can be improved by high-frame-rate (HFR) imaging, dedicated HFR STE algorithms have to be developed to detect very small interframe motions. Therefore, in this article, we propose a novel 2-D STE method, purposely developed for HFR echocardiography. The 2-D motion estimator consists of a two-step algorithm based on the 1-D cross correlations to separately estimate the axial and lateral displacements. The method was first optimized and validated on simulated data giving an accuracy of ~3.3% and ~10.5% for the axial and lateral estimates, respectively. Then, it was preliminarily tested in vivo on ten healthy volunteers showing its clinical applicability and feasibility. Moreover, the extracted clinical markers were in the same range as those reported in the literature. Also, the estimated peak global longitudinal strain was compared with that measured with a clinical scanner showing good correlation and negligible differences (-20.94% versus -20.31%, p -value = 0.44). In conclusion, a novel algorithm for STE was developed: the radio frequency (RF) signals were preferred for the axial motion estimation, while envelope data were preferred for the lateral motion. Furthermore, using 2-D kernels, even for 1-D cross correlation, makes the method less sensitive to noise.

A SEMI-SUPERVISED JOINT LEARNING APPROACH TO LEFT VENTRICULAR SEGMENTATION AND MOTION TRACKING IN ECHOCARDIOGRAPHY

Accurate interpretation and analysis of echocardiography is important in assessing cardiovascular health. However, motion tracking often relies on accurate segmentation of the myocardium, which can be difficult to obtain due to inherent ultrasound properties. In order to address this limitation, we propose a semi-supervised joint learning network that exploits overlapping features in motion tracking and segmentation. The network simultaneously trains two branches: one for motion tracking and one for segmentation. Each branch learns to extract features relevant to their respective tasks and shares them with the other. Learned motion estimations propagate a manually segmented mask through time, which is used to guide future segmentation predictions. Physiological constraints are introduced to enforce realistic cardiac behavior. Experimental results on synthetic and in vivo canine 2D+t echocardiographic sequences outperform some competing methods in both tasks.

Interactive Echocardiography Translation Using Few-Shot GAN Transfer Learning

Background

Interactive echocardiography translation is an efficient educational function to master cardiac anatomy. It strengthens the student's understanding by pixel-level translation between echocardiography and theoretically sketch images. Previous research studies split it into two aspects of image segmentation and synthesis. This split makes it hard to achieve pixel-level corresponding translation. Besides, it is also challenging to leverage deep-learning-based methods in each phase where a handful of annotations are available.

Methods

To address interactive translation with limited annotations, we present a two-step transfer learning approach. Firstly, we train two independent parent networks, the ultrasound to sketch (U2S) parent network and the sketch to ultrasound (S2U) parent network. U2S translation is similar to a segmentation task with sector boundary inference. Therefore, the U2S parent network is trained with the U-Net network on the public segmentation dataset of VOC2012. S2U aims at recovering ultrasound texture. So, the S2U parent network is decoder networks that generate ultrasound data from random input. After pretraining the parent networks, an encoder network is attached to the S2U parent network to translate ultrasound images into sketch images. We jointly transfer learning U2S and S2U within the CGAN framework. Results and conclusion. Quantitative and qualitative contrast from 1-shot, 5-shot, and 10-shot transfer learning show the effectiveness of the proposed algorithm. The interactive translation is achieved with few-shot transfer learning. Thus, the development of new applications from scratch is accelerated. Our few-shot transfer learning has great potential in the biomedical computer-aided image translation field, where annotation data are extremely precious.

Spatio-Temporal Segmentation in 3-D Echocardiographic Sequences Using Fractional Brownian Motion

An important aspect for an improved cardiac functional analysis is the accurate segmentation of the left ventricle (LV). A novel approach for fully-automated segmentation of the LV endocardium and epicardium contours is presented. This is mainly based on the natural physical characteristics of the LV shape structure. Both sides of the LV boundaries exhibit natural elliptical curvatures by having details on various scales, i.e. exhibiting fractal-like characteristics. The fractional Brownian motion (fBm), which is a non-stationary stochastic process, integrates well with the stochastic nature of ultrasound echoes. It has the advantage of representing a wide range of non-stationary signals and can quantify statistical local self-similarity throughout the time-sequence ultrasound images. The locally characterized boundaries of the fBm segmented LV were further iteratively refined using global information by means of second-order moments. The method is benchmarked using synthetic 3D+time echocardiographic sequences for normal and different ischemic cardiomyopathy, and results compared with state-of-the-art LV segmentation. Furthermore, the framework was validated against real data from canine cases with expert-defined segmentations and demonstrated improved accuracy. The fBm-based segmentation algorithm is fully automatic and has the potential to be used clinically together with 3D echocardiography for improved cardiovascular disease diagnosis.

Flow network tracking for spatiotemporal and periodic point matching: Applied to cardiac motion analysis

The accurate quantification of left ventricular (LV) deformation/strain shows significant promise for quantitatively assessing cardiac function for use in diagnosis and therapy planning. However, accurate estimation of the displacement of myocardial tissue and hence LV strain has been challenging due to a variety of issues, including those related to deriving tracking tokens from images and following tissue locations over the entire cardiac cycle. In this work, we propose a point matching scheme where correspondences are modeled as flow through a graphical network. Myocardial surface points are set up as nodes in the network and edges define neighborhood relationships temporally. The novelty lies in the constraints that are imposed on the matching scheme, which render the correspondences one-to-one through the entire cardiac cycle, and not just two consecutive frames. The constraints also encourage motion to be cyclic, which an important characteristic of LV motion. We validate our method by applying it to the estimation of quantitative LV displacement and strain estimation using 8 synthetic and 8 open-chested canine 4D echocardiographic image sequences, the latter with sonomicrometric crystals implanted on the LV wall. We were able to achieve excellent tracking accuracy on the synthetic dataset and observed a good correlation with crystal-based strains on the in-vivo data.

Generalized Bayesian Model Selection for Speckle on Remote Sensing Images

Synthetic aperture radar (SAR) and ultrasound (US) are two important active imaging techniques for remote sensing, both of which are subject to speckle noise caused by coherent summation of back-scattered waves and subsequent nonlinear envelope transformations. Estimating the characteristics of this multiplicative noise is crucial to develop denoising methods and to improve statistical inference from remote sensing images. In this paper, reversible jump Markov chain Monte Carlo (RJMCMC) algorithm has been used with a wider interpretation and a recently proposed RJMCMC-based Bayesian approach, trans-space RJMCMC, has been utilized. The proposed method provides an automatic model class selection mechanism for remote sensing images of SAR and US where the model class space consists of popular envelope distribution families. The proposed method estimates the correct distribution family, as well as the shape and the scale parameters, avoiding performing an exhaustive search. For the experimental analysis, different SAR images of urban, forest and agricultural scenes, and two different US images of a human heart have been used. Simulation results show the efficiency of the proposed method in finding statistical models for speckle.

Guideline-based learning for standard plane extraction in 3-D echocardiography

The extraction of six standard planes in 3-D cardiac ultrasound plays an important role in clinical examination to analyze cardiac function. A guideline-based learning method for efficient and accurate standard plane extraction is proposed. A cardiac ultrasound guideline determines appropriate operation steps for clinical examinations. The idea of guideline-based learning is incorporating machine learning approaches into each stage of the guideline. First, Hough forest with hierarchical search is applied for 3-D feature point detection. Second, initial planes are determined using anatomical regularities according to the guideline. Finally, a regression forest integrated with constraints of plane regularities is applied for refining each plane. The proposed method was evaluated on a 3-D cardiac ultrasound dataset and a synthetic dataset. Compared with other plane extraction methods, it demonstrated an improved accuracy with a significantly faster running time of 0.8    s / volume . Furthermore, it showed the proposed method was robust for a range abnormalities and image qualities, which would be seen in clinical practice.

Robust Optical Flow Estimation in Cardiac Ultrasound Images Using a Sparse Representation

This paper introduces a robust 2-D cardiac motion estimation method. The problem is formulated as an energy minimization with an optical flow-based data fidelity term and two regularization terms imposing spatial smoothness and the sparsity of the motion field in an appropriate cardiac motion dictionary. Robustness to outliers, such as imaging artefacts and anatomical motion boundaries, is introduced using robust weighting functions for the data fidelity term as well as for the spatial and sparse regularizations. The motion fields and the weights are computed jointly using an iteratively re-weighted minimization strategy. The proposed robust approach is evaluated on synthetic data and realistic simulation sequences with available ground-truth by comparing the performance with state-of-the-art algorithms. Finally, the proposed method is validated using two sequences of in vivo images. The obtained results show the interest of the proposed approach for 2-D cardiac ultrasound imaging.

Quantitative Assessment of the Effect of the Out-of-Plane Movement of the Homogenous Ellipsoidal Model of the Left Ventricle on the Deformation Measures Estimated Using 2-D Speckle Tracking-An In-Silico Study

Effect of the out-of-plane (OOP) movement amplitude on estimates of global displacements (radial, circumferential) and strains (radial , circumferential ) was studied in an ellipsoidal model of the left ventricle using finite-element modeling (FEM), synthetic ultrasonic data, and short-axis view. This effect was assessed using median of the absolute relative error (RE) of the global parameters. FEM provided node displacements for synthetic ultrasonic data and reference data generation. Displacements were estimated using block-matching (BM) and B-spline (BS) methods. FEM-derived data analysis, free from errors resulting from speckle tracking, indicated that the tissue motion introduced REs of global strain estimates below 4.5%. The effect of the OOP motion amplitude on strain estimates was strain specific and depended on the displacement estimation method. In the case of , the increase of the OOP amplitude resulted in quasi-linear increase of the RE from approximately 10% to 15%. The modulus of the end-systolic (ES) errors of the estimates almost linearly increased with increasing OOP amplitude approximately from 10% to 16%. REs of the estimate were close to 80% and 40%, respectively, in the case of the BM and BS methods, and increased with increasing OOP amplitude. The modulus of the ES errors of the estimates in the case of the BS method was about -40% and showed low sensitivity to the OOP amplitude; in the BM case, these errors varied approximately from -70% to -58% for OOP amplitude from 0 to 15 mm.

Model-Based Generation of Large Databases of Cardiac Images: Synthesis of Pathological Cine MR Sequences From Real Healthy Cases

Collecting large databases of annotated medical images is crucial for the validation and testing of feature extraction, statistical analysis, and machine learning algorithms. Recent advances in cardiac electromechanical modeling and image synthesis provided a framework to generate synthetic images based on realistic mesh simulations. Nonetheless, their potential to augment an existing database with large amounts of synthetic cases requires further investigation. We build upon these works and propose a revised scheme for synthesizing pathological cardiac sequences from real healthy sequences. Our new pipeline notably involves a much easier registration problem to reduce potential artifacts, and takes advantage of mesh correspondences to generate new data from a given case without additional registration. The output sequences are thoroughly examined in terms of quality and usability on a given application: the assessment of myocardial viability, via the generation of 465 synthetic cine MR sequences (15 healthy and 450 with pathological tissue viability [random location, extent, and grade, up to myocardial infarct]). We demonstrate that: 1) our methodology improves the state-of-the-art algorithms in terms of realism and accuracy of the simulated images and 2) our methodology is well-suited for the generation of large databases at small computational cost.

Realistic Vendor-Specific Synthetic Ultrasound Data for Quality Assurance of 2-D Speckle Tracking Echocardiography: Simulation Pipeline and Open Access Database

Two-dimensional (2-D) echocardiography is the modality of choice in the clinic for the diagnosis of cardiac disease. Hereto, speckle tracking (ST) packages complement visual assessment by the cardiologist by providing quantitative diagnostic markers of global and regional cardiac function (e.g., displacement, strain, and strain-rate). Yet, the reported high vendor-dependence between the outputs of different ST packages raises clinical concern and hampers the widespread dissemination of the ST technology. In part, this is due to the lack of a solid commonly accepted quality assurance pipeline for ST packages. Recently, we have developed a framework to benchmark ST algorithms for 3-D echocardiography by using realistic simulated volumetric echocardiographic recordings. Yet, 3-D echocardiography remains an emerging technology, whereas the compelling clinical concern is, so far, directed to the standardization of 2-D ST only. Therefore, by building upon our previous work, we present in this paper a pipeline to generate realistic synthetic sequences for 2-D ST algorithms. Hereto, the synthetic cardiac motion is obtained from a complex electromechanical heart model, whereas realistic vendor-specific texture is obtained by sampling a real clinical ultrasound recording. By modifying the parameters in our pipeline, we generated an open-access library of 105 synthetic sequences encompassing: 1) healthy and ischemic motion patterns; 2) the most common apical probe orientations; and 3) vendor-specific image quality from seven different systems. Ground truth deformation is also provided to allow performance analysis. The application of the provided data set is also demonstrated in the benchmarking of a recent academic ST algorithm.

A Framework for the Generation of Realistic Synthetic Cardiac Ultrasound and Magnetic Resonance Imaging Sequences From the Same Virtual Patients

The use of synthetic sequences is one of the most promising tools for advanced in silico evaluation of the quantification of cardiac deformation and strain through 3-D ultrasound (US) and magnetic resonance (MR) imaging. In this paper, we propose the first simulation framework which allows the generation of realistic 3-D synthetic cardiac US and MR (both cine and tagging) image sequences from the same virtual patient. A state-of-the-art electromechanical (E/M) model was exploited for simulating groundtruth cardiac motion fields ranging from healthy to various pathological cases, including both ventricular dyssynchrony and myocardial ischemia. The E/M groundtruth along with template MR/US images and physical simulators were combined in a unified framework for generating synthetic data. We efficiently merged several warping strategies to keep the full control of myocardial deformations while preserving realistic image texture. In total, we generated 18 virtual patients, each with synthetic 3-D US, cine MR, and tagged MR sequences. The simulated images were evaluated both qualitatively by showing realistic textures and quantitatively by observing myocardial intensity distributions similar to real data. In particular, the US simulation showed a smoother myocardium/background interface than the state-of-the-art. We also assessed the mechanical properties. The pathological subjects were discriminated from the healthy ones by both global indexes (ejection fraction and the global circumferential strain) and regional strain curves. The synthetic database is comprehensive in terms of both pathology and modality, and has a level of realism sufficient for validation purposes. All the 90 sequences are made publicly available to the research community via an open-access database.

Efficient Two-Pass 3-D Speckle Tracking for Ultrasound Imaging

Speckle tracking based on block matching is the most common method for multi-dimensional motion estimation in ultrasound elasticity imaging. Extension of two-dimensional (2-D) methods to three dimensions (3-D) has been problematic because of the large computational load of 3-D tracking, as well as performance issues related to the low frame (volume) rates of 3-D images. To address both of these problems, we have developed an efficient two-pass tracking method suited to cardiac elasticity imaging. PatchMatch, originally developed for image editing, has been adapted for ultrasound to provide first-pass displacement estimates. Second-pass estimation uses conventional block matching within a much smaller search region. 3-D displacements are then obtained using correlation filtering previously shown to be effective against speckle decorrelation. Both simulated and in vivo canine cardiac results demonstrate that the proposed two-pass method reduces computational cost compared to conventional 3-D exhaustive search by a factor of 10. Moreover, it outperforms one-pass tracking by a factor of about 3 in terms of root-mean-square error relative to available ground-truth displacements.

Motion Estimation in Echocardiography Using Sparse Representation and Dictionary Learning

This paper introduces a new method for cardiac motion estimation in 2-D ultrasound images. The motion estimation problem is formulated as an energy minimization, whose data fidelity term is built using the assumption that the images are corrupted by multiplicative Rayleigh noise. In addition to a classical spatial smoothness constraint, the proposed method exploits the sparse properties of the cardiac motion to regularize the solution via an appropriate dictionary learning step. The proposed method is evaluated on one data set with available ground-truth, including four sequences of highly realistic simulations. The approach is also validated on both healthy and pathological sequences of in vivo data. We evaluate the method in terms of motion estimation accuracy and strain errors and compare the performance with state-of-the-art algorithms. The results show that the proposed method gives competitive results for the considered data. Furthermore, the in vivo strain analysis demonstrates that meaningful clinical interpretation can be obtained from the estimated motion vectors.This paper introduces a new method for cardiac motion estimation in 2-D ultrasound images. The motion estimation problem is formulated as an energy minimization, whose data fidelity term is built using the assumption that the images are corrupted by multiplicative Rayleigh noise. In addition to a classical spatial smoothness constraint, the proposed method exploits the sparse properties of the cardiac motion to regularize the solution via an appropriate dictionary learning step. The proposed method is evaluated on one data set with available ground-truth, including four sequences of highly realistic simulations. The approach is also validated on both healthy and pathological sequences of in vivo data. We evaluate the method in terms of motion estimation accuracy and strain errors and compare the performance with state-of-the-art algorithms. The results show that the proposed method gives competitive results for the considered data. Furthermore, the in vivo strain analysis demonstrates that meaningful clinical interpretation can be obtained from the estimated motion vectors.

Fast left ventricle tracking using localized anatomical affine optical flow

In daily clinical cardiology practice, left ventricle (LV) global and regional function assessment is crucial for disease diagnosis, therapy selection, and patient follow-up. Currently, this is still a time-consuming task, spending valuable human resources. In this work, a novel fast methodology for automatic LV tracking is proposed based on localized anatomically constrained affine optical flow. This novel method can be combined to previously proposed segmentation frameworks or manually delineated surfaces at an initial frame to obtain fully delineated datasets and, thus, assess both global and regional myocardial function. Its feasibility and accuracy were investigated in 3 distinct public databases, namely in realistically simulated 3D ultrasound, clinical 3D echocardiography, and clinical cine cardiac magnetic resonance images. The method showed accurate tracking results in all databases, proving its applicability and accuracy for myocardial function assessment. Moreover, when combined to previous state-of-the-art segmentation frameworks, it outperformed previous tracking strategies in both 3D ultrasound and cardiac magnetic resonance data, automatically computing relevant cardiac indices with smaller biases and narrower limits of agreement compared to reference indices. Simultaneously, the proposed localized tracking method showed to be suitable for online processing, even for 3D motion assessment. Importantly, although here evaluated for LV tracking only, this novel methodology is applicable for tracking of other target structures with minimal adaptations.

Fast Simulation of Dynamic Ultrasound Images Using the GPU

Simulated ultrasound data is a valuable tool for development and validation of quantitative image analysis methods in echocardiography. Unfortunately, simulation time can become prohibitive for phantoms consisting of a large number of point scatterers. The COLE algorithm by Gao et al. is a fast convolution-based simulator that trades simulation accuracy for improved speed. We present highly efficient parallelized CPU and GPU implementations of the COLE algorithm with an emphasis on dynamic simulations involving moving point scatterers. We argue that it is crucial to minimize the amount of data transfers from the CPU to achieve good performance on the GPU. We achieve this by storing the complete trajectories of the dynamic point scatterers as spline curves in the GPU memory. This leads to good efficiency when simulating sequences consisting of a large number of frames, such as B-mode and tissue Doppler data for a full cardiac cycle. In addition, we propose a phase-based subsample delay technique that efficiently eliminates flickering artifacts seen in B-mode sequences when COLE is used without enough temporal oversampling. To assess the performance, we used a laptop computer and a desktop computer, each equipped with a multicore Intel CPU and an NVIDIA GPU. Running the simulator on a high-end TITAN X GPU, we observed two orders of magnitude speedup compared to the parallel CPU version, three orders of magnitude speedup compared to simulation times reported by Gao et al. in their paper on COLE, and a speedup of 27000 times compared to the multithreaded version of Field II, using numbers reported in a paper by Jensen. We hope that by releasing the simulator as an open-source project we will encourage its use and further development.

Infarct Localization From Myocardial Deformation: Prediction and Uncertainty Quantification by Regression From a Low-Dimensional Space

Diagnosing and localizing myocardial infarct is crucial for early patient management and therapy planning. We propose a new method for predicting the location of myocardial infarct from local wall deformation, which has value for risk stratification from routine examinations such as (3D) echocardiography. The pipeline combines non-linear dimensionality reduction of deformation patterns and two multi-scale kernel regressions. Confidence in the diagnosis is assessed by a map of local uncertainties, which integrates plausible infarct locations generated from the space of reduced dimensionality. These concepts were tested on 500 synthetic cases generated from a realistic cardiac electromechanical model, and 108 pairs of 3D echocardiographic sequences and delayed-enhancement magnetic resonance images from real cases. Infarct prediction is made at a spatial resolution around 4 mm, more than 10 times smaller than the current diagnosis, made regionally. Our method is accurate, and significantly outperforms the clinically-used thresholding of the deformation patterns (on real data: sensitivity/specificity of 0.828/0.804, area under the curve: 0.909 versus 0.742 for the most predictive strain component). Uncertainty adds value to refine the diagnosis and eventually re-examine suspicious cases.

Hybrid Strategy to Simulate 3-D Nonlinear Radio-Frequency Ultrasound Using a Variant Spatial PSF

There are several simulators for medical ultrasound (US) applications that can fully compute the nonlinear propagation on the transmitted pulse and the corresponding radio-frequency (RF) images. Creanuis is one recent model used to generate nonlinear RF images; however, the time requirements are long compared with linear models using a convolution strategy. In this paper, we describe an approach using convolution coupled with nonlinear information to create a pseudoacoustic tool that is able to quickly generate realistic US images. Several point-spread functions (PSFs) are computed with Creanuis. These PSFs are extracted at different depths in order to take into account variation in the resolution and apparition of harmonics during propagation. One convolution is then conducted for each PSF to generate a set of nonlinear raw RF images. The final image is obtained by merging these raw images using a PSF-weighting function. This hybrid Creanuis strategy was extended to 2-D, 2-D +t , 3-D, and 3-D +t images for both linear and phased-array geometries. We validated h-Creanuis using the mean deviation between the proposed images and those created using Creanuis and examined their statistical distributions. The mean deviations of Creanuis and h-Creanuis are below 2.5% for fundamental and second-harmonic images. The 3-D +t images obtained demonstrate the correct motion characteristics for speckle in sequences of both fundamental and second-harmonic images.

Detailed Evaluation of Five 3D Speckle Tracking Algorithms Using Synthetic Echocardiographic Recordings

A plethora of techniques for cardiac deformation imaging with 3D ultrasound, typically referred to as 3D speckle tracking techniques, are available from academia and industry. Although the benefits of single methods over alternative ones have been reported in separate publications, the intrinsic differences in the data and definitions used makes it hard to compare the relative performance of different solutions. To address this issue, we have recently proposed a framework to simulate realistic 3D echocardiographic recordings and used it to generate a common set of ground-truth data for 3D speckle tracking algorithms, which was made available online. The aim of this study was therefore to use the newly developed database to contrast non-commercial speckle tracking solutions from research groups with leading expertise in the field. The five techniques involved cover the most representative families of existing approaches, namely block-matching, radio-frequency tracking, optical flow and elastic image registration. The techniques were contrasted in terms of tracking and strain accuracy. The feasibility of the obtained strain measurements to diagnose pathology was also tested for ischemia and dyssynchrony.

Four challenges in medical image analysis from an industrial perspective

Today's medical imaging systems produce a huge amount of images containing a wealth of information. However, the information is hidden in the data and image analysis algorithms are needed to extract it, to make it readily available for medical decisions and to enable an efficient work flow. Advances in medical image analysis over the past 20 years mean there are now many algorithms and ideas available that allow to address medical image analysis tasks in commercial solutions with sufficient performance in terms of accuracy, reliability and speed. At the same time new challenges have arisen. Firstly, there is a need for more generic image analysis technologies that can be efficiently adapted for a specific clinical task. Secondly, efficient approaches for ground truth generation are needed to match the increasing demands regarding validation and machine learning. Thirdly, algorithms for analyzing heterogeneous image data are needed. Finally, anatomical and organ models play a crucial role in many applications, and algorithms to construct patient-specific models from medical images with a minimum of user interaction are needed. These challenges are complementary to the on-going need for more accurate, more reliable and faster algorithms, and dedicated algorithmic solutions for specific applications.

Anatomical Image Registration Using Volume Conservation to Assess Cardiac Deformation From 3D Ultrasound Recordings

Myocardial deformation imaging can provide valuable insights in myocardial mechanics and help in the diagnosis, prognosis and follow-up of cardiac diseases. However, extracting these indices in 3D is challenging due to the limitations in spatial and temporal resolution of the current volumetric ultrasound systems. For this purpose, we developed an anatomical free-form deformation image registration framework which is locally adapted to the anatomy of the heart. In this work we explored whether incorporating a myocardial volume conservation regularizer would improve strain estimates. We evaluated our technique on in silico echo sequences featuring realistic speckle textures and showed the volume conservation regularizer to be beneficial in reducing strain errors further when used in combination with a smoothness penalty. This combination led to more physiological boundary conditions. It also made distinguishing ischemic from normal segments easier in clinical images.