A Framework for the Generation of Realistic Synthetic Cardiac Ultrasound and Magnetic Resonance Imaging Sequences From the Same Virtual Patients

2-Dimensional Echocardiographic Global Longitudinal Strain With Artificial Intelligence Using Open Data From a UK-Wide Collaborative

BACKGROUND

Global longitudinal strain (GLS) is reported to be more reproducible and prognostic than ejection fraction. Automated, transparent methods may increase trust and uptake.

OBJECTIVES

The authors developed open machine-learning-based GLS methodology and validate it using multiexpert consensus from the Unity UK Echocardiography AI Collaborative.

METHODS

We trained a multi-image neural network (Unity-GLS) to identify annulus, apex, and endocardial curve on 6,819 apical 4-, 2-, and 3-chamber images. The external validation dataset comprised those 3 views from 100 echocardiograms. End-systolic and -diastolic frames were each labelled by 11 experts to form consensus tracings and points. They also ordered the echocardiograms by visual grading of longitudinal function. One expert calculated global strain using 2 proprietary packages.

RESULTS

The median GLS, averaged across the 11 individual experts, was -16.1 (IQR: -19.3 to -12.5). Using each case's expert consensus measurement as the reference standard, individual expert measurements had a median absolute error of 2.00 GLS units. In comparison, the errors of the machine methods were: Unity-GLS 1.3, proprietary A 2.5, proprietary B 2.2. The correlations with the expert consensus values were for individual experts 0.85, Unity-GLS 0.91, proprietary A 0.73, proprietary B 0.79. Using the multiexpert visual ranking as the reference, individual expert strain measurements found a median rank correlation of 0.72, Unity-GLS 0.77, proprietary A 0.70, and proprietary B 0.74.

CONCLUSIONS

Our open-source approach to calculating GLS agrees with experts' consensus as strongly as the individual expert measurements and proprietary machine solutions. The training data, code, and trained networks are freely available online.

A simulation-based phantom model for generating synthetic mitral valve image data-application to MRI acquisition planning

PURPOSE

Numerical phantom methods are widely used in the development of medical imaging methods. They enable quantitative evaluation and direct comparison with controlled and known ground truth information. Cardiac magnetic resonance has the potential for a comprehensive evaluation of the mitral valve (MV). The goal of this work is the development of a numerical simulation framework that supports the investigation of MRI imaging strategies for the mitral valve.

METHODS

We present a pipeline for synthetic image generation based on the combination of individual anatomical 3D models with a position-based dynamics simulation of the mitral valve closure. The corresponding images are generated using modality-specific intensity models and spatiotemporal sampling concepts. We test the applicability in the context of MRI imaging strategies for the assessment of the mitral valve. Synthetic images are generated with different strategies regarding image orientation (SAX and rLAX) and spatial sampling density.

RESULTS

The suitability of the imaging strategy is evaluated by comparing MV segmentations against ground truth annotations. The generated synthetic images were compared to ones acquired with similar parameters, and the result is promising. The quantitative analysis of annotation results suggests that the rLAX sampling strategy is preferable for MV assessment, reaching accuracy values that are comparable to or even outperform literature values.

CONCLUSION

The proposed approach provides a valuable tool for the evaluation and optimization of cardiac valve image acquisition. Its application to the use case identifies the radial image sampling strategy as the most suitable for MV assessment through MRI.

A user independent denoising method for x-nuclei MRI and MRS

PURPOSE

X-nuclei (also called non-proton MRI) MRI and spectroscopy are limited by the intrinsic low SNR as compared to conventional proton imaging. Clinical translation of x-nuclei examination warrants the need of a robust and versatile tool improving image quality for diagnostic use. In this work, we compare a novel denoising method with fewer inputs to the current state-of-the-art denoising method.

METHODS

Denoising approaches were compared on human acquisitions of sodium (23 Na) brain, deuterium (2 H) brain, carbon (13 C) heart and brain, and simulated dynamic hyperpolarized 13 C brain scans, with and without additional noise. The current state-of-the-art denoising method Global-local higher order singular value decomposition (GL-HOSVD) was compared to the few-input method tensor Marchenko-Pastur principal component analysis (tMPPCA). Noise-removal was quantified by residual distributions, and statistical analyses evaluated the differences in mean-square-error and Bland-Altman analysis to quantify agreement between original and denoised results of noise-added data.

RESULTS

GL-HOSVD and tMPPCA showed similar performance for the variety of x-nuclei data analyzed in this work, with tMPPCA removing ˜5% more noise on average over GL-HOSVD. The mean ratio between noise-added and denoising reproducibility coefficients of the Bland-Altman analysis when compared to the original are also similar for the two methods with 3.09 ± 1.03 and 2.83 ± 0.79 for GL-HOSVD and tMPPCA, respectively.

CONCLUSION

The strength of tMPPCA lies in the few-input approach, which generalizes well to different data sources. This makes the use of tMPPCA denoising a robust and versatile tool in x-nuclei imaging improvements and the preferred denoising method.

Effects of spatial and temporal resolution on cardiovascular magnetic resonance feature tracking measurements using a simple realistic numerical phantom

OBJECTIVES

To develop a single-slice numerical phantom with known myocardial motion, at several temporal and in-plane spatial resolutions, for testing and comparison of Cardiovascular Magnetic Resonance (CMR) feature tracking (FT) software.

METHODS

The phantom was developed based on CMR acquisitions of one volunteer (acquired cine, tagging cine, T1 map, T2 map, proton density weighted image). The numerical MRI simulator JEMRIS was used, and the phantom was generated at several in-plane spatial resolutions (1.4 × 1.4 mm2 to 3.0 × 3.0 mm2) and temporal resolutions (20 to 40 cardiac phases). Two feature tracking software packages were tested: Medical Image Tracking Toolbox (MITT) and two versions of cvi42 (v5.3.8 and v5.13.7). The effect of resolution on strain results was investigated with reference to ground-truth radial and circumferential strain.

RESULTS

Peak radial strain was consistently undermeasured more for cvi42 v5.13.7 than for v5.3.8. Increased pixel size produced a trend of increased difference from ground-truth peak strain, with the largest changes for cvi42 obtained using v5.13.7 between 1.4 × 1.4 mm2 and 3.0 × 3.0 mm2, at 9.17 percentage points (radial) and 8.42 percentage points (circumferential).

CONCLUSIONS

The results corroborate the presence of intervendor differences in feature tracking results and show the magnitude of strain differences between software versions.

ADVANCES IN KNOWLEDGE

This study shows how temporal and in-plane spatial resolution can affect feature tracking with reference to the ground-truth strain of a numerical phantom. Results reaffirm the need for numerical phantom development for the validation and testing of FT software.

A Framework for Simulating Cardiac MR Images With Varying Anatomy and Contrast

One of the limiting factors for the development and adoption of novel deep-learning (DL) based medical image analysis methods is the scarcity of labeled medical images. Medical image simulation and synthesis can provide solutions by generating ample training data with corresponding ground truth labels. Despite recent advances, generated images demonstrate limited realism and diversity. In this work, we develop a flexible framework for simulating cardiac magnetic resonance (MR) images with variable anatomical and imaging characteristics for the purpose of creating a diversified virtual population. We advance previous works on both cardiac MR image simulation and anatomical modeling to increase the realism in terms of both image appearance and underlying anatomy. To diversify the generated images, we define parameters: 1)to alter the anatomy, 2) to assign MR tissue properties to various tissue types, and 3) to manipulate the image contrast via acquisition parameters. The proposed framework is optimized to generate a substantial number of cardiac MR images with ground truth labels suitable for downstream supervised tasks. A database of virtual subjects is simulated and its usefulness for aiding a DL segmentation method is evaluated. Our experiments show that training completely with simulated images can perform comparable with a model trained with real images for heart cavity segmentation in mid-ventricular slices. Moreover, such data can be used in addition to classical augmentation for boosting the performance when training data is limited, particularly by increasing the contrast and anatomical variation, leading to better regularization and generalization. The database is publicly available at https://osf.io/bkzhm/ and the simulation code will be available at https://github.com/sinaamirrajab/CMRI.

Use of semi-synthetic data for catheter segmentation improvement

In the era of data-driven machine learning algorithms, data is the new oil. For the most optimal results, datasets should be large, heterogeneous and, crucially, correctly labeled. However, data collection and labeling are time-consuming and labor-intensive processes. In the field of medical device segmentation, present during minimally invasive surgery, this leads to a lack of informative data. Motivated by this drawback, we developed an algorithm generating semi-synthetic images based on real ones. The concept of this algorithm is to place a randomly shaped catheter in an empty heart cavity, where the shape of the catheter is generated by forward kinematics of continuum robots. Having implemented the proposed algorithm, we generated new images of heart cavities with various artificial catheters. We compared the results of deep neural networks trained purely on real datasets with respect to networks trained on both real and semi-synthetic datasets, highlighting that semi-synthetic data improves catheter segmentation accuracy. A modified U-Net trained on combined datasets performed the segmentation with a Dice similarity coefficient of 92.6 ± 2.2%, while the same model trained only on real images achieved a Dice similarity coefficient of 86.5 ± 3.6%. Therefore, using semi-synthetic data allows for the decrease of accuracy spread, improves model generalization, reduces subjectivity, shortens the labeling routine, increases the number of samples, and improves the heterogeneity.

MITEA: A dataset for machine learning segmentation of the left ventricle in 3D echocardiography using subject-specific labels from cardiac magnetic resonance imaging

Segmentation of the left ventricle (LV) in echocardiography is an important task for the quantification of volume and mass in heart disease. Continuing advances in echocardiography have extended imaging capabilities into the 3D domain, subsequently overcoming the geometric assumptions associated with conventional 2D acquisitions. Nevertheless, the analysis of 3D echocardiography (3DE) poses several challenges associated with limited spatial resolution, poor contrast-to-noise ratio, complex noise characteristics, and image anisotropy. To develop automated methods for 3DE analysis, a sufficiently large, labeled dataset is typically required. However, ground truth segmentations have historically been difficult to obtain due to the high inter-observer variability associated with manual analysis. We address this lack of expert consensus by registering labels derived from higher-resolution subject-specific cardiac magnetic resonance (CMR) images, producing 536 annotated 3DE images from 143 human subjects (10 of which were excluded). This heterogeneous population consists of healthy controls and patients with cardiac disease, across a range of demographics. To demonstrate the utility of such a dataset, a state-of-the-art, self-configuring deep learning network for semantic segmentation was employed for automated 3DE analysis. Using the proposed dataset for training, the network produced measurement biases of -9 ± 16 ml, -1 ± 10 ml, -2 ± 5 %, and 5 ± 23 g, for end-diastolic volume, end-systolic volume, ejection fraction, and mass, respectively, outperforming an expert human observer in terms of accuracy as well as scan-rescan reproducibility. As part of the Cardiac Atlas Project, we present here a large, publicly available 3DE dataset with ground truth labels that leverage the higher resolution and contrast of CMR, to provide a new benchmark for automated 3DE analysis. Such an approach not only reduces the effect of observer-specific bias present in manual 3DE annotations, but also enables the development of analysis techniques which exhibit better agreement with CMR compared to conventional methods. This represents an important step for enabling more efficient and accurate diagnostic and prognostic information to be obtained from echocardiography.

A Pipeline for the Generation of Synthetic Cardiac Color Doppler

Color Doppler imaging (CDI) is the modality of choice for simultaneous visualization of myocardium and intracavitary flow over a wide scan area. This visualization modality is subject to several sources of error, the main ones being aliasing and clutter. Mitigation of these artifacts is a major concern for better analysis of intracardiac flow. One option to address these issues is through simulations. In this article, we present a numerical framework for generating clinical-like CDI. Synthetic blood vector fields were obtained from a patient-specific computational fluid dynamics CFD model. Realistic texture and clutter artifacts were simulated from real clinical ultrasound cineloops. We simulated several scenarios highlighting the effects of 1) flow acceleration; 2) wall clutter; and 3) transmit wavefronts, on Doppler velocities. As a comparison, an "ideal" color Doppler was also simulated, without these harmful effects. This synthetic dataset is made publicly available and can be used to evaluate the quality of Doppler estimation techniques. Besides, this approach can be seen as a first step toward the generation of comprehensive datasets for training neural networks to improve the quality of Doppler imaging.

Influence of Heart Rate on Left and Right Ventricular Longitudinal Strain in Patients with Chronic Heart Failure

Over the past years, a number of studies have demonstrated the relevance of strain assessed by two-dimensional speckle tracking echocardiography (STE) in evaluating ventricular function. The aim of this study was to analyze changes in left (LV) and right ventricular (RV) longitudinal strain associated with variations of heart rate (HR) in participants with and without chronic heart failure (CHF). We enrolled 45 patients, 38 of these diagnosed with CHF and carrying an implantable cardioverter defibrillator, and seven patients with pacemakers and without CHF. The frequency of atrial stimulation was increased to 90 beats/min and an echocardiogram was performed at each increase of 10 beats/min. Global LV and RV longitudinal strain (LVGLS and RVGLS, respectively) and RV free wall longitudinal strain (RVfwLS) were calculated at each HR. When analyzed as continuous variables, significant reductions in LVGLS were detected at higher HRs, whereas improvements in both RVGLS and RVfwLS were observed. Patients with a worsening of LVGLS (76% overall) were more likely to present lower baseline LV function. Only a few patients (18% for RVGLS and 16% for RVfwLS) exhibited HR-related deteriorations of RV strain measures, which was associated with lower levels of baseline RV function and higher pulmonary systolic pressures. Finally, 21 (47%) and 25 (56%) participants responded with improvements in RVGLS and RVfwLS, respectively. Our findings revealed heterogeneous RV and LV responses to increases in HR. These findings might ultimately be used to optimize cardiac functionality in patients diagnosed with CHF.

Generating Synthetic Labeled Data From Existing Anatomical Models: An Example With Echocardiography Segmentation

Deep learning can bring time savings and increased reproducibility to medical image analysis. However, acquiring training data is challenging due to the time-intensive nature of labeling and high inter-observer variability in annotations. Rather than labeling images, in this work we propose an alternative pipeline where images are generated from existing high-quality annotations using generative adversarial networks (GANs). Annotations are derived automatically from previously built anatomical models and are transformed into realistic synthetic ultrasound images with paired labels using a CycleGAN. We demonstrate the pipeline by generating synthetic 2D echocardiography images to compare with existing deep learning ultrasound segmentation datasets. A convolutional neural network is trained to segment the left ventricle and left atrium using only synthetic images. Networks trained with synthetic images were extensively tested on four different unseen datasets of real images with median Dice scores of 91, 90, 88, and 87 for left ventricle segmentation. These results match or are better than inter-observer results measured on real ultrasound datasets and are comparable to a network trained on a separate set of real images. Results demonstrate the images produced can effectively be used in place of real data for training. The proposed pipeline opens the door for automatic generation of training data for many tasks in medical imaging as the same process can be applied to other segmentation or landmark detection tasks in any modality. The source code and anatomical models are available to other researchers.1 1https://adgilbert.github.io/data-generation/.

Myocardial strain imaging: review of general principles, validation, and sources of discrepancies

Myocardial tissue tracking imaging techniques have been developed for a more accurate evaluation of myocardial deformation (i.e. strain), with the potential to overcome the limitations of ejection fraction (EF) and to contribute, incremental to EF, to the diagnosis and prognosis in cardiac diseases. While most of the deformation imaging techniques are based on the similar principles of detecting and tracking specific patterns within an image, there are intra- and inter-imaging modality inconsistencies limiting the wide clinical applicability of strain. In this review, we aimed to describe the particularities of the echocardiographic and cardiac magnetic resonance deformation techniques, in order to understand the discrepancies in strain measurement, focusing on the potential sources of variation: related to the software used to analyse the data, to the different physics of image acquisition and the different principles of 2D vs. 3D approaches. As strain measurements are not interchangeable, it is highly desirable to work with validated strain assessment tools, in order to derive information from evidence-based data. There is, however, a lack of solid validation of the current tissue tracking techniques, as only a few of the commercial deformation imaging softwares have been properly investigated. We have, therefore, addressed in this review the neglected issue of suboptimal validation of tissue tracking techniques, in order to advocate for this matter.

Fog Computing Employed Computer Aided Cancer Classification System Using Deep Neural Network in Internet of Things Based Healthcare System

Computer assisted automatic smart pattern analysis of cancer affected pixel structure takes critical role in pre-interventional decision making for oral cancer treatment. Internet of Things (IoT) in healthcare systems is now emerging solution for modern e-healthcare system to provide high quality medical care. In this research work, we proposed a novel method which utilizes a modified vesselness measurement and a Deep Convolutional Neural Network (DCNN) to identify the oral cancer region structure in IoT based smart healthcare system. The robust vesselness filtering scheme handles noise while reserving small structures, while the CNN framework considerably improves classification accuracy by deblurring focused region of interest (ROI) through integrating with multi-dimensional information from feature vector selection step. The marked feature vector points are extracted from each connected component in the region and used as input for training the CNN. During classification, each connected part is individually analysed using the trained DCNN by considering the feature vector values that belong to its region. For a training of 1500 image dataset, an accuracy of 96.8% and sensitivity of 92% is obtained. Hence, the results of this work validate that the proposed algorithm is effective and accurate in terms of classification of oral cancer region in accurate decision making. The developed system can be used in IoT based diagnosis in health care systems, where accuracy and real time diagnosis are essential.

Arterial Spin Labeling Images Synthesis From sMRI Using Unbalanced Deep Discriminant Learning

Adequate medical images are often indispensable in contemporary deep learning-based medical imaging studies, although the acquisition of certain image modalities may be limited due to several issues including high costs and patients issues. However, thanks to recent advances in deep learning techniques, the above tough problem can be substantially alleviated by medical images synthesis, by which various modalities including T1/T2/DTI MRI images, PET images, cardiac ultrasound images, retinal images, and so on, have already been synthesized. Unfortunately, the arterial spin labeling (ASL) image, which is an important fMRI indicator in dementia diseases diagnosis nowadays, has never been comprehensively investigated for the synthesis purpose yet. In this paper, ASL images have been successfully synthesized from structural magnetic resonance images for the first time. Technically, a novel unbalanced deep discriminant learning-based model equipped with new ResNet sub-structures is proposed to realize the synthesis of ASL images from structural magnetic resonance images. The extensive experiments have been conducted. Comprehensive statistical analyses reveal that: 1) this newly introduced model is capable to synthesize ASL images that are similar towards real ones acquired by actual scanning; 2) synthesized ASL images obtained by the new model have demonstrated outstanding performance when undergoing rigorous tests of region-based and voxel-based corrections of partial volume effects, which are essential in ASL images processing; and 3) it is also promising that the diagnosis performance of dementia diseases can be significantly improved with the help of synthesized ASL images obtained by the new model, based on a multi-modal MRI dataset containing 355 demented patients in this paper.

A gradient-based optical-flow cardiac motion estimation method for cine and tagged MR images

A new method is proposed to quantify the myocardial motion from both 2D C(ine)-MRI and T(agged)-MRI sequences. The tag pattern offers natural landmarks within the image that makes it possible to accurately quantify the motion within the myocardial wall. Therefore, several methods have been proposed for T-MRI. However, the lack of salient features within the cardiac wall in C-MRI hampers local motion estimation. Our method aims to ensure the local intensity and shape features invariance during motion through the iterative minimization of a cost function via a random walk scheme. The proposed approach is evaluated on realistic simulated C-MRI and T-MRI sequences. The results show more than 53% improvements on displacement estimation, and more than 24% on strain estimation for both C-MRI and T-MRI sequences, as compared to state-of-the-art cardiac motion estimators. Preliminary experiments on clinical data have shown a good ability of the proposed method to detect abnormal motion patterns related to pathology. If those results are confirmed on large databases, this would open up the possibility for more accurate diagnosis of cardiac function from standard C-MRI examinations and also the retrospective study of prior studies.

An overview of deep learning in medical imaging focusing on MRI

What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of deep learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.

Meshfree simulations of ultrasound vector flow imaging using smoothed particle hydrodynamics

Before embarking on a series of in vivo tests, design of ultrasound-flow-imaging modalities are generally more efficient through computational models as multiple configurations can be tested methodically. To that end, simulation models must generate realistic blood flow dynamics and Doppler signals. The current in silico ultrasound simulation techniques suffer mainly from uncertainty in providing accurate trajectories of moving ultrasound scatterers. In mesh-based Eulerian methods, numerical truncation errors from the interpolated velocities, both in the time and space dimensions, can accumulate significantly and make the pathlines unreliable. These errors can distort beam-to-beam inter-correlation present in ultrasound flow imaging. It is thus a technical issue to model a correct motion of the scatterers by considering their interaction with boundaries and neighboring scatterers. We hypothesized that in silico analysis of emerging ultrasonic imaging modalities can be implemented more accurately with meshfree approaches. We developed an original fluid-ultrasound simulation environment based on a meshfree Lagrangian CFD (computational fluid dynamics) formulation, which allows analysis of ultrasound flow imaging. This simulator combines smoothed particle hydrodynamics (SPH) and Fourier-domain linear acoustics (SIMUS  =  simulator for ultrasound imaging). With such a particle-based computation, the fluid particles also acted as individual ultrasound scatterers, resulting in a direct and physically sound fluid-ultrasonic coupling. We used the in-house algorithms for fluid and ultrasound simulations to simulate high-frame-rate vector flow imaging. The potential of the particle-based method was tested in 2D simulations of vector Doppler for the intracarotid flow. The Doppler-based velocity fields were compared with those issued from SPH. The numerical evaluations showed that the vector flow fields obtained by vector Doppler components were in good agreement with the original SPH velocities, with relative errors less than 10% and 2% in the cross-beam and axial directions, respectively. Our results showed that SPH-SIMUS coupling enables direct and realistic simulations of ultrasound flow imaging. The proposed coupled algorithm has also the advantage to be 3D compatible and parallelizable.

Realistic Vendor-Specific Synthetic Ultrasound Data for Quality Assurance of 2-D Speckle Tracking Echocardiography: Simulation Pipeline and Open Access Database

Two-dimensional (2-D) echocardiography is the modality of choice in the clinic for the diagnosis of cardiac disease. Hereto, speckle tracking (ST) packages complement visual assessment by the cardiologist by providing quantitative diagnostic markers of global and regional cardiac function (e.g., displacement, strain, and strain-rate). Yet, the reported high vendor-dependence between the outputs of different ST packages raises clinical concern and hampers the widespread dissemination of the ST technology. In part, this is due to the lack of a solid commonly accepted quality assurance pipeline for ST packages. Recently, we have developed a framework to benchmark ST algorithms for 3-D echocardiography by using realistic simulated volumetric echocardiographic recordings. Yet, 3-D echocardiography remains an emerging technology, whereas the compelling clinical concern is, so far, directed to the standardization of 2-D ST only. Therefore, by building upon our previous work, we present in this paper a pipeline to generate realistic synthetic sequences for 2-D ST algorithms. Hereto, the synthetic cardiac motion is obtained from a complex electromechanical heart model, whereas realistic vendor-specific texture is obtained by sampling a real clinical ultrasound recording. By modifying the parameters in our pipeline, we generated an open-access library of 105 synthetic sequences encompassing: 1) healthy and ischemic motion patterns; 2) the most common apical probe orientations; and 3) vendor-specific image quality from seven different systems. Ground truth deformation is also provided to allow performance analysis. The application of the provided data set is also demonstrated in the benchmarking of a recent academic ST algorithm.