Three-dimensional modeling for functional analysis of cardiac images: a review

Temporal Features-Fused Vision Retentive Network for Echocardiography Image Segmentation

Echocardiography is a widely used cardiac imaging modality in clinical practice. Physicians utilize echocardiography images to measure left ventricular volumes at end-diastole (ED) and end-systole (ES) frames, which are pivotal for calculating the ejection fraction and thus quantitatively assessing cardiac function. However, most existing approaches focus on features from ES frames and ED frames, neglecting the inter-frame correlations in unlabeled frames. Our model is based on an encoder-decoder architecture and consists of two modules: the Temporal Feature Fusion Module (TFFA) and the Vision Retentive Network (Vision RetNet) encoder. The TFFA leverages self-attention to learn inter-frame correlations across multiple consecutive frames and aggregates the features of the temporal-channel dimension through channel aggregation to highlight ambiguity regions. The Vision RetNet encoder introduces explicit spatial priors by constructing a spatial decay matrix using the Manhattan distance. We conducted experiments on the EchoNet-Dynamic dataset and the CAMUS dataset, where our proposed model demonstrates competitive performance. The experimental results indicate that spatial prior information and inter-frame correlations in echocardiography images can enhance the accuracy of semantic segmentation, and inter-frame correlations become even more effective when spatial priors are provided.

Slice2Mesh: 3D Surface Reconstruction From Sparse Slices of Images for the Left Ventricle

Cine MRI is a widely used technique to evaluate left ventricular function and motion, as it captures temporal information. However, due to the limited spatial resolution, cine MRI only provides a few sparse scans at regular positions and orientations, which poses challenges for reconstructing dense 3D cardiac structures, which is essential for better understanding the cardiac structure and motion in a dynamic 3D manner. In this study, we propose a novel learning-based 3D cardiac surface reconstruction method, Slice2Mesh, which directly predicts accurate and high-fidelity 3D meshes from sparse slices of cine MRI images under partial supervision of sparse contour points. Slice2Mesh leverages a 2D UNet to extract image features and a graph convolutional network to predict deformations from an initial template to various 3D surfaces, which enables it to produce topology-consistent meshes that can better characterize and analyze cardiac movement. We also introduce As Rigid As Possible energy in the deformation loss to preserve the intrinsic structure of the predefined template and produce realistic left ventricular shapes. We evaluated our method on 150 clinical test samples and achieved an average chamfer distance of 3.621 mm, outperforming traditional methods by approximately 2.5 mm. We also applied our method to produce 4D surface meshes from cine MRI sequences and utilized a simple SVM model on these 4D heart meshes to identify subjects with myocardial infarction, and achieved a classification sensitivity of 91.8% on 99 test subjects, including 49 abnormal patients, which implies great potential of our method for clinical use.

Automatic multi-view pose estimation in focused cardiac ultrasound

Focused cardiac ultrasound (FoCUS) is a valuable point-of-care method for evaluating cardiovascular structures and function, but its scope is limited by equipment and operator's experience, resulting in primarily qualitative 2D exams. This study presents a novel framework to automatically estimate the 3D spatial relationship between standard FoCUS views. The proposed framework uses a multi-view U-Net-like fully convolutional neural network to regress line-based heatmaps representing the most likely areas of intersection between input images. The lines that best fit the regressed heatmaps are then extracted, and a system of nonlinear equations based on the intersection between view triplets is created and solved to determine the relative 3D pose between all input images. The feasibility and accuracy of the proposed pipeline were validated using a novel realistic in silico FoCUS dataset, demonstrating promising results. Interestingly, as shown in preliminary experiments, the estimation of the 2D images' relative poses enables the application of 3D image analysis methods and paves the way for 3D quantitative assessments in FoCUS examinations.

Bi-DCNet: Bilateral Network with Dilated Convolutions for Left Ventricle Segmentation

Left ventricular segmentation is a vital and necessary procedure for assessing cardiac systolic and diastolic function, while echocardiography is an indispensable diagnostic technique that enables cardiac functionality assessment. However, manually labeling the left ventricular region on echocardiography images is time consuming and leads to observer bias. Recent research has demonstrated that deep learning has the capability to employ the segmentation process automatically. However, on the downside, it still ignores the contribution of all semantic information through the segmentation process. This study proposes a deep neural network architecture based on BiSeNet, named Bi-DCNet. This model comprises a spatial path and a context path, with the former responsible for spatial feature (low-level) acquisition and the latter responsible for contextual semantic feature (high-level) exploitation. Moreover, it incorporates feature extraction through the integration of dilated convolutions to achieve a larger receptive field to capture multi-scale information. The EchoNet-Dynamic dataset was utilized to assess the proposed model, and this is the first bilateral-structured network implemented on this large clinical video dataset for accomplishing the segmentation of the left ventricle. As demonstrated by the experimental outcomes, our method obtained 0.9228 and 0.8576 in DSC and IoU, respectively, proving the structure's effectiveness.

An Overview of Deep Learning Methods for Left Ventricle Segmentation

Cardiac health diseases are one of the key causes of death around the globe. The number of heart patients has considerably increased during the pandemic. Therefore, it is crucial to assess and analyze the medical and cardiac images. Deep learning architectures, specifically convolutional neural networks have profoundly become the primary choice for the assessment of cardiac medical images. The left ventricle is a vital part of the cardiovascular system where the boundary and size perform a significant role in the evaluation of cardiac function. Due to automatic segmentation and good promising results, the left ventricle segmentation using deep learning has attracted a lot of attention. This article presents a critical review of deep learning methods used for the left ventricle segmentation from frequently used imaging modalities including magnetic resonance images, ultrasound, and computer tomography. This study also demonstrates the details of the network architecture, software, and hardware used for training along with publicly available cardiac image datasets and self-prepared dataset details incorporated. The summary of the evaluation matrices with results used by different researchers is also presented in this study. Finally, all this information is summarized and comprehended in order to assist the readers to understand the motivation and methodology of various deep learning models, as well as exploring potential solutions to future challenges in LV segmentation.

Multimodality imaging of a cardiac paraganglioma: A case report

Cardiac paragangliomas (PGLs) are rare extra-adrenal tumors that arise from chromaffin cells of the sympathetic ganglia. PGLs are often diagnosed incidentally, in the absence of symptoms, or with symptoms related to cardiovascular dysfunction. Cardiac computed tomography (CCT) and cardiac magnetic resonance (CMR) can be used to accurately determine the lesion morphology and position as well as providing detailed tissue characterization. A multimodal imaging approach, not yet standardized, could be useful either in diagnosis and monitoring or in treatment planning. In the case reported here, CCT and CMR were performed to define lesion anatomy, and a reconstruction was generated using cinematic rendering (CR) to characterize the PGL angioarchitecture.

Learning Deep Representations of Cardiac Structures for 4D Cine MRI Image Segmentation through Semi-Supervised Learning

Learning good data representations for medical imaging tasks ensures the preservation of relevant information and the removal of irrelevant information from the data to improve the interpretability of the learned features. In this paper, we propose a semi-supervised model-namely, combine-all in semi-supervised learning (CqSL)-to demonstrate the power of a simple combination of a disentanglement block, variational autoencoder (VAE), generative adversarial network (GAN), and a conditioning layer-based reconstructor for performing two important tasks in medical imaging: segmentation and reconstruction. Our work is motivated by the recent progress in image segmentation using semi-supervised learning (SSL), which has shown good results with limited labeled data and large amounts of unlabeled data. A disentanglement block decomposes an input image into a domain-invariant spatial factor and a domain-specific non-spatial factor. We assume that medical images acquired using multiple scanners (different domain information) share a common spatial space but differ in non-spatial space (intensities, contrast, etc.). Hence, we utilize our spatial information to generate segmentation masks from unlabeled datasets using a generative adversarial network (GAN). Finally, to reconstruct the original image, our conditioning layer-based reconstruction block recombines spatial information with random non-spatial information sampled from the generative models. Our ablation study demonstrates the benefits of disentanglement in holding domain-invariant (spatial) as well as domain-specific (non-spatial) information with high accuracy. We further apply a structured L 2 similarity ( S L 2 SIM ) loss along with a mutual information minimizer (MIM) to improve the adversarially trained generative models for better reconstruction. Experimental results achieved on the STACOM 2017 ACDC cine cardiac magnetic resonance (MR) dataset suggest that our proposed (CqSL) model outperforms fully supervised and semi-supervised models, achieving an 83.2% performance accuracy even when using only 1% labeled data. We hypothesize that our proposed model has the potential to become an efficient semantic segmentation tool that may be used for domain adaptation in data-limited medical imaging scenarios, where annotations are expensive. Code, and experimental configurations will be made available publicly.

Computational Analysis of Cardiac Contractile Function

PURPOSE OF REVIEW

Heart failure results in the high incidence and mortality all over the world. Mechanical properties of myocardium are critical determinants of cardiac function, with regional variations in myocardial contractility demonstrated within infarcted ventricles. Quantitative assessment of cardiac contractile function is therefore critical to identify myocardial infarction for the early diagnosis and therapeutic intervention.

RECENT FINDINGS

Current advancement of cardiac functional assessments is in pace with the development of imaging techniques. The methods tailored to advanced imaging have been widely used in cardiac magnetic resonance, echocardiography, and optical microscopy. In this review, we introduce fundamental concepts and applications of representative methods for each imaging modality used in both fundamental research and clinical investigations. All these methods have been designed or developed to quantify time-dependent 2-dimensional (2D) or 3D cardiac mechanics, holding great potential to unravel global or regional myocardial deformation and contractile function from end-systole to end-diastole. Computational methods to assess cardiac contractile function provide a quantitative insight into the analysis of myocardial mechanics during cardiac development, injury, and remodeling.

Comparative study of OCTA algorithms with a high-sensitivity multi-contrast Jones matrix OCT system for human skin imaging

The multi-contrast Jones matrix OCT (JMT) system can not only improve the tissue-specific contrast but also enhance the sensitivity of detecting flow, which is gaining increasing attention. However, for the JMT system, there is currently a lack of studies evaluating and guiding the selection of suitable angiography algorithms to map the most appealing quality of angiograms for clinical use. In this paper, by a homemade high-sensitivity multi-contrast JMT system based 200 kHz swept source, the performance of two complex-signal-based OCTA methods that are insensitive to phase instability and one amplitude-signal-based OCTA method are compared for in-vivo imaging of human skin qualitatively and quantitatively. Six metrics, including vascular connectivity, image contrast-to-noise ratio, image signal-to-noise ratio, vessel diameter index, blood vessel density, and processing time, are assessed. The results show that the vascular networks processed by all OCTA methods and the texture of skin could be visualized simultaneously and markedly. Additionally, the complex-signal-based OCTA methods successfully suppress phase instabilities and even outperform the amplitude-signal-based OCTA algorithm in some indicators. This paper has a certain guiding significance for selecting an appropriate angiography algorithm and expanding the application field with this system.

Sequential shape similarity for active contour based left ventricle segmentation in cardiac cine MR image

Delineation of the boundaries of the Left Ventricle (LV) in cardiac Magnetic Resonance Images (MRI) is a hot topic due to its important diagnostic power. In this paper, an approach is proposed to extract the LV in a sequence of MR images. In the proposed paper, all images in the sequence are segmented simultaneously and the shape of the LV in each image is supposed to be similar to that of the LV in nearby images in the sequence. We coined the novel shape similarity constraint, and it is called sequential shape similarity (SSS in short). The proposed segmentation method takes the Active Contour Model as the base model and our previously proposed Gradient Vector Convolution (GVC) external force is also adopted. With the SSS constraint, the snake contour can accurately delineate the LV boundaries. We evaluate our method on two cardiac MRI datasets and the Mean Absolute Distance (MAD) metric and the Hausdorff Distance (HD) metric demonstrate that the proposed approach has good performance on segmenting the boundaries of the LV.

Learning-Based Regularization for Cardiac Strain Analysis via Domain Adaptation

Reliable motion estimation and strain analysis using 3D+ time echocardiography (4DE) for localization and characterization of myocardial injury is valuable for early detection and targeted interventions. However, motion estimation is difficult due to the low-SNR that stems from the inherent image properties of 4DE, and intelligent regularization is critical for producing reliable motion estimates. In this work, we incorporated the notion of domain adaptation into a supervised neural network regularization framework. We first propose a semi-supervised Multi-Layered Perceptron (MLP) network with biomechanical constraints for learning a latent representation that is shown to have more physiologically plausible displacements. We extended this framework to include a supervised loss term on synthetic data and showed the effects of biomechanical constraints on the network's ability for domain adaptation. We validated the semi-supervised regularization method on in vivo data with implanted sonomicrometers. Finally, we showed the ability of our semi-supervised learning regularization approach to identify infarct regions using estimated regional strain maps with good agreement to manually traced infarct regions from postmortem excised hearts.

Tissue clearing and imaging methods for cardiovascular development

Tissue imaging in 3D using visible light is limited and various clearing techniques were developed to increase imaging depth, but none provides universal solution for all tissues at all developmental stages. In this review, we focus on different tissue clearing methods for 3D imaging of heart and vasculature, based on chemical composition (solvent-based, simple immersion, hyperhydration, and hydrogel embedding techniques). We discuss in detail compatibility of various tissue clearing techniques with visualization methods: fluorescence preservation, immunohistochemistry, nuclear staining, and fluorescent dyes vascular perfusion. We also discuss myocardium visualization using autofluorescence, tissue shrinking, and expansion. Then we overview imaging methods used to study cardiovascular system and live imaging. We discuss heart and vessels segmentation methods and image analysis. The review covers the whole process of cardiovascular system 3D imaging, starting from tissue clearing and its compatibility with various visualization methods to the types of imaging methods and resulting image analysis.

Recovering from missing data in population imaging - Cardiac MR image imputation via conditional generative adversarial nets

Accurate ventricular volume measurements are the primary indicators of normal/abnor- mal cardiac function and are dependent on the Cardiac Magnetic Resonance (CMR) volumes being complete. However, missing or unusable slices owing to the presence of image artefacts such as respiratory or motion ghosting, aliasing, ringing and signal loss in CMR sequences, significantly hinder accuracy of anatomical and functional cardiac quantification, and recovering from those is insufficiently addressed in population imaging. In this work, we propose a new robust approach, coined Image Imputation Generative Adversarial Network (I2-GAN), to learn key features of cardiac short axis (SAX) slices near missing information, and use them as conditional variables to infer missing slices in the query volumes. In I2-GAN, the slices are first mapped to latent vectors with position features through a regression net. The latent vector corresponding to the desired position is then projected onto the slice manifold, conditioned on intensity features through a generator net. The generator comprises residual blocks with normalisation layers that are modulated with auxiliary slice information, enabling propagation of fine details through the network. In addition, a multi-scale discriminator was implemented, along with a discriminator-based feature matching loss, to further enhance performance and encourage the synthesis of visually realistic slices. Experimental results show that our method achieves significant improvements over the state-of-the-art, in missing slice imputation for CMR, with an average SSIM of 0.872. Linear regression analysis yields good agreement between reference and imputed CMR images for all cardiac measurements, with correlation coefficients of 0.991 for left ventricular volume, 0.977 for left ventricular mass and 0.961 for right ventricular volume.

Uncertainty-aware domain alignment for anatomical structure segmentation

Automatic and accurate segmentation of anatomical structures on medical images is crucial for detecting various potential diseases. However, the segmentation performance of established deep neural networks may degenerate on different modalities or devices owing to the significant difference across the domains, a problem known as domain shift. In this work, we propose an uncertainty-aware domain alignment framework to address the domain shift problem in the cross-domain Unsupervised Domain Adaptation (UDA) task. Specifically, we design an Uncertainty Estimation and Segmentation Module (UESM) to obtain the uncertainty map estimation. Then, a novel Uncertainty-aware Cross Entropy (UCE) loss is proposed to leverage the uncertainty information to boost the segmentation performance on highly uncertain regions. To further improve the performance in the UDA task, an Uncertainty-aware Self-Training (UST) strategy is developed to choose the optimal target samples by uncertainty guidance. In addition, the Uncertainty Feature Recalibration Module (UFRM) is applied to enforce the framework to minimize the cross-domain discrepancy. The proposed framework is evaluated on a private cross-device Optical Coherence Tomography (OCT) dataset and a public cross-modality cardiac dataset released by MMWHS 2017. Extensive experiments indicate that the proposed UESM is both efficient and effective for the uncertainty estimation in the UDA task, achieving state-of-the-art performance on both cross-modality and cross-device datasets.

Left ventricle segmentation using a Bayesian approach with distance dependent shape priors

We propose a method for segmentation of the left ventricle in magnetic resonance cardiac images. The framework consists of an initial Bayesian segmentation of the central slice of the volume. This segmentation is used to locate a shape prior for the LV myocardial tissue. This shape prior is determined using the fact that the myocardium is approximately annular as seen in the short-axis. Then a second Bayesian segmentation is performed to obtain the final result. This procedure is repeated for the rest of the slices. An extrapolation of the area of the LV is used to determine a stopping criterion. The method was evaluated on the databases of the Cardiac Atlas project. Our results demonstrate a suitable accuracy for myocardial segmentation (≈0.8 Dice's coefficient). For the endocardium and the epicardium the Dice's coefficients are 0.94 and 0.9 respectively. The accuracy was also evaluated in terms of the Hausdorff distance and the average distance. For the myocardium we obtain 8 mm and 2 mm respectively. Our results demonstrate the capability and merits of the proposed method to estimate the structure of the LV. The method requires minimal user input and generates results with quality comparable to more complex approaches. This paper suggests a new efficient approach for automatic LV quantification based on a Bayesian technique with shape priors with errors comparable to state-of-the-art techniques.

A New Framework for Performing Cardiac Strain Analysis from Cine MRI Imaging in Mice

Cardiac magnetic resonance (MR) imaging is one of the most rigorous form of imaging to assess cardiac function in vivo. Strain analysis allows comprehensive assessment of diastolic myocardial function, which is not indicated by measuring systolic functional parameters using with a normal cine imaging module. Due to the small heart size in mice, it is not possible to perform proper tagged imaging to assess strain. Here, we developed a novel deep learning approach for automated quantification of strain from cardiac cine MR images. Our framework starts by an accurate localization of the LV blood pool center-point using a fully convolutional neural network (FCN) architecture. Then, a region of interest (ROI) that contains the LV is extracted from all heart sections. The extracted ROIs are used for the segmentation of the LV cavity and myocardium via a novel FCN architecture. For strain analysis, we developed a Laplace-based approach to track the LV wall points by solving the Laplace equation between the LV contours of each two successive image frames over the cardiac cycle. Following tracking, the strain estimation is performed using the Lagrangian-based approach. This new automated system for strain analysis was validated by comparing the outcome of these analysis with the tagged MR images from the same mice. There were no significant differences between the strain data obtained from our algorithm using cine compared to tagged MR imaging. Furthermore, we demonstrated that our new algorithm can determine the strain differences between normal and diseased hearts.

Left ventricle automatic segmentation in cardiac MRI using a combined CNN and U-net approach

Cardiovascular diseases can be effectively prevented from worsening through early diagnosis. To this end, various methods have been proposed to detect the disease source by analyzing cardiac magnetic resonance images (MRI), wherein left ventricular segmentation plays an indispensable role. However, since the left ventricle (LV) is easily confused with other regions in cardiac MRI, segmentation of the LV is a challenging problem. To address this issue, we propose a composite model combining CNN and U-net to accurately segment the LV. In our model, CNN is used to locate the region of interest (ROI) and the U-net network achieve segmentation of LV. We used the cardiac MRI datasets of the MICCAI 2009 left ventricular segmentation challenge to train and test our model and demonstrated the accuracy and robustness of the proposed model. The proposed model achieved state-of-the-art results. The metrics are Dice metric (DM), volumetric overlap error (VOE) and Hausdorff distance (HD), in which DM reaches 0.951, VOE reaches 0.053 and HD reaches 3.641.

A deep learning-based approach for automatic segmentation and quantification of the left ventricle from cardiac cine MR images

Cardiac MRI has been widely used for noninvasive assessment of cardiac anatomy and function as well as heart diagnosis. The estimation of physiological heart parameters for heart diagnosis essentially require accurate segmentation of the Left ventricle (LV) from cardiac MRI. Therefore, we propose a novel deep learning approach for the automated segmentation and quantification of the LV from cardiac cine MR images. We aim to achieve lower errors for the estimated heart parameters compared to the previous studies by proposing a novel deep learning segmentation method. Our framework starts by an accurate localization of the LV blood pool center-point using a fully convolutional neural network (FCN) architecture called FCN1. Then, a region of interest (ROI) that contains the LV is extracted from all heart sections. The extracted ROIs are used for the segmentation of LV cavity and myocardium via a novel FCN architecture called FCN2. The FCN2 network has several bottleneck layers and uses less memory footprint than conventional architectures such as U-net. Furthermore, a new loss function called radial loss that minimizes the distance between the predicted and true contours of the LV is introduced into our model. Following myocardial segmentation, functional and mass parameters of the LV are estimated. Automated Cardiac Diagnosis Challenge (ACDC-2017) dataset was used to validate our framework, which gave better segmentation, accurate estimation of cardiac parameters, and produced less error compared to other methods applied on the same dataset. Furthermore, we showed that our segmentation approach generalizes well across different datasets by testing its performance on a locally acquired dataset. To sum up, we propose a deep learning approach that can be translated into a clinical tool for heart diagnosis.

Spatio-Temporal Segmentation in 3-D Echocardiographic Sequences Using Fractional Brownian Motion

An important aspect for an improved cardiac functional analysis is the accurate segmentation of the left ventricle (LV). A novel approach for fully-automated segmentation of the LV endocardium and epicardium contours is presented. This is mainly based on the natural physical characteristics of the LV shape structure. Both sides of the LV boundaries exhibit natural elliptical curvatures by having details on various scales, i.e. exhibiting fractal-like characteristics. The fractional Brownian motion (fBm), which is a non-stationary stochastic process, integrates well with the stochastic nature of ultrasound echoes. It has the advantage of representing a wide range of non-stationary signals and can quantify statistical local self-similarity throughout the time-sequence ultrasound images. The locally characterized boundaries of the fBm segmented LV were further iteratively refined using global information by means of second-order moments. The method is benchmarked using synthetic 3D+time echocardiographic sequences for normal and different ischemic cardiomyopathy, and results compared with state-of-the-art LV segmentation. Furthermore, the framework was validated against real data from canine cases with expert-defined segmentations and demonstrated improved accuracy. The fBm-based segmentation algorithm is fully automatic and has the potential to be used clinically together with 3D echocardiography for improved cardiovascular disease diagnosis.

Reconstruction techniques for cardiac cine MRI

The present survey describes the state-of-the-art techniques for dynamic cardiac magnetic resonance image reconstruction. Additionally, clinical relevance, main challenges, and future trends of this image modality are outlined. Thus, this paper aims to provide a general vision about cine MRI as the standard procedure in functional evaluation of the heart, focusing on technical methodologies.

Automatic skin lesion area determination of basal cell carcinoma using optical coherence tomography angiography and a skeletonization approach: Preliminary results

Cutaneous blood flow plays a key role in numerous physiological and pathological processes and has significant potential to be used as a biomarker to diagnose skin diseases such as basal cell carcinoma (BCC). The determination of the lesion area and vascular parameters within it, such as vessel density, is essential for diagnosis, surgical treatment and follow-up procedures. Here, an automatic skin lesion area determination algorithm based on optical coherence tomography angiography (OCTA) images is presented for the first time. The blood vessels are segmented within the OCTA images and then skeletonized. Subsequently, the skeleton is searched over the volume and numerous quantitative vascular parameters are calculated. The vascular density is then used to segment the lesion area. The algorithm is tested on both nodular and superficial BCC, and comparing with dermatological and histological results, the proposed method provides an accurate, non-invasive, quantitative and automatic tool for BCC lesion area determination.

Quantitative CMR population imaging on 20,000 subjects of the UK Biobank imaging study: LV/RV quantification pipeline and its evaluation

Population imaging studies generate data for developing and implementing personalised health strategies to prevent, or more effectively treat disease. Large prospective epidemiological studies acquire imaging for pre-symptomatic populations. These studies enable the early discovery of alterations due to impending disease, and enable early identification of individuals at risk. Such studies pose new challenges requiring automatic image analysis. To date, few large-scale population-level cardiac imaging studies have been conducted. One such study stands out for its sheer size, careful implementation, and availability of top quality expert annotation; the UK Biobank (UKB). The resulting massive imaging datasets (targeting ca. 100,000 subjects) has put published approaches for cardiac image quantification to the test. In this paper, we present and evaluate a cardiac magnetic resonance (CMR) image analysis pipeline that properly scales up and can provide a fully automatic analysis of the UKB CMR study. Without manual user interactions, our pipeline performs end-to-end image analytics from multi-view cine CMR images all the way to anatomical and functional bi-ventricular quantification. All this, while maintaining relevant quality controls of the CMR input images, and resulting image segmentations. To the best of our knowledge, this is the first published attempt to fully automate the extraction of global and regional reference ranges of all key functional cardiovascular indexes, from both left and right cardiac ventricles, for a population of 20,000 subjects imaged at 50 time frames per subject, for a total of one million CMR volumes. In addition, our pipeline provides 3D anatomical bi-ventricular models of the heart. These models enable the extraction of detailed information of the morphodynamics of the two ventricles for subsequent association to genetic, omics, lifestyle habits, exposure information, and other information provided in population imaging studies. We validated our proposed CMR analytics pipeline against manual expert readings on a reference cohort of 4620 subjects with contour delineations and corresponding clinical indexes. Our results show broad significant agreement between the manually obtained reference indexes, and those automatically computed via our framework. 80.67% of subjects were processed with mean contour distance of less than 1 pixel, and 17.50% with mean contour distance between 1 and 2 pixels. Finally, we compare our pipeline with a recently published approach reporting on UKB data, and based on deep learning. Our comparison shows similar performance in terms of segmentation accuracy with respect to human experts.

Automatic quantification of the LV function and mass: A deep learning approach for cardiovascular MRI

OBJECTIVE

This paper proposes a novel approach for automatic left ventricle (LV) quantification using convolutional neural networks (CNN).

METHODS

The general framework consists of one CNN for detecting the LV, and another for tissue classification. Also, three new deep learning architectures were proposed for LV quantification. These new CNNs introduce the ideas of sparsity and depthwise separable convolution into the U-net architecture, as well as, a residual learning strategy level-to-level. To this end, we extend the classical U-net architecture and use the generalized Jaccard distance as optimization objective function.

RESULTS

The CNNs were trained and evaluated with 140 patients from two public cardiovascular magnetic resonance datasets (Sunnybrook and Cardiac Atlas Project) by using a 5-fold cross-validation strategy. Our results demonstrate a suitable accuracy for myocardial segmentation ( ∼ 0.9 Dice's coefficient), and a strong correlation with the most relevant physiological measures: 0.99 for end-diastolic and end-systolic volume, 0.97 for the left myocardial mass, 0.95 for the ejection fraction and 0.93 for the stroke volume and cardiac output.

CONCLUSION

Our simulation and clinical evaluation results demonstrate the capability and merits of the proposed CNN to estimate different structural and functional features such as LV mass and EF which are commonly used for both diagnosis and treatment of different pathologies.

SIGNIFICANCE

This paper suggests a new approach for automatic LV quantification based on deep learning where errors are comparable to the inter- and intra-operator ranges for manual contouring.

Deep Learning-based Method for Fully Automatic Quantification of Left Ventricle Function from Cine MR Images: A Multivendor, Multicenter Study

Purpose To develop a deep learning-based method for fully automated quantification of left ventricular (LV) function from short-axis cine MR images and to evaluate its performance in a multivendor and multicenter setting. Materials and Methods This retrospective study included cine MRI data sets obtained from three major MRI vendors in four medical centers from 2008 to 2016. Three convolutional neural networks (CNNs) with the U-NET architecture were trained on data sets of increasing variability: (a) a single-vendor, single-center, homogeneous cohort of 100 patients (CNN1); (b) a single-vendor, multicenter, heterogeneous cohort of 200 patients (CNN2); and (c) a multivendor, multicenter, heterogeneous cohort of 400 patients (CNN3). All CNNs were tested on an independent multivendor, multicenter data set of 196 patients. CNN performance was evaluated with respect to the manual annotations from three experienced observers in terms of (a) LV detection accuracy, (b) LV segmentation accuracy, and (c) LV functional parameter accuracy. Automatic and manual results were compared with the paired Wilcoxon test, Pearson correlation, and Bland-Altman analysis. Results CNN3 achieved the highest performance on the independent testing data set. The average perpendicular distance compared with manual analysis was 1.1 mm ± 0.3 for CNN3, compared with 1.5 mm ± 1.0 for CNN1 (P < .05) and 1.3 mm ± 0.6 for CNN2 (P < .05). The LV function parameters derived from CNN3 showed a high correlation (r² ≥ 0.98) and agreement with those obtained by experts for data sets from different vendors and centers. Conclusion A deep learning-based method trained on a data set with high variability can achieve fully automated and accurate cine MRI analysis on multivendor, multicenter cine MRI data. © RSNA, 2018 See also the editorial by Colletti in this issue.

Cardiac Motion Evolution Model for Analysis of Functional Changes Using Tensor Decomposition and Cross-Sectional Data

Cardiac disease can reduce the ability of the ventricles to function well enough to sustain long-term pumping efficiency. Recent advances in cardiac motion tracking have led to improvements in the analysis of cardiac function. We propose a method to study cohort effects related to age with respect to cardiac function. The proposed approach makes use of a recent method for describing cardiac motion of a given subject using a polyaffine model, which gives a compact parameterization that reliably and accurately describes the cardiac motion across populations. Using this method, a data tensor of motion parameters is extracted for a given population. The partial least squares method for higher order arrays is used to build a model to describe the motion parameters with respect to age, from which a model of motion given age is derived. Based on the cross-sectional statistical analysis with the data tensor of each subject treated as an observation along time, the left ventricular motion over time of Tetralogy of Fallot patients is analysed to understand the temporal evolution of functional abnormalities in this population compared to healthy motion dynamics.

Semi-automatic Methods for Airway and Adjacent Vessel Measurement in Bronchiectasis Patterns in Lung HRCT Images of Cystic Fibrosis Patients

Airway and vessel characterization of bronchiectasis patterns in lung high-resolution computed tomography (HRCT) images of cystic fibrosis (CF) patients is very important to compute the score of disease severity. We propose a hybrid and evolutionary optimized threshold and model-based method for characterization of airway and vessel in lung HRCT images of CF patients. First, the initial model of airway and vessel is obtained using the enhanced threshold-based method. Then, the model is fitted to the actual image by optimizing its parameters using particle swarm optimization (PSO) evolutionary algorithm. The experimental results demonstrated the outperformance of the proposed method over its counterpart in R-squared, mean and variance of error, and run time. Moreover, the proposed method outperformed its counterpart for airway inner diameter/vessel diameter (AID/VD) and airway wall thickness/vessel diameter (AWT/VD) biomarkers in R-squared and slope of regression analysis.

Statistical shape modeling of the left ventricle: myocardial infarct classification challenge

Statistical shape modeling is a powerful tool for visualizing and quantifying geometric and functional patterns of the heart. After myocardial infarction (MI), the left ventricle typically remodels in response to physiological challenges. Several methods have been proposed in the literature to describe statistical shape changes. Which method best characterizes left ventricular remodeling after MI is an open research question. A better descriptor of remodeling is expected to provide a more accurate evaluation of disease status in MI patients. We therefore designed a challenge to test shape characterization in MI given a set of three-dimensional left ventricular surface points. The training set comprised 100 MI patients, and 100 asymptomatic volunteers (AV). The challenge was initiated in 2015 at the Statistical Atlases and Computational Models of the Heart workshop, in conjunction with the MICCAI conference. The training set with labels was provided to participants, who were asked to submit the likelihood of MI from a different (validation) set of 200 cases (100 AV and 100 MI). Sensitivity, specificity, accuracy and area under the receiver operating characteristic curve were used as the outcome measures. The goals of this challenge were to (1) establish a common dataset for evaluating statistical shape modeling algorithms in MI, and (2) test whether statistical shape modeling provides additional information characterizing MI patients over standard clinical measures. Eleven groups with a wide variety of classification and feature extraction approaches participated in this challenge. All methods achieved excellent classification results with accuracy ranges from 0.83 to 0.98. The areas under the receiver operating characteristic curves were all above 0.90. Four methods showed significantly higher performance than standard clinical measures. The dataset and software for evaluation are available from the Cardiac Atlas Project website1.

A Meshfree Representation for Cardiac Medical Image Computing

The prominent advantage of meshfree method, is the way to build the representation of computational domain, based on the nodal points without any explicit meshing connectivity. Therefore, meshfree method can conveniently process the numerical computation inside interested domains with large deformation or inhomogeneity. In this paper, we adopt the idea of meshfree representation into cardiac medical image analysis in order to overcome the difficulties caused by large deformation and inhomogeneous materials of the heart. In our implementation, as element-free Galerkin method can efficiently build a meshfree representation using its shape function with moving least square fitting, we apply this meshfree method to handle large deformation or inhomogeneity for solving cardiac segmentation and motion tracking problems. We evaluate the performance of meshfree representation on a synthetic heart data and an in-vivo cardiac MRI image sequence. Results showed that the error of our framework against the ground truth was 0.1189 ± 0.0672 while the error of the traditional FEM was 0.1793 ± 0.1166. The proposed framework has minimal consistency constraints, handling large deformation and material discontinuities are simple and efficient, and it provides a way to avoid the complicated meshing procedures while preserving the accuracy with a relatively small number of nodes.

Atlas-Based Computational Analysis of Heart Shape and Function in Congenital Heart Disease

Approximately 1% of all babies are born with some form of congenital heart defect. Many serious forms of CHD can now be surgically corrected after birth, which has led to improved survival into adulthood. However, many patients require serial monitoring to evaluate progression of heart failure and determine timing of interventions. Accurate multidimensional quantification of regional heart shape and function is required for characterizing these patients. A computational atlas of single ventricle and biventricular heart shape and function enables quantification of remodeling in terms of z scores in relation to specific reference populations. Progression of disease can then be monitored effectively by longitudinal evaluation of z scores. A biomechanical analysis of cardiac function in relation to population variation enables investigation of the underlying mechanisms for developing pathology. Here, we summarize recent progress in this field, with examples in single ventricle and biventricular congenital pathologies.

Automated localization and segmentation techniques for B-mode ultrasound images: A review

B-mode ultrasound imaging is used extensively in medicine. Hence, there is a need to have efficient segmentation tools to aid in computer-aided diagnosis, image-guided interventions, and therapy. This paper presents a comprehensive review on automated localization and segmentation techniques for B-mode ultrasound images. The paper first describes the general characteristics of B-mode ultrasound images. Then insight on the localization and segmentation of tissues is provided, both in the case in which the organ/tissue localization provides the final segmentation and in the case in which a two-step segmentation process is needed, due to the desired boundaries being too fine to locate from within the entire ultrasound frame. Subsequenly, examples of some main techniques found in literature are shown, including but not limited to shape priors, superpixel and classification, local pixel statistics, active contours, edge-tracking, dynamic programming, and data mining. Ten selected applications (abdomen/kidney, breast, cardiology, thyroid, liver, vascular, musculoskeletal, obstetrics, gynecology, prostate) are then investigated in depth, and the performances of a few specific applications are compared. In conclusion, future perspectives for B-mode based segmentation, such as the integration of RF information, the employment of higher frequency probes when possible, the focus on completely automatic algorithms, and the increase in available data are discussed.

Automatic initialization and quality control of large-scale cardiac MRI segmentations

Continuous advances in imaging technologies enable ever more comprehensive phenotyping of human anatomy and physiology. Concomitant reduction of imaging costs has resulted in widespread use of imaging in large clinical trials and population imaging studies. Magnetic Resonance Imaging (MRI), in particular, offers one-stop-shop multidimensional biomarkers of cardiovascular physiology and pathology. A wide range of analysis methods offer sophisticated cardiac image assessment and quantification for clinical and research studies. However, most methods have only been evaluated on relatively small databases often not accessible for open and fair benchmarking. Consequently, published performance indices are not directly comparable across studies and their translation and scalability to large clinical trials or population imaging cohorts is uncertain. Most existing techniques still rely on considerable manual intervention for the initialization and quality control of the segmentation process, becoming prohibitive when dealing with thousands of images. The contributions of this paper are three-fold. First, we propose a fully automatic method for initializing cardiac MRI segmentation, by using image features and random forests regression to predict an initial position of the heart and key anatomical landmarks in an MRI volume. In processing a full imaging database, the technique predicts the optimal corrective displacements and positions in relation to the initial rough intersections of the long and short axis images. Second, we introduce for the first time a quality control measure capable of identifying incorrect cardiac segmentations with no visual assessment. The method uses statistical, pattern and fractal descriptors in a random forest classifier to detect failures to be corrected or removed from subsequent statistical analysis. Finally, we validate these new techniques within a full pipeline for cardiac segmentation applicable to large-scale cardiac MRI databases. The results obtained based on over 1200 cases from the Cardiac Atlas Project show the promise of fully automatic initialization and quality control for population studies.

Dynamic updating atlas for heart segmentation with a nonlinear field-based model

BACKGROUND

Segmentation of cardiac computed tomography (CT) images is an effective method for assessing the dynamic function of the heart and lungs. In the atlas-based heart segmentation approach, the quality of segmentation usually relies upon atlas images, and the selection of those reference images is a key step. The optimal goal in this selection process is to have the reference images as close to the target image as possible.

METHODS

This study proposes an atlas dynamic update algorithm using a scheme of nonlinear deformation field. The proposed method is based on the features among double-source CT (DSCT) slices. The extraction of these features will form a base to construct an average model and the created reference atlas image is updated during the registration process. A nonlinear field-based model was used to effectively implement a 4D cardiac segmentation.

RESULTS

The proposed segmentation framework was validated with 14 4D cardiac CT sequences. The algorithm achieved an acceptable accuracy (1.0-2.8 mm).

CONCLUSION

Our proposed method that combines a nonlinear field-based model and dynamic updating atlas strategies can provide an effective and accurate way for whole heart segmentation. The success of the proposed method largely relies on the effective use of the prior knowledge of the atlas and the similarity explored among the to-be-segmented DSCT sequences.

Segmentation of Left and Right Ventricles in Cardiac MRI Using Active Contours

Segmentation of left and right ventricles plays a crucial role in quantitatively analyzing the global and regional information in the cardiac magnetic resonance imaging (MRI). In MRI, the intensity inhomogeneity and weak or blurred object boundaries are the problems, which makes it difficult for the intensity-based segmentation methods to properly delineate the regions of interests (ROI). In this paper, a hybrid signed pressure force function (SPF) is proposed, which yields both local and global image fitted differences in an additive fashion. A characteristic term is also introduced in the SPF function to restrict the contour within the ROI. The overlapping dice index and Hausdorff-Distance metrics have been used over cardiac datasets for quantitative validation. Using 2009 LV MICCAI validation dataset, the proposed method yields DSC values of 0.95 and 0.97 for endocardial and epicardial contours, respectively. Using 2012 RV MICCAI dataset, for the endocardial region, the proposed method yields DSC values of 0.97 and 0.90 and HD values of 8.51 and 7.67 for ED and ES, respectively. For the epicardial region, it yields DSC values of 0.92 and 0.91 and HD values of 6.47 and 9.34 for ED and ES, respectively. Results show its robustness in the segmentation application of the cardiac MRI.

Cardiac imaging: working towards fully-automated machine analysis & interpretation

INTRODUCTION

Non-invasive imaging plays a critical role in managing patients with cardiovascular disease. Although subjective visual interpretation remains the clinical mainstay, quantitative analysis facilitates objective, evidence-based management, and advances in clinical research. This has driven developments in computing and software tools aimed at achieving fully automated image processing and quantitative analysis. In parallel, machine learning techniques have been used to rapidly integrate large amounts of clinical and quantitative imaging data to provide highly personalized individual patient-based conclusions. Areas covered: This review summarizes recent advances in automated quantitative imaging in cardiology and describes the latest techniques which incorporate machine learning principles. The review focuses on the cardiac imaging techniques which are in wide clinical use. It also discusses key issues and obstacles for these tools to become utilized in mainstream clinical practice. Expert commentary: Fully-automated processing and high-level computer interpretation of cardiac imaging are becoming a reality. Application of machine learning to the vast amounts of quantitative data generated per scan and integration with clinical data also facilitates a move to more patient-specific interpretation. These developments are unlikely to replace interpreting physicians but will provide them with highly accurate tools to detect disease, risk-stratify, and optimize patient-specific treatment. However, with each technological advance, we move further from human dependence and closer to fully-automated machine interpretation.

Spatiotemporal Strategies for Joint Segmentation and Motion Tracking From Cardiac Image Sequences

Although accurate and robust estimations of the deforming cardiac geometry and kinematics from cine tomographic medical image sequences remain a technical challenge, they have significant clinical value. Traditionally, boundary or volumetric segmentation and motion estimation problems are considered as two sequential steps, even though the order of these processes can be different. In this paper, we present an integrated, spatiotemporal strategy for the simultaneous joint recovery of these two ill-posed problems. We use a mesh-free Galerkin formulation as the representation and computation platform, and adopt iterative procedures to solve the governing equations. Specifically, for each nodal point, the external driving forces are individually constructed through the integration of data-driven edginess measures, prior spatial distributions of myocardial tissues, temporal coherence of image-derived salient features, imaging/image-derived Eulerian velocity information, and cyclic motion model of myocardial behavior. The proposed strategy is accurate and very promising application results are shown from synthetic data, magnetic resonance (MR) phase contrast, tagging image sequences, and gradient echo cine MR image sequences.

Left ventricle segmentation via two-layer level sets with circular shape constraint

This paper proposes a circular shape constraint and a novel two-layer level set method for the segmentation of the left ventricle (LV) from short-axis magnetic resonance images without training any shape models. Since the shape of LV throughout the apex-base axis is close to a ring shape, we propose a circle fitting term in the level set framework to detect the endocardium. The circle fitting term imposes a penalty on the evolving contour from its fitting circle, and thereby handles quite well with issues in LV segmentation, especially the presence of outflow track in basal slices and the intensity overlap between TPM and the myocardium. To extract the whole myocardium, the circle fitting term is incorporated into two-layer level set method. The endocardium and epicardium are respectively represented by two specified level contours of the level set function, which are evolved by an edge-based and a region-based active contour model. The proposed method has been quantitatively validated on the public data set from MICCAI 2009 challenge on the LV segmentation. Experimental results and comparisons with state-of-the-art demonstrate the accuracy and robustness of our method.