A shape-space-based approach to tracking myocardial borders and quantifying regional left-ventricular function applied in echocardiography

Model-Agnostic Temporal Regularizer for Object Localization Using Motion Fields

Video analysis often requires locating and tracking target objects. In some applications, the localization system has access to the full video, which allows fine-grain motion information to be estimated. This paper proposes capturing this information through motion fields and using it to improve the localization results. The learned motion fields act as a model-agnostic temporal regularizer that can be used with any localization system based on keypoints. Unlike optical flow-based strategies, our motion fields are estimated from the model domain, based on the trajectories described by the object keypoints. Therefore, they are not affected by poor imaging conditions. The benefits of the proposed strategy are shown on three applications: 1) segmentation of cardiac magnetic resonance; 2) facial model alignment; and 3) vehicle tracking. In each case, combining popular localization methods with the proposed regularizer leads to improvement in overall accuracies and reduces gross errors.

Shape constraint function for artery tracking in ultrasound images

Ultrasound guided regional anesthesia (UGRA) has emerged as a powerful technique for pain management in the operating theatre. It uses ultrasound imaging to visualize anatomical structures, the needle insertion and the delivery of the anesthetic around the targeted nerve block. Detection of the nerves is a difficult task, however, due to the poor quality of the ultrasound images. Recent developments in pattern recognition and machine learning have heightened the need for computer aided systems in many applications. This type of system can improve UGRA practice. In many imaging situations nerves are not salient in images. Generally, practitioners rely on the arteries as key anatomical structures to confirm the positions of the nerves, making artery tracking an important aspect for UGRA procedure. However, artery tracking in a noisy environment is a challenging problem, due to the instability of the features. This paper proposes a new method for real-time artery tracking in ultrasound images. It is based on shape information to correct tracker location errors. A new objective function is proposed, which defines an artery as an elliptical shape, enabling its robust fitting in a noisy environment. This approach is incorporated in two well-known tracking algorithms, and shows a systematic improvement over the original trackers. Evaluations were performed on 71 videos of different axillary nerve blocks. The results obtained demonstrated the validity of the proposed method.

Comparative analysis of active contour and convolutional neural network in rapid left-ventricle volume quantification using echocardiographic imaging

In cardiology, ultrasound is often used to diagnose heart disease associated with myocardial infarction. This study aims to develop robust segmentation techniques for segmenting the left ventricle (LV) in ultrasound images to check myocardium movement during heartbeat. The proposed technique utilizes machine learning (ML) techniques such as the active contour (AC) and convolutional neural networks (CNNs) for segmentation. Medical experts determine the consistency between the proposed ML approach, which is a state-of-the-art deep learning method, and the manual segmentation approach. These methods are compared in terms of performance indicators such as the ventricular area (VA), ventricular maximum diameter (VMXD), ventricular minimum diameter (VMID), and ventricular long axis angle (AVLA) measurements. Furthermore, the Dice similarity coefficient, Jaccard index, and Hausdorff distance are measured to estimate the agreement of the LV segmented results between the automatic and visual approaches. The obtained results indicate that the proposed techniques for LV segmentation are useful and practical. There is no significant difference between the use of AC and CNN in image segmentation; however, the AC method could obtain comparable accuracy as the CNN method using less training data and less run-time.

One Shot Segmentation: Unifying Rigid Detection and Non-Rigid Segmentation Using Elastic Regularization

This paper proposes a novel approach for the non-rigid segmentation of deformable objects in image sequences, which is based on one-shot segmentation that unifies rigid detection and non-rigid segmentation using elastic regularization. The domain of application is the segmentation of a visual object that temporally undergoes a rigid transformation (e.g., affine transformation) and a non-rigid transformation (i.e., contour deformation). The majority of segmentation approaches to solve this problem are generally based on two steps that run in sequence: a rigid detection, followed by a non-rigid segmentation. In this paper, we propose a new approach, where both the rigid and non-rigid segmentation are performed in a single shot using a sparse low-dimensional manifold that represents the visual object deformations. Given the multi-modality of these deformations, the manifold partitions the training data into several patches, where each patch provides a segmentation proposal during the inference process. These multiple segmentation proposals are merged using the classification results produced by deep belief networks (DBN) that compute the confidence on each segmentation proposal. Thus, an ensemble of DBN classifiers is used for estimating the final segmentation. Compared to current methods proposed in the field, our proposed approach is advantageous in four aspects: (i) it is a unified framework to produce rigid and non-rigid segmentations; (ii) it uses an ensemble classification process, which can help the segmentation robustness; (iii) it provides a significant reduction in terms of the number of dimensions of the rigid and non-rigid segmentations search spaces, compared to current approaches that divide these two problems; and (iv) this lower dimensionality of the search space can also reduce the need for large annotated training sets to be used for estimating the DBN models. Experiments on the problem of left ventricle endocardial segmentation from ultrasound images, and lip segmentation from frontal facial images using the extended Cohn-Kanade (CK+) database, demonstrate the potential of the methodology through qualitative and quantitative evaluations, and the ability to reduce the search and training complexities without a significant impact on the segmentation accuracy.

A novel U-Net approach to segment the cardiac chamber in magnetic resonance images with ghost artifacts

OBJECTIVE

We propose a robust technique for segmenting magnetic resonance images of post-atrial septal occlusion intervention in the cardiac chamber.

METHODS

A variant of the U-Net architecture is used to perform atrial segmentation via a deep convolutional neural network, and we compare performance with the Kass snake model. It can be used to determine the surgical success of atrial septal occlusion (ASO) pre- and post- the implantation of the septal occluder, which is based on the volume restoration of the right atria (RA) and left atria (LA).

RESULTS

The method was evaluated on a test dataset containing 550 two-dimensional image slices, outperforming conventional active contouring regarding the Dice similarity coefficient, Jaccard index, and Hausdorff distance, and achieving segmentation in the presence of ghost artifacts that occlude the atrium outline. This problem has been unsolvable using traditional machine learning algorithm pertaining to active contouring via the Kass snake algorithm. Moreover, the proposed technique is closer to manual segmentation than the snakes active contour model in mean of atrial area (M-AA), mean of atrial maximum diameter (M-AMXD), mean atrial minimum diameter (M-AMID), and mean angle of the atrial long axis (M-AALA).

CONCLUSION

After segmentation, we compute the volume ratio of right to left atria, obtaining a smaller ratio that indicates better restoration. Hence, the proposed technique allows to evaluate the surgical success of atrial septal occlusion and may support diagnosis regarding the accurate evaluation of atrial septal defects before and after occlusion procedures.

Cardiac Chamber Segmentation Using Deep Learning on Magnetic Resonance Images from Patients Before and After Atrial Septal Occlusion Surgery

We propose a robust technique for segmenting magnetic resonance images of post-atrial septal occlusion intervention in the cardiac chamber. The technique can be used to determine the surgical outcomes of atrial septal defects before and after implantation of a septal occluder, which intends to provide volume restoration of the right and left atria. A variant of the U-Net architecture is used to perform atrial segmentation via a deep convolutional neural network. The method was evaluated on a dataset containing 550 two-dimensional image slices, outperforming conventional active contouring regarding the Dice similarity coefficient, Jaccard index, and Hausdorff distance, and achieving segmentation in the presence of ghost artifacts that occlude the atrium outline. Moreover, the proposed technique is closer to manual segmentation than the snakes active contour model. After segmentation, we computed the volume ratio of right to left atria, obtaining a smaller ratio that indicates better restoration. Hence, the proposed technique allows to evaluate the surgical success of atrial septal occlusion and may support diagnosis regarding the accurate evaluation of atrial septal defects before and after occlusion procedures.

DW-Net: A cascaded convolutional neural network for apical four-chamber view segmentation in fetal echocardiography

Fetal echocardiography (FE) is a widely used medical examination for early diagnosis of congenital heart disease (CHD). The apical four-chamber view (A4C) is an important view among early FE images. Accurate segmentation of crucial anatomical structures in the A4C view is a useful and important step for early diagnosis and timely treatment of CHDs. However, it is a challenging task due to several unfavorable factors: (a) artifacts and speckle noise produced by ultrasound imaging. (b) category confusion caused by the similarity of anatomical structures and variations of scanning angles. (c) missing boundaries. In this paper, we propose an end-to-end DW-Net for accurate segmentation of seven important anatomical structures in the A4C view. The network comprises two components: 1) a Dilated Convolutional Chain (DCC) for "gridding issue" reduction, multi-scale contextual information aggregation and accurate localization of cardiac chambers. 2) a W-Net for gaining more precise boundaries and yielding refined segmentation results. Extensive experiments of the proposed method on a dataset of 895 A4C views have demonstrated that DW-Net can achieve good segmentation results, including the Dice Similarity Coefficient (DSC) of 0.827, the Pixel Accuracy (PA) of 0.933, the AUC of 0.990 and it substantially outperformed some well-known segmentation methods. Our work was highly valued by experienced clinicians. The accurate and automatic segmentation of the A4C view using the proposed DW-Net can benefit further extractions of useful clinical indicators in early FE and improve the prenatal diagnostic accuracy and efficiency of CHDs.

Deformable multimodal registration for navigation in beating-heart cardiac surgery

PURPOSE

Minimally invasive beating-heart surgery is currently performed using endoscopes and without navigation. Registration of intraoperative ultrasound to a preoperative cardiac CT scan is a valuable step toward image-guided navigation.

METHODS

The registration was achieved by first extracting a representative point set from each ultrasound image in the sequence using a deformable registration. A template shape representing the cardiac chambers was deformed through a hierarchy of affine transformations to match each ultrasound image using a generalized expectation maximization algorithm. These extracted point sets were matched to the CT by exhaustively searching over a large number of precomputed slices of 3D geometry. The result is a similarity transformation mapping the intraoperative ultrasound to preoperative CT.

RESULTS

Complete data sets were acquired for four patients. Transesophageal echocardiography ultrasound sequences were deformably registered to a model of oriented points with a mean error of 2.3 mm. Ultrasound and CT scans were registered to a mean of 3 mm, which is comparable to the error of 2.8 mm expected by merging ultrasound registration with uncertainty of cardiac CT.

CONCLUSION

The proposed algorithm registered 3D CT with dynamic 2D intraoperative imaging. The algorithm aligned the images in both space and time, needing neither dynamic CT imaging nor intraoperative electrocardiograms. The accuracy was sufficient for navigation in thoracoscopically guided beating-heart surgery.

Deep Learning on Sparse Manifolds for Faster Object Segmentation

We propose a new combination of deep belief networks and sparse manifold learning strategies for the 2D segmentation of non-rigid visual objects. With this novel combination, we aim to reduce the training and inference complexities while maintaining the accuracy of machine learning-based non-rigid segmentation methodologies. Typical non-rigid object segmentation methodologies divide the problem into a rigid detection followed by a non-rigid segmentation, where the low dimensionality of the rigid detection allows for a robust training (i.e., a training that does not require a vast amount of annotated images to estimate robust appearance and shape models) and a fast search process during inference. Therefore, it is desirable that the dimensionality of this rigid transformation space is as small as possible in order to enhance the advantages brought by the aforementioned division of the problem. In this paper, we propose the use of sparse manifolds to reduce the dimensionality of the rigid detection space. Furthermore, we propose the use of deep belief networks to allow for a training process that can produce robust appearance models without the need of large annotated training sets. We test our approach in the segmentation of the left ventricle of the heart from ultrasound images and lips from frontal face images. Our experiments show that the use of sparse manifolds and deep belief networks for the rigid detection stage leads to segmentation results that are as accurate as the current state of the art, but with lower search complexity and training processes that require a small amount of annotated training data.

Contour Tracking with a Spatio-Temporal Intensity Moment

Standard edge detection operators such as the Laplacian of Gaussian and the gradient of Gaussian can be used to track contours in image sequences. When using edge operators, a contour, which is determined on a frame of the sequence, is simply used as a starting contour to locate the nearest contour on the subsequent frame. However, the strategy used to look for the nearest edge points may not work when tracking contours of non isolated gray level discontinuities. In these cases, strategies derived from the optical flow equation, which look for similar gray level distributions, appear to be more appropriate since these can work with a lower frame rate than that needed for strategies based on pure edge detection operators. However, an optical flow strategy tends to propagate the localization errors through the sequence and an additional edge detection procedure is essential to compensate for such a drawback. In this paper a spatio-temporal intensity moment is proposed which integrates the two basic functions of edge detection and tracking.

Realtime Automatic Assessment of Cardiac Function in Echocardiography

Assessment of cardiac function by echocardiography is challenging for nonexperts. In a patient with dyspnea, quantification of the mitral annular excursion (MAE) and velocities is important for the diagnosis of heart failure. The displacement of the atrioventricular (AV) plane is a good indicator of systolic left ventricular function, while the peak velocities give supplementary information about the systolic and diastolic function. By measuring these parameters automatically, a preliminary diagnosis can be given by the nonexpert. We propose an automatic algorithm to localize the mitral annular points in an apical four-chamber view and estimate the MAE, as well as the systolic, early diastolic, and late diastolic tissue peak velocities, by using a deformable ventricle model for orientation and tissue Doppler data for tracking. Automatic parameter estimates from 367 tissue Doppler recordings were compared to reference measurements by experienced cardiologists to assess the accuracy of the estimation, as well as the ability to correctly detect reduced MAE, which we defined as less than 10 mm. The dataset consisted of 200 recordings from a patient population and 167 healthy from a population study. When considering the average of the septal and lateral values, the estimation error for the MAE had a standard deviation of 2.1 mm, which was reduced to 1.9 mm when excluding recordings for which the automatic segmentation failed to locate the AV plane (41 recordings). The corresponding standard deviations for the peak velocities were around 1 cm/s. The classification of MAE was correct in 90% of the cases and had a sensitivity of 83% and a specificity of 92%. We conclude that the algorithm has good accuracy and note that the estimation error for the MAE was comparable to interobserver and methodology agreements reported in the literature.

Estimation of Large-Scale Organ Motion in B-Mode Ultrasound Image Sequences: A Survey

Reviewed here are methods developed for following (i.e., tracking) structures in medical B-mode ultrasound time sequences during large-scale motion. The resulting motion estimation problem and its key components are defined. The main tracking approaches are described, and their strengths and weaknesses are discussed. Existing motion estimation methods, tested on multiple in vivo sequences, are categorized with respect to their clinical applications, namely, cardiac, respiratory and muscular motion. A large number of works in this field had to be discarded as thorough validation of the results was missing. The remaining relevant works identified indicate the possibility of reaching an average tracking accuracy up to 1-2 mm. Real-time performance can be achieved using several methods. Yet only very few of these have progressed to clinical practice. The latest trends include incorporation of complementary and prior information. Advances are expected from common evaluation databases and 4-D ultrasound scanning technologies.

Automatic 3-D segmentation of endocardial border of the left ventricle from ultrasound images

The segmentation of the left ventricle (LV) is an important task to assess the cardiac function in ultrasound images of the heart. This paper presents a novel methodology for the segmentation of the LV in three-dimensional (3-D) echocardiographic images based on the probabilistic data association filter (PDAF). The proposed methodology begins by initializing a 3-D deformable model either semiautomatically, with user input, or automatically, and it comprises the following feature hierarchical approach: 1) edge detection in the vicinity of the surface (low-level features); 2) edge grouping to obtain potential LV surface patches (mid-level features); and 3) patch filtering using a shape-PDAF framework (high-level features). This method provides good performance accuracy in 20 echocardiographic volumes, and compares favorably with the state-of-the-art segmentation methodologies proposed in the recent literature.

Segmentation of uterine fibroid ultrasound images using a dynamic statistical shape model in HIFU therapy

Segmenting the lesion areas from ultrasound (US) images is an important step in the intra-operative planning of high-intensity focused ultrasound (HIFU). However, accurate segmentation remains a challenge due to intensity inhomogeneity, blurry boundaries in HIFU US images and the deformation of uterine fibroids caused by patient's breathing or external force. This paper presents a novel dynamic statistical shape model (SSM)-based segmentation method to accurately and efficiently segment the target region in HIFU US images of uterine fibroids. For accurately learning the prior shape information of lesion boundary fluctuations in the training set, the dynamic properties of stochastic differential equation and Fokker-Planck equation are incorporated into SSM (referred to as SF-SSM). Then, a new observation model of lesion areas (named to RPFM) in HIFU US images is developed to describe the features of the lesion areas and provide a likelihood probability to the prior shape given by SF-SSM. SF-SSM and RPFM are integrated into active contour model to improve the accuracy and robustness of segmentation in HIFU US images. We compare the proposed method with four well-known US segmentation methods to demonstrate its superiority. The experimental results in clinical HIFU US images validate the high accuracy and robustness of our approach, even when the quality of the images is unsatisfactory, indicating its potential for practical application in HIFU therapy.

Contour tracking in echocardiographic sequences via sparse representation and dictionary learning

This paper presents a dynamical appearance model based on sparse representation and dictionary learning for tracking both endocardial and epicardial contours of the left ventricle in echocardiographic sequences. Instead of learning offline spatiotemporal priors from databases, we exploit the inherent spatiotemporal coherence of individual data to constraint cardiac contour estimation. The contour tracker is initialized with a manual tracing of the first frame. It employs multiscale sparse representation of local image appearance and learns online multiscale appearance dictionaries in a boosting framework as the image sequence is segmented frame-by-frame sequentially. The weights of multiscale appearance dictionaries are optimized automatically. Our region-based level set segmentation integrates a spectrum of complementary multilevel information including intensity, multiscale local appearance, and dynamical shape prediction. The approach is validated on twenty-six 4D canine echocardiographic images acquired from both healthy and post-infarct canines. The segmentation results agree well with expert manual tracings. The ejection fraction estimates also show good agreement with manual results. Advantages of our approach are demonstrated by comparisons with a conventional pure intensity model, a registration-based contour tracker, and a state-of-the-art database-dependent offline dynamical shape model. We also demonstrate the feasibility of clinical application by applying the method to four 4D human data sets.

Automatic assessment of regional and global wall motion abnormalities in echocardiography images by nonlinear dimensionality reduction

PURPOSE

Identification and assessment of left ventricular (LV) global and regional wall motion (RWM) abnormalities are essential for clinical evaluation of various cardiovascular diseases. Currently, this evaluation is performed visually which is highly dependent on the training and experience of echocardiographers and thus is prone to considerable interobserver and intraobserver variability. This paper presents a new automatic method, based on nonlinear dimensionality reduction (NLDR) for global wall motion evaluation and also detection and classification of RWM abnormalities of LV wall in a three-point scale as follows: (1) normokinesia, (2) hypokinesia, and (3) akinesia.

METHODS

Isometric feature mapping (Isomap) is one of the most popular NLDR algorithms. In this paper, a modified version of Isomap algorithm, where image to image distance metric is computed using nonrigid registration, is applied on two-dimensional (2D) echocardiography images of one cycle of heart. By this approach, nonlinear information in these images is embedded in a 2D manifold and each image is characterized by a point on the constructed manifold. This new representation visualizes the relationship between these images based on LV volume changes. Then, a new global and regional quantitative index from the resultant manifold is proposed for global wall motion estimation and also classification of RWM of LV wall in a three-point scale. Obtained results by our method are quantitatively evaluated to those obtained visually by two experienced echocardiographers as the reference (gold standard) on 10 healthy volunteers and 14 patients.

RESULTS

Linear regression analysis between the proposed global quantitative index and the global wall motion score index and also with LV ejection fraction obtained by reference experienced echocardiographers resulted in the correlation coefficients of 0.85 and 0.90, respectively. Comparison between the proposed automatic RWM scoring and the reference visual scoring resulted in an absolute agreement of 82% and a relative agreement of 97%.

CONCLUSIONS

The proposed diagnostic method can be used as a useful tool as well as a reference visual assessment by experienced echocardiographers for global wall motion estimation and also classification of RWM abnormalities of LV wall in a three-point scale in clinical evaluations.

Histotripsy cardiac therapy system integrated with real-time motion correction

Histotripsy has shown promise in non-invasive cardiac therapy for neonatal and fetal applications. However, for cardiac applications in general, and especially in the adult heart, cardiac and respiratory motion may affect treatment accuracy and efficacy. In this article, we describe a histotripsy-mediated cardiac therapy system integrated with a fast motion tracking algorithm and treatment monitoring using ultrasound imaging. Motion tracking is performed by diamond search block matching in real-time ultrasound images using a reference image of the moving target, refined by Kalman filtering. As proof of feasibility, this algorithm was configured to track 2-D target motion and then electronically adjust the focus of a 1-MHz annular therapy array to correct for axial motion. This integrated motion tracking system is capable of sub-millimeter accuracy for displacements of 0-15 mm and velocities of 0-80 mm/s, with a maximum error less than 3 mm. Tissue phantom tests indicated that treatment efficiency and lesion size using motion tracking over displacements of 0-15 mm and velocities of 0-42 mm/s are comparable to those achieved when treating stationary targets. In vivo validation was conducted in an open-chest canine model, where the system provided 24 min of motion-corrected histotripsy therapy in the live beating heart, generating a targeted lesion on the atrial septum. Based on this proof of feasibility and the natural extension of these techniques to three dimensions, we anticipate a full motion correction system would be feasible and beneficial for non-invasive cardiac therapy.

Combining multiple dynamic models and deep learning architectures for tracking the left ventricle endocardium in ultrasound data

We present a new statistical pattern recognition approach for the problem of left ventricle endocardium tracking in ultrasound data. The problem is formulated as a sequential importance resampling algorithm such that the expected segmentation of the current time step is estimated based on the appearance, shape, and motion models that take into account all previous and current images and previous segmentation contours produced by the method. The new appearance and shape models decouple the affine and nonrigid segmentations of the left ventricle to reduce the running time complexity. The proposed motion model combines the systole and diastole motion patterns and an observation distribution built by a deep neural network. The functionality of our approach is evaluated using a dataset of diseased cases containing 16 sequences and another dataset of normal cases comprised of four sequences, where both sets present long axis views of the left ventricle. Using a training set comprised of diseased and healthy cases, we show that our approach produces more accurate results than current state-of-the-art endocardium tracking methods in two test sequences from healthy subjects. Using three test sequences containing different types of cardiopathies, we show that our method correlates well with interuser statistics produced by four cardiologists.

Automatic classification of left ventricular regional wall motion abnormalities in echocardiography images using nonrigid image registration

Identification and classification of left ventricular (LV) regional wall motion (RWM) abnormalities on echocardiograms has fundamental clinical importance for various cardiovascular disease assessments especially in ischemia. In clinical practice, this evaluation is still performed visually which is highly dependent on training and experience of the echocardiographers and therefore suffers from significant interobserver and intraobserver variability. This paper presents a new automatic technique, based on nonrigid image registration for classifying the RWM of LV in a three-point scale. In this algorithm, we register all images of one cycle of heart to a reference image (end-diastolic image) using a hierarchical parametric model. This model is based on an affine transformation for modeling the global LV motion and a B-spline free-form deformation transformation for modeling the local LV deformation. We consider image registration as a multiresolution optimization problem. Finally, a new regional quantitative index based on resultant parameters of the hierarchical transformation model is proposed for classifying RWM in a three-point scale. The results obtained by our method are quantitatively evaluated to those obtained by two experienced echocardiographers visually as gold standard on ten healthy volunteers and 14 patients (two apical views) and resulted in an absolute agreement of 83 % and a relative agreement of 99 %. Therefore, this diagnostic system can be used as a useful tool as well as reference visual assessment to classify RWM abnormalities in clinical evaluation.

Imaging techniques for cardiac strain and deformation: comparison of echocardiography, cardiac magnetic resonance and cardiac computed tomography

Myocardial function assessment is essential for determining the health of the myocardium. Global assessment of myocardial function is widely performed (by estimating the ejection fraction), but many common cardiac diseases initially affect the myocardium on a regional, rather than global basis. Regional myocardial wall motion can be quantified using myocardial strain analysis (a normalized measure of deformation). Myocardial strain can be measured in terms of three normal strains (longitudinal strain, radial strain and circumferential) and six shear strains. Cardiac MRI (cMRI) is usually considered the reference standard for measurement of myocardial strain. The most common cMRI method, termed tagged cMRI, allows full, 3D assessment of regional strain. However, due to its complexity and lengthy times for analysis, tagged cMRI is not usually used outside of academic centers. Tagged cMRI is also primarily used only in research studies. Echocardiography combined with tissue Doppler imaging or a speckle tracking technique is now widely available in the clinical setting. Myocardial strain measurement by echocardiography shows reasonable agreement with cMRI. Limited standardization and differences between vendors represent current limitations of the technique. Cardiac computed tomography (CCT) is the newest and most rapidly growing modality for noninvasive imaging of the heart. While CCT studies are most commonly applied to assess the coronary arteries, CCT is easily adapted to provide functional information for both the left and right ventricles. New methods for CCT assessment of regional myocardial function are being developed. This review outlines the current literature on imaging techniques related to cardiac strain analysis and discusses the strengths and weaknesses of various methods for myocardial strain analysis.

Measuring Left Ventricular Volumes in Two-Dimensional Echocardiography Image Sequence Using Level-set Method for Automatic Detection of End-Diastole and End-systole Frames

BACKGROUND

Identifying End-Diastole (ED) and End-Systole (ES) frames is highly important in the process of evaluating cardiac function and measuring global parameters accurately, such as Ejection Fraction (EF), Cardiac Output (CO) and Stroke Volume.

OBJECTIVES

The current study aimed to develop a new method based on measuring volume changes in Left Ventricle (LV) during cardiac cycle.

MATERIAL AND METHODS

For this purpose, the Level Set method was used both in detecting endocardium border and quantifying cardiac function of all frames.

RESULTS

Demonstrating LV volumes displays ED and ES frames and the volumes used in calculating the required parameters.

CONCLUSIONS

Since ES and ED frames exist in iso-volumic phases of the cardiac cycle with minimum and maximum values of LV volume signals, such peaks can be utilized in finding related frames.

Mitral valve prolapse detection using landmark extraction from echocardiography sequences

The mitral valve is one of the four valves of the heart, whose function is to keep the blood flow in the physiological direction when the heart contracts. There is no satisfactory method allowing an automated assessment for Mitral Valve Prolapse (MVP) detection. In this paper an algorithm is proposed for detecting MVPs automatically from an echocardiography sequence. Our algorithm has two steps; first landmarks are extracted from the echocardiography sequence. Then landmarks are tracked in the whole frames of a sequence. In order to detect MVP and isolate it from a normal mitral motion, we extracted some features (such as maximum deviation of valve angle and spectral power ratio) from the motion pattern of a mitral valve and we gave these features to a SVM classifier. The results show that the mitral motion trajectory may have good discriminative features for detecting MVP (87% specificity and 84% sensitivity).

Automated septum thickness measurement--a Kalman filter approach

Interventricular septum thickness in end-diastole (IVSd) is one of the key parameters in cardiology. This paper presents a fast algorithm, suitable for pocket-sized ultrasound devices, for measurement of IVSd using 2D B-mode parasternal long axis images. The algorithm is based on a deformable model of the septum and the mitral valve. The model shape is estimated using an extended Kalman filter. A feasibility study using 32 unselected recordings is presented. The recordings originate from a database consisting of subjects from a normal healthy population. Five patients with suspected hypertrophy were included in the study. Reference B-mode measurements were made by two cardiologists. A paired t-test revealed a non-significant mean difference, compared to the B-mode reference, of (mean±SD) 0.14±1.36 mm (p=0.532). Pearson's correlation coefficient was 0.79 (p<0.001). The results are comparable to the variability between the two cardiologists, which was found to be 1.29±1.23 mm (p<0.001). The results indicate that the method has potential as a tool for rapid assessment of IVSd.

Left ventricle wall motion quantification from echocardiographic images by non-rigid image registration

PURPOSE

The aim of this study is to evaluate the efficiency of applying a new non-rigid image registration method on two-dimensional echocardiographic images for computing the left ventricle (LV) myocardial motion field over a cardiac cycle.

METHODS

The key feature of our method is to register all images in the sequence to a reference image (end-diastole image) using a hierarchical transformation model, which is a combination of an affine transformation for modeling the global LV motion and a free-form deformation (FFD) transformation based on B-splines for modeling the local LV deformation. Registration is done by minimizing a cost function associated with the image similarity based on a global pixel-based matching and the smoothness of transformation. The algorithm uses a fast and robust optimization strategy using a multiresolution approach for the estimation of parameters of the deformation model. The proposed algorithm is evaluated for calculating the displacement curves of two expert-identified anatomical landmarks in apical views of the LV for 10 healthy volunteers and 14 subjects with pathology. The proposed algorithm is also evaluated for classifying the regional LV wall motion abnormality using the calculation of the strain value at the end of systole in 288 segments as scored by two consensual experienced echocardiographers in a three-point scale: 1: normokinesia, 2: hypokinesia, and 3: akinesia. Moreover, we compared the results of the proposed registration algorithm to those previously obtained using the other image registration methods.

RESULTS

Regarding to the reference two experienced echocardiographers, the results demonstrate the proposed algorithm more accurately estimates the displacement curve of the two anatomical landmarks in apical views than the other registration methods in all data set. Moreover, the p values of the t test for the strain value of each segment at the end of systole measured by the proposed algorithm show higher differences than the other registration method. These differences are between each pair of scores in all segments and in three segments of septum independently.

CONCLUSIONS

The clinical results show that the proposed algorithm can improve both the calculation of the displacement curve of every point of LV during a cardiac cycle and the classification of regional LV wall motion abnormality. Therefore, this diagnostic system can be used as a useful tool for clinical evaluation of the regional LV function.

Algorithm versus physicians variability evaluation in the cardiac chambers extraction

Congenital heart diseases are present in eight of every 1000 newborns. The diagnosis of those pathologies usually depends on the available imaging methods. A correct diagnosis requires a detailed observation of the heart chambers, wall motions, valves function, and quantitative evaluation of the cavity volumes. For that goal numerous automatic algorithms have been proposed to segment the echocardiographic images. In this paper, the authors evaluate the performance of a level set algorithm based on the phase symmetry approach and on a new logarithmic-based stopping function to extract the heart cavity contours simultaneously, and in a fully automatic way. The extracted cardiac borders are then statistically compared with the ones manually sketched by four physicians on a set of 240 cavities. Nonparametric statistical tests are conducted on the data using several figures of merit, in order to study the inter- and intraobserver variabilities among the four physicians and the level set algorithm, concerning to the extracted contours. The results show there is a great concordance about all the used similarity indexes. A higher interobserver variability was found among the physicians than the variability obtained when the algorithm versus physician performance is compared. The statistical analysis suggests the proposed algorithm produces results similar to the ones provided by the physicians.

Regional cardiac motion and strain estimation in three-dimensional echocardiography: a validation study in thick-walled univentricular phantoms

Automatic quantification of regional left ventricular deformation in volumetric ultrasound data remains challenging. Many methods have been proposed to extract myocardial motion, including techniques using block matching, phase-based correlation, differential optical flow methods, and image registration. Our lab previously presented an approach based on elastic registration of subsequent volumes using a B-spline representation of the underlying transformation field. Encouraging results were obtained for the assessment of global left ventricular function, but a thorough validation on a regional level was still lacking. For this purpose, univentricular thick-walled cardiac phantoms were deformed in an experimental setup to locally assess strain accuracy against sonomicrometry as a reference method and to assess whether regions containing stiff inclusions could be detected. Our method showed good correlations against sonomicrometry: r(2) was 0.96, 0.92, and 0.84 for the radial (ε(RR)), longitudinal (ε(LL)), and circumferential (ε(CC)) strain, respectively. Absolute strain errors and strain drift were low for ε(LL) (absolute mean error: 2.42%, drift: -1.05%) and ε(CC) (error: 1.79%, drift: -1.33%) and slightly higher for ε(RR) (error: 3.37%, drift: 3.05%). The discriminative power of our methodology was adequate to resolve full transmural inclusions down to 17 mm in diameter, although the inclusion-to-surrounding tissue stiffness ratio was required to be at least 5:2 (absolute difference of 39.42 kPa). When the inclusion-to-surrounding tissue stiffness ratio was lowered to approximately 2:1 (absolute difference of 22.63 kPa), only larger inclusions down to 27 mm in diameter could still be identified. Radial strain was found not to be reliable in identifying dysfunctional regions.

The segmentation of the left ventricle of the heart from ultrasound data using deep learning architectures and derivative-based search methods

We present a new supervised learning model designed for the automatic segmentation of the left ventricle (LV) of the heart in ultrasound images. We address the following problems inherent to supervised learning models: 1) the need of a large set of training images; 2) robustness to imaging conditions not present in the training data; and 3) complex search process. The innovations of our approach reside in a formulation that decouples the rigid and nonrigid detections, deep learning methods that model the appearance of the LV, and efficient derivative-based search algorithms. The functionality of our approach is evaluated using a data set of diseased cases containing 400 annotated images (from 12 sequences) and another data set of normal cases comprising 80 annotated images (from two sequences), where both sets present long axis views of the LV. Using several error measures to compute the degree of similarity between the manual and automatic segmentations, we show that our method not only has high sensitivity and specificity but also presents variations with respect to a gold standard (computed from the manual annotations of two experts) within interuser variability on a subset of the diseased cases. We also compare the segmentations produced by our approach and by two state-of-the-art LV segmentation models on the data set of normal cases, and the results show that our approach produces segmentations that are comparable to these two approaches using only 20 training images and increasing the training set to 400 images causes our approach to be generally more accurate. Finally, we show that efficient search methods reduce up to tenfold the complexity of the method while still producing competitive segmentations. In the future, we plan to include a dynamical model to improve the performance of the algorithm, to use semisupervised learning methods to reduce even more the dependence on rich and large training sets, and to design a shape model less dependent on the training set.

A dynamical appearance model based on multiscale sparse representation: segmentation of the left ventricle from 4D echocardiography

The spatio-temporal coherence in data plays an important role in echocardiographic segmentation. While learning offline dynamical priors from databases has received considerable attention, these priors may not be suitable for post-infarct patients and children with congenital heart disease. This paper presents a dynamical appearance model (DAM) driven by individual inherent data coherence. It employs multi-scale sparse representation of local appearance, learns online multiscale appearance dictionaries as the image sequence is segmented sequentially, and integrates a spectrum of complementary multiscale appearance information including intensity, multiscale local appearance, and dynamical shape predictions. It overcomes the limitations of database-driven statistical models and applies to a broader range of subjects. Results on 26 4D canine echocardiographic images acquired from both healthy and post-infarct subjects show that our method significantly improves segmentation accuracy and robustness compared to a conventional intensity model and our previous single-scale sparse representation method.

Ultrasonic image analysis and image-guided interventions

The fields of medical image analysis and computer-aided interventions deal with reducing the large volume of digital images (X-ray, computed tomography, magnetic resonance imaging (MRI), positron emission tomography and ultrasound (US)) to more meaningful clinical information using software algorithms. US is a core imaging modality employed in these areas, both in its own right and used in conjunction with the other imaging modalities. It is receiving increased interest owing to the recent introduction of three-dimensional US, significant improvements in US image quality, and better understanding of how to design algorithms which exploit the unique strengths and properties of this real-time imaging modality. This article reviews the current state of art in US image analysis and its application in image-guided interventions. The article concludes by giving a perspective from clinical cardiology which is one of the most advanced areas of clinical application of US image analysis and describing some probable future trends in this important area of ultrasonic imaging research.

Fast automatic measurement of mitral annulus excursion using a pocket-sized ultrasound system

We present a fast, automatic method for mitral annulus excursion measurement using pocket-sized ultrasound (PSU). The motivation is to provide PSU users with a rapid measurement of cardiac systolic function. The algorithm combines low-frame-rate tolerance, computational efficiency and automation in a novel way. The method uses a speckle-tracking scheme, initialized and constrained by a deformable model. A feasibility study using 30 apical four-chamber PSU recordings from an unselected patient population revealed an error of (mean ± SD) -1.80 ± 1.96 mm, p < 0.001, when compared with manual anatomic m-mode analysis using laptop scanner data. When only septal side excursion was measured, the mean error was -0.27 ± 1.89 mm, p < 0.001. The accuracy is comparable with previously reported results using semiautomatic methods and full-size scanners. The computation time of 3.7 ms/frame on a laptop computer suggests that a real-time implementation on a PSU device is feasible.

The evaluation of single-view and multi-view fusion 3D echocardiography using image-driven segmentation and tracking

Real-time 3D echocardiography (RT3DE) promises a more objective and complete cardiac functional analysis by dynamic 3D image acquisition. Despite several efforts towards automation of left ventricle (LV) segmentation and tracking, these remain challenging research problems due to the poor-quality nature of acquired images usually containing missing anatomical information, speckle noise, and limited field-of-view (FOV). Recently, multi-view fusion 3D echocardiography has been introduced as acquiring multiple conventional single-view RT3DE images with small probe movements and fusing them together after alignment. This concept of multi-view fusion helps to improve image quality and anatomical information and extends the FOV. We now take this work further by comparing single-view and multi-view fused images in a systematic study. In order to better illustrate the differences, this work evaluates image quality and information content of single-view and multi-view fused images using image-driven LV endocardial segmentation and tracking. The image-driven methods were utilized to fully exploit image quality and anatomical information present in the image, thus purposely not including any high-level constraints like prior shape or motion knowledge in the analysis approaches. Experiments show that multi-view fused images are better suited for LV segmentation and tracking, while relatively more failures and errors were observed on single-view images.

Adaptively learning local shape statistics for prostate segmentation in ultrasound

Automatic segmentation of the prostate from 2-D transrectal ultrasound (TRUS) is a highly desired tool in many clinical applications. However, it is a very challenging task, especially for segmenting the base and apex of the prostate due to the large shape variations in those areas compared to the midgland, which leads many existing segmentation methods to fail. To address the problem, this paper presents a novel TRUS video segmentation algorithm using both global population-based and patient-specific local shape statistics as shape constraint. By adaptively learning shape statistics in a local neighborhood during the segmentation process, the algorithm can effectively capture the patient-specific shape statistics and quickly adapt to the local shape changes in the base and apex areas. The learned shape statistics is then used as the shape constraint in a deformable model for TRUS video segmentation. The proposed method can robustly segment the entire gland of the prostate with significantly improved performance in the base and apex regions, compared to other previously reported methods. Our method was evaluated using 19 video sequences obtained from different patients and the average mean absolute distance error was 1.65 ± 0.47 mm.

Ultrasound image segmentation and tissue characterization

Ultrasound image segmentation deals with delineating the boundaries of structures, as a step towards semi-automated or fully automated measurement of dimensions or for characterizing tissue regions. Ultrasound tissue characterization (UTC) is driven by knowledge of the physics of ultrasound and its interactions with biological tissue, and has traditionally used signal modelling and analysis to characterize and differentiate between healthy and diseased tissue. Thus, both aim to enhance the capabilities of ultrasound as a quantitative tool in clinical medicine, and the two end goals can be the same, namely to characterize the health of tissue. This article reviews both research topics, and finds that the two fields are becoming more tightly coupled, even though there are key challenges to overcome in each area, influenced by factors such as more open software-based ultrasound system architectures, increased computational power, and advances in imaging transducer design.

A coupled deformable model for tracking myocardial borders from real-time echocardiography using an incompressibility constraint

Real-time three-dimensional (RT3D) echocardiography is a new image acquisition technique that allows instantaneous acquisition of volumetric images for quantitative assessment of cardiac morphology and function. To quantify many important diagnostic parameters, such as ventricular volume, ejection fraction, and cardiac output, an automatic algorithm to delineate the left ventricle (LV) from RT3D echocardiographic images is essential. While a number of efforts have been made towards segmentation of the LV endocardial (ENDO) boundaries, the segmentation of epicardial (EPI) boundaries remains problematic. In this paper, we present a coupled deformable model that addresses this problem. The idea behind our method is that the volume of the myocardium is close to being constant during a cardiac cycle and our model uses this coupling as an important constraint. We employ two surfaces, each driven by the image-derived information that takes into account ultrasound physics by modeling the speckle statistics using the Nakagami distribution while maintaining the coupling. By simultaneously evolving two surfaces, the final segmentation of the myocardium is thus achieved. Results from 80 sets of synthetic data and 286 sets of real canine data were evaluated against the ground truth and against outlines from three independent observers, respectively. We show that results obtained with our incompressibility constraint were more accurate than those obtained without constraint or with a wall thickness constraint, and were comparable to those from manual segmentation.

Fast and automatic LV mass calculation from echocardiographic images via B-spline snake model and Markov random fields

Left ventricular (LV) mass has several important diagnostic and indicative implications. In this paper, a fast and accurate technique for detection of inner and outer boundaries of LV and, consequently, calculation of LV mass from apical 4-chamber echocardiographic images is presented. For detection of the inner boundary, a modified B-spline snake is proposed, which relies merely on image intensity and obviates the need for computationally-demanding image forces. The outer boundary is then obtained using a Markov random fields model in the neighborhood of the estimated inner border. Experimental validation of the proposed technique demonstrates remarkable improvement over conventional algorithms.

A Dynamical Shape Prior for LV Segmentation from RT3D Echocardiography

Real-time three-dimensional (RT3D) echocardiography is the newest generation of three-dimensional (3-D) echocardiography. Segmentation of RT3D echocardiographic images is essential for determining many important diagnostic parameters. In cardiac imaging, since the heart is a moving organ, prior knowledge regarding its shape and motion patterns becomes an important component for the segmentation task. However, most previous cardiac models are either static models (SM), which neglect the temporal coherence of a cardiac sequence or generic dynamical models (GDM), which neglect the inter-subject variability of cardiac motion. In this paper, we present a subject-specific dynamical model (SSDM) which simultaneously handles inter-subject variability and cardiac dynamics (intra-subject variability). It can progressively predict the shape and motion patterns of a new sequence at the current frame based on the shapes observed in the past frames. The incorporation of this SSDM into the segmentation process is formulated in a recursive Bayesian framework. This results in a segmentation of each frame based on the intensity information of the current frame, as well as on the prediction from the previous frames. Quantitative results on 15 RT3D echocardiographic sequences show that automatic segmentation with SSDM is superior to that of either SM or GDM, and is comparable to manual segmentation.

Detection and measurement of fetal anatomies from ultrasound images using a constrained probabilistic boosting tree

We propose a novel method for the automatic detection and measurement of fetal anatomical structures in ultrasound images. This problem offers a myriad of challenges, including: difficulty of modeling the appearance variations of the visual object of interest, robustness to speckle noise and signal dropout, and large search space of the detection procedure. Previous solutions typically rely on the explicit encoding of prior knowledge and formulation of the problem as a perceptual grouping task solved through clustering or variational approaches. These methods are constrained by the validity of the underlying assumptions and usually are not enough to capture the complex appearances of fetal anatomies. We propose a novel system for fast automatic detection and measurement of fetal anatomies that directly exploits a large database of expert annotated fetal anatomical structures in ultrasound images. Our method learns automatically to distinguish between the appearance of the object of interest and background by training a constrained probabilistic boosting tree classifier. This system is able to produce the automatic segmentation of several fetal anatomies using the same basic detection algorithm. We show results on fully automatic measurement of biparietal diameter (BPD), head circumference (HC), abdominal circumference (AC), femur length (FL), humerus length (HL), and crown rump length (CRL). Notice that our approach is the first in the literature to deal with the HL and CRL measurements. Extensive experiments (with clinical validation) show that our system is, on average, close to the accuracy of experts in terms of segmentation and obstetric measurements. Finally, this system runs under half second on a standard dual-core PC computer.

Learning active shape models for bifurcating contours

Statistical shape models are often learned from examples based on landmark correspondences between annotated examples. A method is proposed for learning such models from contours with inconsistent bifurcations and loops. Automatic segmentation of tibial and femoral contours in knee X-ray images is investigated as a step towards reliable, quantitative radiographic analysis of osteoarthritis for diagnosis and assessment of progression. Results are presented using various features, the Mahalanobis distance, distance weighted K-nearest neighbours, and two relevance vector machine-based methods as quality of fit measure.

Boosting and nonparametric based tracking of tagged MRI cardiac boundaries

In this paper we present an accurate cardiac boundary tracking method for 2D tagged MRI time sequences. This method naturally integrates the motion and the static local appearance features and generates accurate boundary criteria via a boosting approach. We extend the conventional Adaboost classifier into a posterior probability form, which can be embedded in a particle filtering-based shape tracking framework. To make the tracking process more robust and faster, we use a PCA subspace shape representation to constrain the shape variation and lower the dimensionality. We also learn two shape-dynamic models for systole and diastole separately, to predict the shape evolution. Our tracking method incorporates the static appearance, the motion appearance, the shape constraints, and the dynamic prediction in a unified way. The proposed method has been implemented on 50 tagged MRI sequences. The experimental results show the accuracy and robustness of our approach.

Incorporating temporal information for ventricular contour detection in echocardiographic image sequences

A novel way to incorporate temporal information with level set algorithm is proposed to counter the dropout problem when detecting ventricular contours in echocardiographic raphic image sequences. The temporal information ided embed- ed into the speed term of the level set equation. By identifying the ventricular contours as strong or weak segments, the weak segments are strengthened based on temporal information from neighboring frames. Hence disrupted heart wall boundary structure information due to dropout can be recovered. A Gaussian Mixture Model (GMM) is employed to compute thresholds separating the segments. A weight and a strengthening ng factor are used to control the information recovery process. Experimental results show the proposed method exhibits good performance when tracking the ventricular boundary in real echocardiographic data.

Snakes based segmentation of the common carotid artery intima media

Ultrasound measurements of the human carotid artery walls are conventionally obtained by manually tracing interfaces between tissue layers. In this study we present a snakes segmentation technique for detecting the intima-media layer of the far wall of the common carotid artery (CCA) in longitudinal ultrasound images, by applying snakes, after normalization, speckle reduction, and normalization and speckle reduction. The proposed technique utilizes an improved snake initialization method, and an improved validation of the segmentation method. We have tested and clinically validated the segmentation technique on 100 longitudinal ultrasound images of the carotid artery based on manual measurements by two vascular experts, and a set of different evaluation criteria based on statistical measures and univariate statistical analysis. The results showed that there was no significant difference between all the snakes segmentation measurements and the manual measurements. For the normalized despeckled images, better snakes segmentation results with an intra-observer error of 0.08, a coefficient of variation of 12.5%, best Bland-Altman plot with smaller differences between experts (0.01, 0.09 for Expert1 and Expert 2, respectively), and a Hausdorff distance of 5.2, were obtained. Therefore, the pre-processing of ultrasound images of the carotid artery with normalization and speckle reduction, followed by the snakes segmentation algorithm can be used successfully in the measurement of IMT complementing the manual measurements. The present results are an expansion of data published earlier as an extended abstract in IFMBE Proceedings (Loizou et al. IEEE Int X Mediterr Conf Medicon Med Biol Eng POS-03 499:1-4, 2004).

Real-time tracking of the left ventricle in 3D echocardiography using a state estimation approach

In this paper we present a framework for real-time tracking of deformable contours in volumetric datasets. The framework supports composite deformation models, controlled by parameters for contour shape in addition to global pose. Tracking is performed in a sequential state estimation fashion, using an extended Kalman filter, with measurement processing in information space to effectively predict and update contour deformations in real-time. A deformable B-spline surface coupled with a global pose transform is used to model shape changes of the left ventricle of the heart. Successful tracking of global motion and local shape changes without user intervention is demonstrated on a dataset consisting of 21 3D echocardiography recordings. Real-time tracking using the proposed approach requires a modest CPU load of 13% on a modern computer. The segmented volumes compare to a semi-automatic segmentation tool with 95% limits of agreement in the interval 4.1 +/- 24.6 ml (r = 0.92).