Left ventricle segmentation combining deep learning and deformable models with anatomical constraints

Segmentation of the left ventricle is a key approach in Cardiac Magnetic Resonance Imaging for calculating biomarkers in diagnosis. Since there is substantial effort required from the expert, many automatic segmentation methods have been proposed, in which deep learning networks have obtained remarkable performance. However, one of the main limitations of these approaches is the production of segmentations that contain anatomical errors. To avoid this limitation, we propose a new fully-automatic left ventricle segmentation method combining deep learning and deformable models. We propose a new level set energy formulation that includes exam-specific information estimated from the deep learning segmentation and shape constraints. The method is part of a pipeline containing pre-processing steps and a failure correction post-processing step. Experiments were conducted with the Sunnybrook and ACDC public datasets, and a private dataset. Results suggest that the method is competitive, that it can produce anatomically consistent segmentations, has good generalization ability, and is often able to estimate biomarkers close to the expert.

Inter-subject registration-based one-shot segmentation with alternating union network for cardiac MRI images

Medical image segmentation based on deep-learning networks makes great progress in assisting disease diagnosis. However, currently, the training of most networks still requires a large amount of data with labels. In reality, labeling a considerable number of medical images is challenging and time-consuming. In order to tackle this challenge, a new one-shot segmentation framework for cardiac MRI images based on an inter-subject registration model called Alternating Union Network (AUN) is proposed in this study. The label of the source image is warped with deformation fields discovered from AUN to segment target images directly. Initially, the volumes are pre-processed by aligning affinely and adjusting the global intensity to simplify subsequent deformation registration. AUN consists of two kinds of subnetworks trained alternately to optimize segmentation gradually. The first kind of subnetwork takes a pair of volumes as inputs and registers them using global intensity similarity. The second kind of subnetwork, which takes the predicted labels generated from the previous subnetwork and the labels refined using the information of intrinsic anatomical structures of interest as inputs, is intensity-independent and focuses attention on registering structures of interest. Specifically, the input of AUN is a pair of a labeled image with the texture in regions of interest removed and a target image. Additionally, a new similarity measurement more appropriate for registering such image pair is defined as Local Squared Error (LSE). The proposed registration-based one-shot segmentation pays attention to the problem of the lack of labeled medical images. In AUN, only one labeled volume is required and a large number of unlabeled ones can be leveraged to improve segmentation performance, which has great advantages in clinical application. In addition, the intensity-independent subnetwork and LSE proposed in this study empower the framework to segment medical images with complicated intensity distribution.

Anatomical knowledge based level set segmentation of cardiac ventricles from MRI

This paper represents a novel level set framework for segmentation of cardiac left ventricle (LV) and right ventricle (RV) from magnetic resonance images based on anatomical structures of the heart. We first propose a level set approach to recover the endocardium and epicardium of LV by using a bi-layer level set (BILLS) formulation, in which the endocardium and epicardium are represented by the 0-level set and k-level set of a level set function. Furthermore, the recovery of LV endocardium and epicardium is achieved by a level set evolution process, called convexity preserving bi-layer level set (CP-BILLS). During the CP-BILLS evolution, the 0-level set and k-level set simultaneously evolve and move toward the true endocardium and epicardium under the guidance of image information and the impact of the convexity preserving mechanism as well. To eliminate the manual selection of the k-level, we develop an algorithm for automatic selection of an optimal k-level. As a result, the obtained endocardial and epicardial contours are convex and consistent with the anatomy of cardiac ventricles. For segmentation of the whole ventricle, we extend this method to the segmentation of RV and myocardium of both left and right ventricles by using a convex shape decomposition (CSD) structure of cardiac ventricles based on anatomical knowledge. Experimental results demonstrate promising performance of our method. Compared with some traditional methods, our method exhibits superior performance in terms of segmentation accuracy and algorithm stability. Our method is comparable with the state-of-the-art deep learning-based method in terms of segmentation accuracy and algorithm stability, but our method has no need for training and the manual segmentation of the training data.

A Novel Type-2 Fuzzy C-Means Clustering for Brain MR Image Segmentation

The fuzzy C -means (FCM) clustering procedure is an unsupervised form of grouping the homogenous pixels of an image in the feature space into clusters. A brain magnetic resonance (MR) image is affected by noise and intensity inhomogeneity (IIH) during the acquisition process. FCM has been used in MR brain tissue segmentation. However, it does not consider the neighboring pixels for computing the membership values, thereby misclassifying the noisy pixels. The inaccurate cluster centers obtained in FCM do not address the problem of IIH. A fixed value of the fuzzifier ( m ) used in FCM brings uncertainty in controlling the fuzziness of the extracted clusters. To resolve these issues, we suggest a novel type-2 adaptive weighted spatial FCM (AWSFCM) clustering algorithm for MR brain tissue segmentation. The idea of type-2 FCM applied to the problem on hand is new and is reported in this article. The application of the proposed technique to the problem of MR brain tissue segmentation replaces the fixed fuzzifier value with a fuzzy linguistic fuzzifier value ( M ). The introduction of the spatial information in the membership function reduces the misclassification of noisy pixels. Furthermore, the incorporation of adaptive weights into the cluster center update function improves the accuracy of the final cluster centers, thereby reducing the effect of IIH. The suggested algorithm is evaluated using T1-w, T2-w, and proton density (PD) brain MR image slices. The performance is justified in terms of qualitative and quantitative measures followed by statistical analysis. The outcomes demonstrate the superiority and robustness of the algorithm in comparison to the state-of-the-art methods. This article is useful for the cybernetics application.

Multi-level semantic adaptation for few-shot segmentation on cardiac image sequences

Obtaining manual labels is time-consuming and labor-intensive on cardiac image sequences. Few-shot segmentation can utilize limited labels to learn new tasks. However, it suffers from two challenges: spatial-temporal distribution bias and long-term information bias. These challenges derive from the impact of the time dimension on cardiac image sequences, resulting in serious over-adaptation. In this paper, we propose the multi-level semantic adaptation (MSA) for few-shot segmentation on cardiac image sequences. The MSA addresses the two biases by exploring the domain adaptation and the weight adaptation on the semantic features in multiple levels, including sequence-level, frame-level, and pixel-level. First, the MSA proposes the dual-level feature adjustment for domain adaptation in spatial and temporal directions. This adjustment explicitly aligns the frame-level feature and the sequence-level feature to improve the model adaptation on diverse modalities. Second, the MSA explores the hierarchical attention metric for weight adaptation in the frame-level feature and the pixel-level feature. This metric focuses on the similar frame and the target region to promote the model discrimination on the border features. The extensive experiments demonstrate that our MSA is effective in few-shot segmentation on cardiac image sequences with three modalities, i.e. MR, CT, and Echo (e.g. the average Dice is 0.9243), as well as superior to the ten state-of-the-art methods.

Left ventricle motion estimation for cine MR images using sparse representation with shape constraint

PURPOSE

To propose a left ventricle (LV) motion estimation method based on sparse representation, in order to handle the spatial-varying intensity distortions caused by tissue deformation.

METHODS

For each myocardial landmark, an adaptive dictionary was generated by learning transformations from a training dataset. Then the landmark was tracked using sparse representation. Next, a point distribution model was applied to the overall tracking results. Finally, the dense displacement field of the LV myocardium was estimated based on the correspondence between each landmark. Using the dense displacement field estimated, the circumferential strain was calculated to assess the myocardial function. The performance of the proposed method was quantified by the average perpendicular distance (APD), the Dice metric, and the mean symmetric contour distance (SCD).

RESULTS

Comparing to the state-of-the-art techniques, the smallest value of APD and SCD, and the highest value of Dice can be obtained using the proposed method, for three public cardiac datasets. Moreover, the mean value of strain difference between the proposed method and the commercial software Medis Suite MR was -0.01, while the intraclass correlation coefficient between these two methods was 0.91.

CONCLUSIONS

The proposed method could estimate the dense displacement field of the LV accurately, which outperforms other state-of-the-art techniques. The circumferential strain derived from the proposed method was in excellent agreement with that derived from the Medis Suite MR software, while segmental strain abnormalities were detected for most of the subjects with heart diseases, which indicates the potential of the proposed method for clinical usage.

Classification of Myocardial 18F-FDG PET Uptake Patterns Using Deep Learning

Purpose

To perform automated myocardial segmentation and uptake classification from whole-body fluorine 18 fluorodeoxyglucose (FDG) PET.

Materials and Methods

In this retrospective study, consecutive patients who underwent FDG PET imaging for oncologic indications were included (July-August 2018). The left ventricle (LV) on whole-body FDG PET images was manually segmented and classified as showing no myocardial uptake, diffuse uptake, or partial uptake. A total of 609 patients (mean age, 64 years ± 14 [standard deviation]; 309 women) were included and split between training (60%, 365 patients), validation (20%, 122 patients), and testing (20%, 122 patients) datasets. Two sequential neural networks were developed to automatically segment the LV and classify the myocardial uptake pattern using segmentation and classification training data provided by human experts. Linear regression was performed to correlate findings from human experts and deep learning. Classification performance was evaluated using receiver operating characteristic (ROC) analysis.

Results

There was moderate agreement of uptake pattern between experts and deep learning (as a fraction of correctly categorized images) with 78% (36 of 46) for no uptake, 71% (34 of 48) for diffuse uptake, and 71% (20 of 28) for partial uptake. There was no bias in LV volume for partial or diffuse uptake categories (P = .56); however, deep learning underestimated LV volumes in the no uptake category. There was good correlation for LV volume (R ² = 0.35, b = .71). ROC analysis showed the area under the curve for classifying no uptake and diffuse uptake was high (> 0.90) but lower for partial uptake (0.77). The feasibility of a myocardial uptake index (MUI) for quantifying the degree of myocardial activity patterns was shown, and there was excellent visual agreement between MUI and uptake patterns.

Conclusion

Deep learning was able to segment and classify myocardial uptake patterns on FDG PET images.Keywords: PET, Heart, Computer Aided Diagnosis, Computer Application-Detection/DiagnosisSupplemental material is available for this article.©RSNA, 2021.

Convexity preserving level set for left ventricle segmentation

In clinical applications of cardiac left ventricle (LV) segmentation, the segmented LV is desired to include the cavity, trabeculae, and papillary muscles, which form a convex shape. However, the intensities of trabeculae and papillary muscles are similar to myocardium. Consequently, segmentation algorithms may easily misclassify trabeculae and papillary muscles as myocardium. In this paper, we propose a level set method with a convexity preserving mechanism to ensure the convexity of the segmented LV. In the proposed level set method, the curvature of the level set contours is used to control their convexity, such that the level set contour is finally deformed as a convex shape. The experimental results and the comparison with other level set methods show the advantage of our method in terms of segmentation accuracy. Compared with the state-of-the-art methods using deep-learning, our method is able to achieve comparable segmentation accuracy without the need for training, while the deep-learning based method requires a large set of training data and high-quality manual segmentation. Therefore, our method can be conveniently used in situation where training data and their manual segmentation are not available.

Fully‑automated deep‑learning segmentation of pediatric cardiovascular magnetic resonance of patients with complex congenital heart diseases

BACKGROUND

For the growing patient population with congenital heart disease (CHD), improving clinical workflow, accuracy of diagnosis, and efficiency of analyses are considered unmet clinical needs. Cardiovascular magnetic resonance (CMR) imaging offers non-invasive and non-ionizing assessment of CHD patients. However, although CMR data facilitates reliable analysis of cardiac function and anatomy, clinical workflow mostly relies on manual analysis of CMR images, which is time consuming. Thus, an automated and accurate segmentation platform exclusively dedicated to pediatric CMR images can significantly improve the clinical workflow, as the present work aims to establish.

METHODS

Training artificial intelligence (AI) algorithms for CMR analysis requires large annotated datasets, which are not readily available for pediatric subjects and particularly in CHD patients. To mitigate this issue, we devised a novel method that uses a generative adversarial network (GAN) to synthetically augment the training dataset via generating synthetic CMR images and their corresponding chamber segmentations. In addition, we trained and validated a deep fully convolutional network (FCN) on a dataset, consisting of [Formula: see text] pediatric subjects with complex CHD, which we made publicly available. Dice metric, Jaccard index and Hausdorff distance as well as clinically-relevant volumetric indices are reported to assess and compare our platform with other algorithms including U-Net and cvi42, which is used in clinics.

RESULTS

For congenital CMR dataset, our FCN model yields an average Dice metric of [Formula: see text] and [Formula: see text] for LV at end-diastole and end-systole, respectively, and [Formula: see text] and [Formula: see text] for RV at end-diastole and end-systole, respectively. Using the same dataset, the cvi42, resulted in [Formula: see text], [Formula: see text], [Formula: see text] and [Formula: see text] for LV and RV at end-diastole and end-systole, and the U-Net architecture resulted in [Formula: see text], [Formula: see text], [Formula: see text] and [Formula: see text] for LV and RV at end-diastole and end-systole, respectively.

CONCLUSIONS

The chambers' segmentation results from our fully-automated method showed strong agreement with manual segmentation and no significant statistical difference was found by two independent statistical analyses. Whereas cvi42 and U-Net segmentation results failed to pass the t-test. Relying on these outcomes, it can be inferred that by taking advantage of GANs, our method is clinically relevant and can be used for pediatric and congenital CMR segmentation and analysis.

Generalizable fully automated multi-label segmentation of four-chamber view echocardiograms based on deep convolutional adversarial networks

A major issue in translation of the artificial intelligence platforms for automatic segmentation of echocardiograms to clinics is their generalizability. The present study introduces and verifies a novel generalizable and efficient fully automatic multi-label segmentation method for four-chamber view echocardiograms based on deep fully convolutional networks (FCNs) and adversarial training. For the first time, we used generative adversarial networks for pixel classification training, a novel method in machine learning not currently used for cardiac imaging, to overcome the generalization problem. The method's performance was validated against manual segmentations as the ground-truth. Furthermore, to verify our method's generalizability in comparison with other existing techniques, we compared our method's performance with a state-of-the-art method on our dataset in addition to an independent dataset of 450 patients from the CAMUS (cardiac acquisitions for multi-structure ultrasound segmentation) challenge. On our test dataset, automatic segmentation of all four chambers achieved a dice metric of 92.1%, 86.3%, 89.6% and 91.4% for LV, RV, LA and RA, respectively. LV volumes' correlation between automatic and manual segmentation were 0.94 and 0.93 for end-diastolic volume and end-systolic volume, respectively. Excellent agreement with chambers' reference contours and significant improvement over previous FCN-based methods suggest that generative adversarial networks for pixel classification training can effectively design generalizable fully automatic FCN-based networks for four-chamber segmentation of echocardiograms even with limited number of training data.

Fully Automated 3D Cardiac MRI Localisation and Segmentation Using Deep Neural Networks

Cardiac magnetic resonance (CMR) imaging is used widely for morphological assessment and diagnosis of various cardiovascular diseases. Deep learning approaches based on 3D fully convolutional networks (FCNs), have improved state-of-the-art segmentation performance in CMR images. However, previous methods have employed several pre-processing steps and have focused primarily on segmenting low-resolutions images. A crucial step in any automatic segmentation approach is to first localize the cardiac structure of interest within the MRI volume, to reduce false positives and computational complexity. In this paper, we propose two strategies for localizing and segmenting the heart ventricles and myocardium, termed multi-stage and end-to-end, using a 3D convolutional neural network. Our method consists of an encoder–decoder network that is first trained to predict a coarse localized density map of the target structure at a low resolution. Subsequently, a second similar network employs this coarse density map to crop the image at a higher resolution, and consequently, segment the target structure. For the latter, the same two-stage architecture is trained end-to-end. The 3D U-Net with some architectural changes (referred to as 3D DR-UNet) was used as the base architecture in this framework for both the multi-stage and end-to-end strategies. Moreover, we investigate whether the incorporation of coarse features improves the segmentation. We evaluate the two proposed segmentation strategies on two cardiac MRI datasets, namely, the Automatic Cardiac Segmentation Challenge (ACDC) STACOM 2017, and Left Atrium Segmentation Challenge (LASC) STACOM 2018. Extensive experiments and comparisons with other state-of-the-art methods indicate that the proposed multi-stage framework consistently outperforms the rest in terms of several segmentation metrics. The experimental results highlight the robustness of the proposed approach, and its ability to generate accurate high-resolution segmentations, despite the presence of varying degrees of pathology-induced changes to cardiac morphology and image appearance, low contrast, and noise in the CMR volumes.

A Deep Learning Segmentation Approach in Free-Breathing Real-Time Cardiac Magnetic Resonance Imaging

OBJECTIVES

The purpose of this study was to segment the left ventricle (LV) blood pool, LV myocardium, and right ventricle (RV) blood pool of end-diastole and end-systole frames in free-breathing cardiac magnetic resonance (CMR) imaging. Automatic and accurate segmentation of cardiac structures could reduce the postprocessing time of cardiac function analysis.

METHOD

We proposed a novel deep learning network using a residual block for the segmentation of the heart and a random data augmentation strategy to reduce the training time and the problem of overfitting. Automated cardiac diagnosis challenge (ACDC) data were used for training, and the free-breathing CMR data were used for validation and testing.

RESULTS

The average Dice was 0.919 (LV), 0.806 (myocardium), and 0.818 (RV). The average IoU was 0.860 (LV), 0.699 (myocardium), and 0.761 (RV).

CONCLUSIONS

The proposed method may aid in the segmentation of cardiac images and improves the postprocessing efficiency of cardiac function analysis.

Multi-atlas cardiac PET segmentation

PURPOSE

We propose a multi-atlas based segmentation method for cardiac PET and SPECT images to deal with the high variability of tracer uptake characteristics in myocardium. In addition, we verify its performance by comparing it to the manual segmentation and single-atlas based approach, using dynamic myocardial PET.

METHODS

Twelve left coronary artery ligated SD rats underwent ([¹⁸F]fluoropentyl) triphenylphosphonium salt PET/CT scans. Atlas-based segmentation is based on the spatial normalized template with pre-defined region-of-interest (ROI) for each anatomical or functional structure. To generate multiple left ventricular (LV) atlases, each LV image was segmented manually and divided into angular segments. The segmentation methods performances were compared in regional count information using leave-one-out cross-validation. Additionally, the polar-maps of kinetic parameters were estimated.

RESULTS

In all images, the highest r² template yielded the lowest root-mean-square error (RMSE) between the source image and the best-matching templates ranged between 0.91-0.97 and 0.06-0.11, respectively. The single-atlas and multi-atlas based ROIs yielded remarkably different perfusion distributions: only the multi-atlas based segmentation showed equivalent high correlation results (r² = 0.92) with the manual segmentation compared with the single-atlas based (r² = 0.88). The high perfusion value underestimation was remarkable in single-atlas based segmentation.

CONCLUSIONS

The main advantage of the proposed multi-atlas based cardiac segmentation method is that it does not require any prior information on the tracer distribution to be incorporated into the image segmentation algorithms. Therefore, the same procedure suggested here is applicable to any other cardiac PET or SPECT imaging agents without modification.

Image-Aligned Dynamic Liver Reconstruction Using Intra-Operative Field of Views for Minimal Invasive Surgery

During hepatic minimal invasive surgery (MIS), 3-D reconstruction of a liver surface by interpreting the geometry of its soft tissues is achieving attractions. One of the major issues to be addressed in MIS is liver deformation. Moreover, it severely inhibits free sight and dexterity of tissue manipulation, which causes its intra-operative morphology and soft tissue motion altered as compared to its pre-operative shape. While many applications focus on 3-D reconstruction of rigid or semi-rigid scenes, the techniques applied in hepatic MIS must be able to cope with a dynamic and deformable environment. We propose an efficient technique for liver surface reconstruction based on the structure from motion to handle liver deformation. The reconstructed liver will assist surgeons to visualize liver surface more efficiently with better depth perception. We use the intra-operative field of views to generate 3-D template mesh from a dense keypoint cloud. We estimate liver deformation by finding best correspondence between 3-D templates and reconstruct a liver image to calculate translation and rotational motions. Our technique then finely tunes deformed surface by adding smoothness using shading cues. Up till now, this technique is not used for solving the human liver deformation problem. Our approach is tested and validated with synthetic as well as real in vivo data, which reveal that the reconstruction accuracy can be enhanced using our approach even in challenging laparoscopic environments.

FoCA: A new framework of coupled geometric active contours for segmentation of 3D cardiac magnetic resonance images

In this paper, a new framework of coupled active contours (FoCA) is proposed for segmentation of the left ventricle myocardium, in cardiac magnetic resonance (CMR) images, without primary learning and user-driven segmentation. Primarily, we suggest a pair of coupled geometric active contours (GACs) for segmentation of the endo- and epicardial boundaries of the left ventricle in every CMR slice. The energy functional of each active contour includes the edge and shape terms of the STACS energy functional, regulator term of the local binary fitting (LBF), and new region and coupling terms. Two new patch-based region terms, inspired by LBF and piecewise model, are proposed to effectively handle intensity inhomogeneity of CMR images. Furthermore, a coupling energy term is added to the epicardial energy functional to avoid intersection with the endocardial curve. For 3D implementation, every 2D active contour in each slice is effectively jointed to the corresponding curves in the previous and next slices (of the same volume) by using a new coupling energy term, obtained by extending the 2D length-shortening regulator. Also, the initial contour and algorithm parameters are automatically regulated. Finally, 3D+t implementation is performed by using the sequential initialization method. Experimental results demonstrated that the proposed method provided superior solution quality compared to a large number of counterpart algorithms by using two well-known frequently-used databases.