Multistage hybrid active appearance model matching: segmentation of left and right ventricles in cardiac MR images

CI-UNet: melding convnext and cross-dimensional attention for robust medical image segmentation

Deep learning-based methods have recently shown great promise in medical image segmentation task. However, CNN-based frameworks struggle with inadequate long-range spatial dependency capture, whereas Transformers suffer from computational inefficiency and necessitate substantial volumes of labeled data for effective training. To tackle these issues, this paper introduces CI-UNet, a novel architecture that utilizes ConvNeXt as its encoder, amalgamating the computational efficiency and feature extraction capabilities. Moreover, an advanced attention mechanism is proposed to captures intricate cross-dimensional interactions and global context. Extensive experiments on two segmentation datasets, namely BCSD, and CT2USforKidneySeg, confirm the excellent performance of the proposed CI-UNet as compared to other segmentation methods.

Dilated convolution network with edge fusion block and directional feature maps for cardiac MRI segmentation

Cardiac magnetic resonance imaging (MRI) segmentation task refers to the accurate segmentation of ventricle and myocardium, which is a prerequisite for evaluating the soundness of cardiac function. With the development of deep learning in medical imaging, more and more heart segmentation methods based on deep learning have been proposed. Due to the fuzzy boundary and uneven intensity distribution of cardiac MRI, some existing methods do not make full use of multi-scale characteristic information and have the problem of ambiguity between classes. In this paper, we propose a dilated convolution network with edge fusion block and directional feature maps for cardiac MRI segmentation. The network uses feature fusion module to preserve boundary information, and adopts the direction field module to obtain the feature maps to improve the original segmentation features. Firstly, multi-scale feature information is obtained and fused through dilated convolutional layers of different scales while downsampling. Secondly, in the decoding stage, the edge fusion block integrates the edge features into the side output of the encoder and concatenates them with the upsampled features. Finally, the concatenated features utilize the direction field to improve the original segmentation features and generate the final result. Our propose method conducts comprehensive comparative experiments on the automated cardiac diagnosis challenge (ACDC) and myocardial pathological segmentation (MyoPS) datasets. The results show that the proposed cardiac MRI segmentation method has better performance compared to other existing methods.

CyCoSeg: A Cyclic Collaborative Framework for Automated Medical Image Segmentation

Deep neural networks have been tremendously successful at segmenting objects in images. However, it has been shown they still have limitations on challenging problems such as the segmentation of medical images. The main reason behind this lower success resides in the reduced size of the object in the image. In this paper we overcome this limitation through a cyclic collaborative framework, CyCoSeg. The proposed framework is based on a deep active shape model (D-ASM), which provides prior information about the shape of the object, and a semantic segmentation network (SSN). These two models collaborate to reach the desired segmentation by influencing each other: SSN helps D-ASM identify relevant keypoints in the image through an Expectation Maximization formulation, while D-ASM provides a segmentation proposal that guides the SSN. This cycle is repeated until both models converge. Extensive experimental evaluation shows CyCoSeg boosts the performance of the baseline models, including several popular SSNs, while avoiding major architectural modifications. The effectiveness of our method is demonstrated on the left ventricle segmentation on two benchmark datasets, where our approach achieves one of the most competitive results in segmentation accuracy. Furthermore, its generalization is demonstrated for lungs and kidneys segmentation in CT scans.

Automatic cardiac evaluations using a deep video object segmentation network

BACKGROUND

Accurate cardiac volume and function assessment have valuable and significant diagnostic implications for patients suffering from ventricular dysfunction and cardiovascular disease. This study has focused on finding a reliable assistant to help physicians have more reliable and accurate cardiac measurements using a deep neural network. EchoRCNN is a semi-automated neural network for echocardiography sequence segmentation using a combination of mask region-based convolutional neural network image segmentation structure with reference-guided mask propagation video object segmentation network.

RESULTS

The proposed method accurately segments the left and right ventricle regions in four-chamber view echocardiography series with a dice similarity coefficient of 94.03% and 94.97%, respectively. Further post-processing procedures on the segmented left and right ventricle regions resulted in a mean absolute error of 3.13% and 2.03% for ejection fraction and fractional area change parameters, respectively.

CONCLUSION

This study has achieved excellent performance on the left and right ventricle segmentation, leading to more accurate estimations of vital cardiac parameters such as ejection fraction and fractional area change parameters in the left and right ventricle functionalities, respectively. The results represent that our method can predict an assured, accurate, and reliable cardiac function diagnosis in clinical screenings.

Light-Weight Localization and Scale-Independent Multi-gate UNET Segmentation of Left and Right Ventricles in MRI Images

PURPOSE

Heart segmentation in cardiac magnetic resonance images is heavily used during the assessment of left ventricle global function. Automation of the segmentation is crucial to standardize the analysis. This study aims at developing a CNN-based framework to aid the clinical measurements of the left ventricle and right ventricle in cardiac magnetic resonance images.

METHODS

We propose a fully automated framework for localization and segmentation of the left ventricle and right ventricle in both short- and long-axis views from cardiac magnetic resonance images. The localization module utilizes a light-weight model that detects the region of interest and feeds it to the segmentation model. Also, we propose the Multi-Gate block as an extension to the UNet to boost the segmentation performance by aggregating multi-scale features. Comparison between our proposed method and the baseline UNet was performed to show the gain in the overall performance. The reliability of the model was assessed by testing the method against cardiac magnetic resonance images with different levels of noise and deformations.

RESULTS

Heart localization accuracy was 0.59 and 1.75 pixels in both short- and long-axis views respectively. Left and right ventricle blood-pool segmentation Dice was (0.93, 0.90) in end-systole and (0.97, 0.95) in end-diastole. The left ventricle myocardium was segmented accurately with Dice of 0.91 and 0.90 in end-systole and end-diastole respectively. Left ventricle ejection fraction was found to be highly correlated with the gold standard with r = 0.987. Moreover, the proposed pipeline is fast, achieving 0.002 sec per image on average.

CONCLUSION

Adding the Multi-Gate Dilated Inception Block has boosted the performance of UNet architecture and has shown generalization ability when tested on noisy and deformed cardiac magnetic resonance images. The proposed method has proven its wide applicability and reliability for heart detection when tested on different datasets.

Ultrafast Computation of Left Ventricular Ejection Fraction by Using Temporal Intensity Variation in Cine Cardiac Magnetic Resonance

Cardiac magnetic resonance enables comprehensive cardiac evaluation; however, intense time and labor requirements for data acquisition and processing have discouraged many clinicians from using it. We have developed an alternative image-processing algorithm that requires minimal user interaction: an ultrafast algorithm that computes left ventricular ejection fraction (LVEF) by using temporal intensity variation in cine balanced steady-state free precession (bSSFP) short-axis images, with or without contrast medium. We evaluated the algorithm's performance against an expert observer's analysis for segmenting the LV cavity in 65 study participants (LVEF range, 12%-70%). In 12 instances, contrast medium was administered before cine imaging. Bland-Altman analysis revealed quantitative effects of LV basal, midcavity, and apical morphologic variation on the algorithm's accuracy. Total computation time for the LV stack was <2.5 seconds. The algorithm accurately delineated endocardial boundaries in 1,132 of 1,216 slices (93%). When contours in the extreme basal and apical slices were not adequate, they were replaced with manually drawn contours. The Bland-Altman mean differences were <1.2 mL (0.8%) for end-diastolic volume, <5 mL (6%) for end-systolic volume, and <3% for LVEF. Standard deviation of the difference was ≤4.1% of LV volume for all sections except the midcavity in end-systole (8.3% of end-systolic volume). We conclude that temporal intensity variation-based ultrafast LVEF computation is clinically accurate across a range of LV shapes and wall motions and is suitable for postcontrast cine SSFP imaging. Our algorithm enables real-time processing of cine bSSFP images on a commercial scanner console within 3 seconds in an unobtrusive automated process.

Residual Learning: A New Paradigm to Improve Deep Learning-Based Segmentation of the Left Ventricle in Magnetic Resonance Imaging Cardiac Images

BACKGROUND

Recently, magnetic resonance imaging (MRI) has become a useful tool for the early detection of heart failure. A vital step of this process is a valid measurement of the left ventricle's properties, which seriously depends on the accurate segmentation of the heart in captured images. Although various schemes have been tested for this segmentation so far, the latest proposed methods have used the concept of deep learning to estimate the range of the left ventricle in cardiac MRI images. While deep learning methods can lead to better results than their classical alternatives, but unfortunately, the gradient vanishing and exploding problems may hamper their efficiency for the accurate segmentation of the left ventricle in MRI heart images.

METHODS

In this article, a new concept called residual learning is utilized to improve the performance of deep learning schemes against gradient vanishing problems. For this purpose, the Residual Network of Residual Network (i.e., Residual of Residual) substructure is utilized inside the main deep learning architecture (e.g., Unet), which provides more significant detection indexes.

RESULTS AND CONCLUSION

The proposed method's performances and its alternatives were evaluated on Sunnybrook Cardiac Data as a reliable dataset in the left ventricle segmentation. The results show that the detection parameters are improved at least by 5%, 3.5%, 8.1%, and 11.4% compared to its deep alternatives in terms of Jaccard, Dice, precision, and false-positive rate indexes, respectively. These improvements were made when the recall parameter was reduced to a negligible value (i.e., approximately 1%). Overall, the proposed method can be used as a suitable tool for more accurate detection of the left ventricle in MRI images.

DBAN: Adversarial Network With Multi-Scale Features for Cardiac MRI Segmentation

With the development of medical artificial intelligence, automatic magnetic resonance image (MRI) segmentation method is quite desirable. Inspired by the power of deep neural networks, a novel deep adversarial network, dilated block adversarial network (DBAN), is proposed to perform left ventricle, right ventricle, and myocardium segmentation in short-axis cardiac MRI. DBAN contains a segmentor along with a discriminator. In the segmentor, the dilated block (DB) is proposed to capture, and aggregate multi-scale features. The segmentor can produce segmentation probability maps while the discriminator can differentiate the segmentation probability map, and the ground truth at the pixel level. In addition, confidence probability maps generated by the discriminator can guide the segmentor to modify segmentation probability maps. Extensive experiments demonstrate that DBAN has achieved the state-of-the-art performance on the ACDC dataset. Quantitative analyses indicate that cardiac function indices from DBAN are similar to those from clinical experts. Therefore, DBAN can be a potential candidate for short-axis cardiac MRI segmentation in clinical applications.

Interactive Echocardiography Translation Using Few-Shot GAN Transfer Learning

Background

Interactive echocardiography translation is an efficient educational function to master cardiac anatomy. It strengthens the student's understanding by pixel-level translation between echocardiography and theoretically sketch images. Previous research studies split it into two aspects of image segmentation and synthesis. This split makes it hard to achieve pixel-level corresponding translation. Besides, it is also challenging to leverage deep-learning-based methods in each phase where a handful of annotations are available.

Methods

To address interactive translation with limited annotations, we present a two-step transfer learning approach. Firstly, we train two independent parent networks, the ultrasound to sketch (U2S) parent network and the sketch to ultrasound (S2U) parent network. U2S translation is similar to a segmentation task with sector boundary inference. Therefore, the U2S parent network is trained with the U-Net network on the public segmentation dataset of VOC2012. S2U aims at recovering ultrasound texture. So, the S2U parent network is decoder networks that generate ultrasound data from random input. After pretraining the parent networks, an encoder network is attached to the S2U parent network to translate ultrasound images into sketch images. We jointly transfer learning U2S and S2U within the CGAN framework. Results and conclusion. Quantitative and qualitative contrast from 1-shot, 5-shot, and 10-shot transfer learning show the effectiveness of the proposed algorithm. The interactive translation is achieved with few-shot transfer learning. Thus, the development of new applications from scratch is accelerated. Our few-shot transfer learning has great potential in the biomedical computer-aided image translation field, where annotation data are extremely precious.

An integrated deep learning framework for joint segmentation of blood pool and myocardium

Simultaneous and automatic segmentation of the blood pool and myocardium is an important precondition for early diagnosis and pre-operative planning in patients with complex congenital heart disease. However, due to the high diversity of cardiovascular structures and changes in mechanical properties caused by cardiac defects, the segmentation task still faces great challenges. To overcome these challenges, in this study we propose an integrated multi-task deep learning framework based on the dilated residual and hybrid pyramid pooling network (DRHPPN) for joint segmentation of the blood pool and myocardium. The framework consists of three closely connected progressive sub-networks. An inception module is used to realize the initial multi-level feature representation of cardiovascular images. A dilated residual network (DRN), as the main body of feature extraction and pixel classification, preliminary predicts segmentation regions. A hybrid pyramid pooling network (HPPN) is designed for facilitating the aggregation of local information to global information, which complements DRN. Extensive experiments on three-dimensional cardiovascular magnetic resonance (CMR) images (the available dataset of the MICCAI 2016 HVSMR challenge) demonstrate that our approach can accurately segment the blood pool and myocardium and achieve competitive performance compared with state-of-the-art segmentation methods.

Dynamic pixel-wise weighting-based fully convolutional neural networks for left ventricle segmentation in short-axis MRI

Left ventricle (LV) segmentation in cardiac MRI is an essential procedure for quantitative diagnosis of various cardiovascular diseases. In this paper, we present a novel fully automatic left ventricle segmentation approach based on convolutional neural networks. The proposed network fully takes advantages of the hierarchical architecture and integrate the multi-scale feature together for segmenting the myocardial region of LV. Moreover, we put forward a dynamic pixel-wise weighting strategy, which can dynamically adjust the weight of each pixel according to the segmentation accuracy of upper layer and force the pixel classifier to take more attention on the misclassified ones. By this way, the LV segmentation performance of our method can be improved a lot especially for the apical and basal slices in cine MR images. The experiments on the CAP database demonstrate that our method achieves a substantial improvement compared with other well-know deep learning methods. Beside these, we discussed two major limitations in convolutional neural networks-based semantic segmentation methods for LV segmentation.

Integrated 3D Anatomical Model for Automatic Myocardial Segmentation in Cardiac CT Imagery

Segmentation of epicardial and endocardial boundaries is a critical step in diagnosing cardiovascular function in heart patients. The manual tracing of organ contours in Computed Tomography Angiography (CTA) slices is subjective, time-consuming and impractical in clinical setting. We propose a novel multi-dimensional automatic edge detection algorithm based on shape priors and principal component analysis (PCA). We have developed a highly customized parametric model for implicit representations of segmenting curves (3D) for Left Ventricle (LV), Right Ventricle (RV), and Epicardium (Epi) used simultaneously to achieve myocardial segmentation. We have combined these representations in a region-based image modeling framework with high level constraints enabling the modeling of complex cardiac anatomical structures to automatically guide the segmentation of endo/epicardial boundaries. Test results on 30 short-axis CTA datasets show robust segmentation with error (mean ± std mm) of (1.46 ± 0.41), (2.06 ± 0.65), (2.88 ± 0.59) for LV, RV and Epi respectively.

Symmetric Face Normalization

Image registration is an important process in image processing which is used to improve the performance of computer vision related tasks. In this paper, a novel self-registration method, namely symmetric face normalization (SFN) algorithm, is proposed. There are three contributions in this paper. Firstly, a self-normalization algorithm for face images is proposed, which normalizes a face image to be reflection symmetric horizontally. It has the advantage that no face model needs to be built, which is always severely time-consuming. Moreover, it can be considered as a pre-processing procedure which greatly decreases the parameters needed to be adjusted. Secondly, an iterative algorithm is designed to solve the self-normalization algorithm. Finally, SFN is applied to the between-image alignment problem, which results in the symmetric face alignment (SFA) algorithm. Experiments performed on face databases show that the accuracy of SFN is higher than 0.95 when the translation on the x-axis is lower than 15 pixels, or the rotation angle is lower than 18°. Moreover, the proposed SFA outperforms the state-of-the-art between-image alignment algorithm in efficiency (about four times) without loss of accuracy.

Cardiac Image Segmentation Using Memory Persistence Methodology

This paper presents a novel computer-aided framework for cardiac image segmentation using a methodology based on memory persistence. The primary concept is to mimic the process of human cognition in the segmentation of time-varying images (i.e., 2D + time or 3D + time), by remembering and exploiting results of previously segmented frames, to aid in segmentation of the region of interest with poor or ambiguous boundaries. The framework involves an intelligent image segmentation process which incorporates an automatic contour initialization mechanism, and a segmentation refinement mechanism that iteratively improves the segmentation results. The proposed framework is general and can integrate most existing image segmentation algorithms in the literature. The experimental results show the benefits of the proposed framework achieving insensitivity to contour initialization, high automation and better segmentation accuracy as compared to the original algorithm and its standard temporal constraint version.

Deep Learning-based Method for Fully Automatic Quantification of Left Ventricle Function from Cine MR Images: A Multivendor, Multicenter Study

Purpose To develop a deep learning-based method for fully automated quantification of left ventricular (LV) function from short-axis cine MR images and to evaluate its performance in a multivendor and multicenter setting. Materials and Methods This retrospective study included cine MRI data sets obtained from three major MRI vendors in four medical centers from 2008 to 2016. Three convolutional neural networks (CNNs) with the U-NET architecture were trained on data sets of increasing variability: (a) a single-vendor, single-center, homogeneous cohort of 100 patients (CNN1); (b) a single-vendor, multicenter, heterogeneous cohort of 200 patients (CNN2); and (c) a multivendor, multicenter, heterogeneous cohort of 400 patients (CNN3). All CNNs were tested on an independent multivendor, multicenter data set of 196 patients. CNN performance was evaluated with respect to the manual annotations from three experienced observers in terms of (a) LV detection accuracy, (b) LV segmentation accuracy, and (c) LV functional parameter accuracy. Automatic and manual results were compared with the paired Wilcoxon test, Pearson correlation, and Bland-Altman analysis. Results CNN3 achieved the highest performance on the independent testing data set. The average perpendicular distance compared with manual analysis was 1.1 mm ± 0.3 for CNN3, compared with 1.5 mm ± 1.0 for CNN1 (P < .05) and 1.3 mm ± 0.6 for CNN2 (P < .05). The LV function parameters derived from CNN3 showed a high correlation (r² ≥ 0.98) and agreement with those obtained by experts for data sets from different vendors and centers. Conclusion A deep learning-based method trained on a data set with high variability can achieve fully automated and accurate cine MRI analysis on multivendor, multicenter cine MRI data. © RSNA, 2018 See also the editorial by Colletti in this issue.

Convolutional Neural Network With Shape Prior Applied to Cardiac MRI Segmentation

In this paper, we present a novel convolutional neural network architecture to segment images from a series of short-axis cardiac magnetic resonance slices (CMRI). The proposed model is an extension of the U-net that embeds a cardiac shape prior and involves a loss function tailored to the cardiac anatomy. Since the shape prior is computed offline only once, the execution of our model is not limited by its calculation. Our system takes as input raw magnetic resonance images, requires no manual preprocessing or image cropping and is trained to segment the endocardium and epicardium of the left ventricle, the endocardium of the right ventricle, as well as the center of the left ventricle. With its multiresolution grid architecture, the network learns both high and low-level features useful to register the shape prior as well as accurately localize the borders of the cardiac regions. Experimental results obtained on the Automatic Cardiac Diagnostic Challenge - Medical Image Computing and Computer Assisted Intervention (ACDC-MICCAI) 2017 dataset show that our model segments multislices CMRI (left and right ventricle contours) in 0.18 s with an average Dice coefficient of [Formula: see text] and an average 3-D Hausdorff distance of [Formula: see text] mm.

Automatic fetal body and amniotic fluid segmentation from fetal ultrasound images by encoder-decoder network with inner layers

This paper explores the effectiveness of applying a deep learning based method to segment the amniotic fluid and fetal tissues in fetal ultrasound (US) images. The deeply learned model firstly encodes the input image into down scaled feature maps by convolution and pooling structures, then up-scale the feature maps to confidence maps by corresponded un-pooling and convolution layers. Additional convolution layers with 1×1 sized kernels are adopted to enhance the feature representations, which could be used to further improve the discriminative learning of our model. We effectively update the weights of the network by fine-tuning on part of the layers from a pre-trained model. By conducting experiments using clinical data, the feasibility of our proposed approach is compared and discussed. The result proves that this work achieves satisfied results for segmentation of specific anatomical structures from US images.

Computational assessment of stomach tumor volume from multi-slice computerized tomography images in presence of type 2 cancer

Background: The multi–slice computerized tomography (MSCT) is a medical imaging modality that has been used to determine the size and location of the stomach cancer. Additionally, MSCT is considered the best modality for the staging of gastric cancer. One way to assess the type 2 cancer of stomach is by detecting the pathological structure with an image segmentation approach. The tumor segmentation of MSCT gastric cancer images enables the diagnosis of the disease condition, for a given patient, without using an invasive method as surgical intervention.

Methods: This approach consists of three stages. The initial stage, an image enhancement, consists of a method for correcting non homogeneities present in the background of MSCT images. Then, a segmentation stage using a clustering method allows to obtain the adenocarcinoma morphology. In the third stage, the pathology region is reconstructed and then visualized with a three–dimensional (3–D) computer graphics procedure based on marching cubes algorithm. In order to validate the segmentations, the Dice score is used as a metric function useful for comparing the segmentations obtained using the proposed method with respect to ground truth volumes traced by a clinician.

Results: A total of 8 datasets available for patients diagnosed, from the cancer data collection of the project, Cancer Genome Atlas Stomach Adenocarcinoma (TCGASTAD) is considered in this research. The volume of the type 2 stomach tumor is estimated from the 3–D shape computationally segmented from the each dataset. These 3–D shapes are computationally reconstructed and then used to assess the morphopathology macroscopic features of this cancer.

Conclusions: The segmentations obtained are useful for assessing qualitatively and quantitatively the stomach type 2 cancer. In addition, this type of segmentation allows the development of computational models that allow the planning of virtual surgical processes related to type 2 cancer.

Computational assessment of stomach tumor volume from multi-slice computerized tomography images in presence of type 2 cancer

Background: The multi-slice computerized tomography (MSCT) is a medical imaging modality that has been used to determine the size and location of the stomach cancer. Additionally, MSCT is considered the best modality for the staging of gastric cancer. One way to assess the type 2 cancer of stomach is by detecting the pathological structure with an image segmentation approach. The tumor segmentation of MSCT gastric cancer images enables the diagnosis of the disease condition, for a given patient, without using an invasive method as surgical intervention. Methods: This approach consists of three stages. The initial stage, an image enhancement, consists of a method for correcting non homogeneities present in the background of MSCT images. Then, a segmentation stage using a clustering method allows to obtain the adenocarcinoma morphology. In the third stage, the pathology region is reconstructed and then visualized with a three-dimensional (3-D) computer graphics procedure based on marching cubes algorithm. In order to validate the segmentations, the Dice score is used as a metric function useful for comparing the segmentations obtained using the proposed method with respect to ground truth volumes traced by a clinician. Results: A total of 8 datasets available for patients diagnosed, from the cancer data collection of the project, Cancer Genome Atlas Stomach Adenocarcinoma (TCGASTAD) is considered in this research. The volume of the type 2 stomach tumor is estimated from the 3-D shape computationally segmented from the each dataset. These 3-D shapes are computationally reconstructed and then used to assess the morphopathology macroscopic features of this cancer. Conclusions: The segmentations obtained are useful for assessing qualitatively and quantitatively the stomach type 2 cancer. In addition, this type of segmentation allows the development of computational models that allow the planning of virtual surgical processes related to type 2 cancer.

Development and clinical validation of a hybrid method for semiautomated left ventricle endocardial and epicardial boundary extraction on cine-magnetic resonance images

We describe a hybrid method for left ventricle (LV) endocardial and epicardial segmentation on cardiac magnetic resonance (CMR) images requiring minimal operator intervention. Endocardium extraction results from the union of three independent estimations based on adaptive thresholding, region growing, and active contour with Chan-Vese energy function. Epicardium segmentation relies on conditional morphological dilation of the endocardial mask followed by active contour optimization. The proposed method was first evaluated using an open access database of 18 CMR for which expert manual contouring was available. The method was further validated on a retrospective cohort of 29 patients, who underwent CMR with expert manual segmentation. Regarding the open access database, similarity (Dice index) between hybrid and expert segmentations was good for end-diastolic (ED) endocardium (0.92), end-systolic (ES) endocardium (0.88), and ED epicardium (0.92). As for derived LV parameters, concordance (Lin's coefficient) was good for ED volume (0.91), ES volume (0.93), ejection fraction (EF; 0.89), and fair for myocardial mass (MM; 0.74). Regarding the retrospective patient study, concordance between expert and hybrid estimations was excellent for ED volume (0.95), ES volume (0.96), good for EF (0.86), and fair for MM (0.71). Hybrid segmentation resulted in small biases ([Formula: see text] for ED volume, [Formula: see text] for ES volume, [Formula: see text] for EF, and [Formula: see text] for MM) with little clinical relevance and acceptable for routine practice. The quickness and robustness of the proposed hybrid method and its ability to provide LV volumes, functions, and masses highly concordant with those given by expert segmentation support its pertinence for routine clinical use.

A Survey of Graph Cuts/Graph Search Based Medical Image Segmentation

Medical image segmentation is a fundamental and challenging problem for analyzing medical images. Among different existing medical image segmentation methods, graph-based approaches are relatively new and show good features in clinical applications. In the graph-based method, pixels or regions in the original image are interpreted into nodes in a graph. By considering Markov random field to model the contexture information of the image, the medical image segmentation problem can be transformed into a graph-based energy minimization problem. This problem can be solved by the use of minimum s-t cut/ maximum flow algorithm. This review is devoted to cut-based medical segmentation methods, including graph cuts and graph search for region and surface segmentation. Different varieties of cut-based methods, including graph-cuts-based methods, model integrated graph cuts methods, graph-search-based methods, and graph search/graph cuts based methods, are systematically reviewed. Graph cuts and graph search with deep learning technique are also discussed.

Fully automated segmentation of the left ventricle in cine cardiac MRI using neural network regression

BACKGROUND

Left ventricle (LV) structure and functions are the primary assessment performed in most clinical cardiac MRI protocols. Fully automated LV segmentation might improve the efficiency and reproducibility of clinical assessment.

PURPOSE

To develop and validate a fully automated neural network regression-based algorithm for segmentation of the LV in cardiac MRI, with full coverage from apex to base across all cardiac phases, utilizing both short axis (SA) and long axis (LA) scans.

STUDY TYPE

Cross-sectional survey; diagnostic accuracy.

SUBJECTS

In all, 200 subjects with coronary artery diseases and regional wall motion abnormalities from the public 2011 Left Ventricle Segmentation Challenge (LVSC) database; 1140 subjects with a mix of normal and abnormal cardiac functions from the public Kaggle Second Annual Data Science Bowl database.

FIELD STRENGTH/SEQUENCE

1.5T, steady-state free precession.

ASSESSMENT

Reference standard data generated by experienced cardiac radiologists. Quantitative measurement and comparison via Jaccard and Dice index, modified Hausdorff distance (MHD), and blood volume.

STATISTICAL TESTS

Paired t-tests compared to previous work.

RESULTS

Tested against the LVSC database, we obtained 0.77 ± 0.11 (Jaccard index) and 1.33 ± 0.71 mm (MHD), both metrics demonstrating statistically significant improvement (P < 0.001) compared to previous work. Tested against the Kaggle database, the signed difference in evaluated blood volume was +7.2 ± 13.0 mL and -19.8 ± 18.8 mL for the end-systolic (ES) and end-diastolic (ED) phases, respectively, with a statistically significant improvement (P < 0.001) for the ED phase.

DATA CONCLUSION

A fully automated LV segmentation algorithm was developed and validated against a diverse set of cardiac cine MRI data sourced from multiple imaging centers and scanner types. The strong performance overall is suggestive of practical clinical utility.

LEVEL OF EVIDENCE

3 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2018.

Multi-Views Fusion CNN for Left Ventricular Volumes Estimation on Cardiac MR Images

OBJECTIVE

Left ventricular (LV) volume estimation is a critical procedure for cardiac disease diagnosis. The objective of this paper is to address a direct LV volume prediction task.

METHODS

In this paper, we propose a direct volume prediction method based on the end-to-end deep convolutional neural networks. We study the end-to-end LV volume prediction method in items of the data preprocessing, network structure, and multiview fusion strategy. The main contributions of this paper are the following aspects. First, we propose a new data preprocessing method on cardiac magnetic resonance (CMR). Second, we propose a new network structure for end-to-end LV volume estimation. Third, we explore the representational capacity of different slices and propose a fusion strategy to improve the prediction accuracy.

RESULTS

The evaluation results show that the proposed method outperforms other state-of-the-art LV volume estimation methods on the open accessible benchmark datasets. The clinical indexes derived from the predicted volumes agree well with the ground truth ( ${\rm{EDV:R}}^{{\rm 2}}={\text{0.974}}$, ${\rm{RMSE\,}}= {\text{9.6}}{\rm{\,ml}}$; ${\rm{ESV:R}}^{{\rm 2}}={\text{0.976}}$, ${\rm{RMSE}}= {\text{7.1}}\,{\text{ml}}$; ${\rm{EF:R}}^{{\rm 2}} ={\text{0.828}}$, ${\rm{RMSE}}= {\text{4.71}}\% $).

CONCLUSION

Experimental results prove that the proposed method may be useful for the LV volume prediction task.

SIGNIFICANCE

The proposed method not only has application potential for cardiac diseases screening for large-scale CMR data, but also can be extended to other medical image research fields.

Deep Learning on Sparse Manifolds for Faster Object Segmentation

We propose a new combination of deep belief networks and sparse manifold learning strategies for the 2D segmentation of non-rigid visual objects. With this novel combination, we aim to reduce the training and inference complexities while maintaining the accuracy of machine learning-based non-rigid segmentation methodologies. Typical non-rigid object segmentation methodologies divide the problem into a rigid detection followed by a non-rigid segmentation, where the low dimensionality of the rigid detection allows for a robust training (i.e., a training that does not require a vast amount of annotated images to estimate robust appearance and shape models) and a fast search process during inference. Therefore, it is desirable that the dimensionality of this rigid transformation space is as small as possible in order to enhance the advantages brought by the aforementioned division of the problem. In this paper, we propose the use of sparse manifolds to reduce the dimensionality of the rigid detection space. Furthermore, we propose the use of deep belief networks to allow for a training process that can produce robust appearance models without the need of large annotated training sets. We test our approach in the segmentation of the left ventricle of the heart from ultrasound images and lips from frontal face images. Our experiments show that the use of sparse manifolds and deep belief networks for the rigid detection stage leads to segmentation results that are as accurate as the current state of the art, but with lower search complexity and training processes that require a small amount of annotated training data.

Cardiac imaging: working towards fully-automated machine analysis & interpretation

INTRODUCTION

Non-invasive imaging plays a critical role in managing patients with cardiovascular disease. Although subjective visual interpretation remains the clinical mainstay, quantitative analysis facilitates objective, evidence-based management, and advances in clinical research. This has driven developments in computing and software tools aimed at achieving fully automated image processing and quantitative analysis. In parallel, machine learning techniques have been used to rapidly integrate large amounts of clinical and quantitative imaging data to provide highly personalized individual patient-based conclusions. Areas covered: This review summarizes recent advances in automated quantitative imaging in cardiology and describes the latest techniques which incorporate machine learning principles. The review focuses on the cardiac imaging techniques which are in wide clinical use. It also discusses key issues and obstacles for these tools to become utilized in mainstream clinical practice. Expert commentary: Fully-automated processing and high-level computer interpretation of cardiac imaging are becoming a reality. Application of machine learning to the vast amounts of quantitative data generated per scan and integration with clinical data also facilitates a move to more patient-specific interpretation. These developments are unlikely to replace interpreting physicians but will provide them with highly accurate tools to detect disease, risk-stratify, and optimize patient-specific treatment. However, with each technological advance, we move further from human dependence and closer to fully-automated machine interpretation.

Automatic segmentation of left ventricle cavity from short-axis cardiac magnetic resonance images

In this paper, a computational framework is proposed to perform a fully automatic segmentation of the left ventricle (LV) cavity from short-axis cardiac magnetic resonance (CMR) images. In the initial phase, the region of interest (ROI) is automatically identified on the first image frame of the CMR slices. This is done by partitioning the image into different regions using a standard fuzzy c-means (FCM) clustering algorithm where the LV region is identified according to its intensity, size and circularity in the image. Next, LV segmentation is performed within the identified ROI by using a novel clustering method that utilizes an objective functional with a dissimilarity measure that incorporates a circular shape function. This circular shape-constrained FCM algorithm is able to differentiate pixels with similar intensity but are located in different regions (e.g. LV cavity and non-LV cavity), thus improving the accuracy of the segmentation even in the presence of papillary muscles. In the final step, the segmented LV cavity is propagated to the adjacent image frame to act as the ROI. The segmentation and ROI propagation are then iteratively executed until the segmentation has been performed for the whole cardiac sequence. Experiment results using the LV Segmentation Challenge validation datasets show that our proposed framework can achieve an average perpendicular distance (APD) shift of 2.23 ± 0.50 mm and the Dice metric (DM) index of 0.89 ± 0.03, which is comparable to the existing cutting edge methods. The added advantage over state of the art is that our approach is fully automatic, does not need manual initialization and does not require a prior trained model.

3D Fast Automatic Segmentation of Kidney Based on Modified AAM and Random Forest

In this paper, a fully automatic method is proposed to segment the kidney into multiple components: renal cortex, renal column, renal medulla and renal pelvis, in clinical 3D CT abdominal images. The proposed fast automatic segmentation method of kidney consists of two main parts: localization of renal cortex and segmentation of kidney components. In the localization of renal cortex phase, a method which fully combines 3D Generalized Hough Transform (GHT) and 3D Active Appearance Models (AAM) is applied to localize the renal cortex. In the segmentation of kidney components phase, a modified Random Forests (RF) method is proposed to segment the kidney into four components based on the result from localization phase. During the implementation, a multithreading technology is applied to speed up the segmentation process. The proposed method was evaluated on a clinical abdomen CT data set, including 37 contrast-enhanced volume data using leave-one-out strategy. The overall true-positive volume fraction and false-positive volume fraction were 93.15%, 0.37% for renal cortex segmentation; 83.09%, 0.97% for renal column segmentation; 81.92%, 0.55% for renal medulla segmentation; and 80.28%, 0.30% for renal pelvis segmentation, respectively. The average computational time of segmenting kidney into four components took 20 seconds.

A review of heart chamber segmentation for structural and functional analysis using cardiac magnetic resonance imaging

Cardiovascular magnetic resonance (CMR) has become a key imaging modality in clinical cardiology practice due to its unique capabilities for non-invasive imaging of the cardiac chambers and great vessels. A wide range of CMR sequences have been developed to assess various aspects of cardiac structure and function, and significant advances have also been made in terms of imaging quality and acquisition times. A lot of research has been dedicated to the development of global and regional quantitative CMR indices that help the distinction between health and pathology. The goal of this review paper is to discuss the structural and functional CMR indices that have been proposed thus far for clinical assessment of the cardiac chambers. We include indices definitions, the requirements for the calculations, exemplar applications in cardiovascular diseases, and the corresponding normal ranges. Furthermore, we review the most recent state-of-the art techniques for the automatic segmentation of the cardiac boundaries, which are necessary for the calculation of the CMR indices. Finally, we provide a detailed discussion of the existing literature and of the future challenges that need to be addressed to enable a more robust and comprehensive assessment of the cardiac chambers in clinical practice.

An Efficient Object Tracking Method on Quad-/Oc-Trees

We introduce a fast error-free tracking method applicable to sequences of two and three dimensional images. The core idea is to use Quadtree (resp. Octree) data structures for representing the spatial discretization of an image in two (resp. three) spatial dimensions. This representation enables one to merge into large computational cells the regions that can be faithfully described with such a coarse representation, thus significantly reducing the total number of degrees of freedom that are processed, without compromising accuracy. This encoding is particularly effective in the case of algorithms based on moving fronts, since the adaptive refinement provides a natural means to focus the processing resources on information near the moving front. In this paper, we use an existing contour based tracker and reformulate it to the case of Quad-/Oc-tree data structures. Relevant mathematical assumptions and derivations are presented for this purpose. We then demonstrate that, on standard bio-medical image sequences, a speed up of 5X is easily achieved in 2D and about 10X in 3D.

An Algorithm for the Segmentation of Highly Abnormal Hearts Using a Generic Statistical Shape Model

Statistical shape models (SSMs) have been widely employed in cardiac image segmentation. However, in conditions that induce severe shape abnormality and remodeling, such as in the case of pulmonary hypertension (PH) or hypertrophic cardiomyopathy (HCM), a single SSM is rarely capable of capturing the anatomical variability in the extremes of the distribution. This work presents a new algorithm for the segmentation of severely abnormal hearts. The algorithm is highly flexible, as it does not require a priori knowledge of the involved pathology or any specific parameter tuning to be applied to the cardiac image under analysis. The fundamental idea is to approximate the gross effect of the abnormality with a virtual remodeling transformation between the patient-specific geometry and the average shape of the reference model (e.g., average normal morphology). To define this mapping, a set of landmark points are automatically identified during boundary point search, by estimating the reliability of the candidate points. With the obtained transformation, the feature points extracted from the patient image volume are then projected onto the space of the reference SSM, where the model is used to effectively constrain and guide the segmentation process. The extracted shape in the reference space is finally propagated back to the original image of the abnormal heart to obtain the final segmentation. Detailed validation with patients diagnosed with PH and HCM shows the robustness and flexibility of the technique for the segmentation of highly abnormal hearts of different pathologies.

Clinical feasibility of a myocardial signal intensity threshold-based semi-automated cardiac magnetic resonance segmentation method

OBJECTIVES

To assess the accuracy and efficiency of a threshold-based, semi-automated cardiac MRI segmentation algorithm in comparison with conventional contour-based segmentation and aortic flow measurements.

METHODS

Short-axis cine images of 148 patients (55 ± 18 years, 81 men) were used to evaluate left ventricular (LV) volumes and mass (LVM) using conventional and threshold-based segmentations. Phase-contrast images were used to independently measure stroke volume (SV). LV parameters were evaluated by two independent readers.

RESULTS

Evaluation times using the conventional and threshold-based methods were 8.4 ± 1.9 and 4.2 ± 1.3 min, respectively (P < 0.0001). LV parameters measured by the conventional and threshold-based methods, respectively, were end-diastolic volume (EDV) 146 ± 59 and 134 ± 53 ml; end-systolic volume (ESV) 64 ± 47 and 59 ± 46 ml; SV 82 ± 29 and 74 ± 28 ml (flow-based 74 ± 30 ml); ejection fraction (EF) 59 ± 16 and 58 ± 17%; and LVM 141 ± 55 and 159 ± 58 g. Significant differences between the conventional and threshold-based methods were observed in EDV, ESV, and LVM mesurements; SV from threshold-based and flow-based measurements were in agreement (P > 0.05) but were significantly different from conventional analysis (P < 0.05). Excellent inter-observer agreement was observed.

CONCLUSIONS

Threshold-based LV segmentation provides improved accuracy and faster assessment compared to conventional contour-based methods.

KEY POINTS

• Threshold-based left ventricular segmentation provides time-efficient assessment of left ventricular parameters • The threshold-based method can discriminate between blood and papillary muscles • This method provides improved accuracy compared to aortic flow measurements as a reference.

Computer Vision Techniques for Transcatheter Intervention

Minimally invasive transcatheter technologies have demonstrated substantial promise for the diagnosis and the treatment of cardiovascular diseases. For example, transcatheter aortic valve implantation is an alternative to aortic valve replacement for the treatment of severe aortic stenosis, and transcatheter atrial fibrillation ablation is widely used for the treatment and the cure of atrial fibrillation. In addition, catheter-based intravascular ultrasound and optical coherence tomography imaging of coronary arteries provides important information about the coronary lumen, wall, and plaque characteristics. Qualitative and quantitative analysis of these cross-sectional image data will be beneficial to the evaluation and the treatment of coronary artery diseases such as atherosclerosis. In all the phases (preoperative, intraoperative, and postoperative) during the transcatheter intervention procedure, computer vision techniques (e.g., image segmentation and motion tracking) have been largely applied in the field to accomplish tasks like annulus measurement, valve selection, catheter placement control, and vessel centerline extraction. This provides beneficial guidance for the clinicians in surgical planning, disease diagnosis, and treatment assessment. In this paper, we present a systematical review on these state-of-the-art methods. We aim to give a comprehensive overview for researchers in the area of computer vision on the subject of transcatheter intervention. Research in medical computing is multi-disciplinary due to its nature, and hence, it is important to understand the application domain, clinical background, and imaging modality, so that methods and quantitative measurements derived from analyzing the imaging data are appropriate and meaningful. We thus provide an overview on the background information of the transcatheter intervention procedures, as well as a review of the computer vision techniques and methodologies applied in this area.

Right ventricular segmentation in cardiac MRI with moving mesh correspondences

This study investigates automatic propagation of the right ventricle (RV) endocardial and epicardial boundaries in 4D (3D+time) magnetic resonance imaging (MRI) sequences. Based on a moving mesh (or grid generation) framework, the proposed algorithm detects the endocardium and epicardium within each cardiac phase via point-to-point correspondences. The proposed method has the following advantages over prior RV segmentation works: (1) it removes the need for a time-consuming, manually built training set; (2) it does not make prior assumptions as to the intensity distributions or shape; (3) it provides a sequence of corresponding points over time, a comprehensive input that can be very useful in cardiac applications other than segmentation, e.g., regional wall motion analysis; and (4) it is more flexible for congenital heart disease where the RV undergoes high variations in shape. Furthermore, the proposed method allows comprehensive RV volumetric analysis over the complete cardiac cycle as well as automatic detections of end-systolic and end-diastolic phases because it provides a segmentation for each time step. Evaluated quantitatively over the 48-subject data set of the MICCAI 2012 RV segmentation challenge, the proposed method yielded an average Dice score of 0.84±0.11 for the epicardium and 0.79±0.17 for the endocardium. Further, quantitative evaluations of the proposed approach in comparisons to manual contours over 23 infant hypoplastic left heart syndrome patients yielded a Dice score of 0.82±0.14, which demonstrates the robustness of the algorithm.

Mathematical model in left ventricle segmentation

In this paper a parametric model of the left ventricle is presented. Its task is to estimate the myocardium shape on those slices, on which the segmentation algorithm has outlined the structure incorrectly. The aim of using the model on improperly segmented slices is to improve the accuracy of computing cardiac hemodynamic parameters and the heart mass. The proposed model works with any segmentation algorithm. The usefulness of the model is the largest while determining the myocardium at end-systole and calculating the heart mass. In case of the segmentation algorithm applied in this study, the error decreased from clinically unacceptable to acceptable after using the presented model.

A novel level set method for segmentation of left and right ventricles from cardiac MR images

In this paper, we propose a novel level set method for segmentation of cardiac left and right ventricles based on the distance regularized level set evolution (DRLSE) framework [7] and the distance regularized two-layer level set (DR2LS) model [17]. First, DRLSE is applied to obtain a preliminary segmentation of left and right ventricles, which is then used to initialize the endocardial contour, which is represented by the zero level contour of the level set function in our method. Then, the epi-cardial contour is represented by a different level contour of the same level set function. These two level sets are optimized by an energy minimization process to best fit the true endocardium and epicardium. In order to ensure smoothly varying distance between the two level contours, we introduce a distance regularization constraint in the energy function. With the region-scalable fitting (RSF) energy [8] as the data term, our method is able to deal with intensity inhomogeneities in the images, which is a main source of difficulty in image segmentation. Our method has been tested on cardiac MR images with promising results.

Fully automatic detection of salient features in 3-d transesophageal images

Most automated segmentation approaches to the mitral valve and left ventricle in 3-D echocardiography require a manual initialization. In this article, we propose a fully automatic scheme to initialize a multicavity segmentation approach in 3-D transesophageal echocardiography by detecting the left ventricle long axis, the mitral valve and the aortic valve location. Our approach uses a probabilistic and structural tissue classification to find structures such as the mitral and aortic valves; the Hough transform for circles to find the center of the left ventricle; and multidimensional dynamic programming to find the best position for the left ventricle long axis. For accuracy and agreement assessment, the proposed method was evaluated in 19 patients with respect to manual landmarks and as initialization of a multicavity segmentation approach for the left ventricle, the right ventricle, the left atrium, the right atrium and the aorta. The segmentation results revealed no statistically significant differences between manual and automated initialization in a paired t-test (p > 0.05). Additionally, small biases between manual and automated initialization were detected in the Bland-Altman analysis (bias, variance) for the left ventricle (-0.04, 0.10); right ventricle (-0.07, 0.18); left atrium (-0.01, 0.03); right atrium (-0.04, 0.13); and aorta (-0.05, 0.14). These results indicate that the proposed approach provides robust and accurate detection to initialize a multicavity segmentation approach without any user interaction.

A Hybrid Method for Endocardial Contour Extraction of Right Ventricle in 4-Slices from 3D Echocardiography Dataset

This paper presents a hybrid method to extract endocardial contour of the right ventricular (RV) in 4-slices from 3D echocardiography dataset. The overall framework comprises four processing phases. In Phase I, the region of interest (ROI) is identified by estimating the cavity boundary. Speckle noise reduction and contrast enhancement were implemented in Phase II as preprocessing tasks. In Phase III, the RV cavity region was segmented by generating intensity threshold which was used for once for all frames. Finally, Phase IV is proposed to extract the RV endocardial contour in a complete cardiac cycle using a combination of shape-based contour detection and improved radial search algorithm. The proposed method was applied to 16 datasets of 3D echocardiography encompassing the RV in long-axis view. The accuracy of experimental results obtained by the proposed method was evaluated qualitatively and quantitatively. It has been done by comparing the segmentation results of RV cavity based on endocardial contour extraction with the ground truth. The comparative analysis results show that the proposed method performs efficiently in all datasets with overall performance of 95% and the root mean square distances (RMSD) measure in terms of mean ± SD was found to be 2.21 ± 0.35 mm for RV endocardial contours.