Four-chamber heart modeling and automatic segmentation for 3-D cardiac CT volumes using marginal space learning and steerable features

Marginal Shape Deep Learning: Applications to Pediatric Lung Field Segmentation

Representation learning through deep learning (DL) architecture has shown tremendous potential for identification, localization, and texture classification in various medical imaging modalities. However, DL applications to segmentation of objects especially to deformable objects are rather limited and mostly restricted to pixel classification. In this work, we propose marginal shape deep learning (MaShDL), a framework that extends the application of DL to deformable shape segmentation by using deep classifiers to estimate the shape parameters. MaShDL combines the strength of statistical shape models with the automated feature learning architecture of DL. Unlike the iterative shape parameters estimation approach of classical shape models that often leads to a local minima, the proposed framework is robust to local minima optimization and illumination changes. Furthermore, since the direct application of DL framework to a multi-parameter estimation problem results in a very high complexity, our framework provides an excellent run-time performance solution by independently learning shape parameter classifiers in marginal eigenspaces in the decreasing order of variation. We evaluated MaShDL for segmenting the lung field from 314 normal and abnormal pediatric chest radiographs and obtained a mean Dice similarity coefficient of 0.927 using only the four highest modes of variation (compared to 0.888 with classical ASM1 (p-value=0.01) using same configuration). To the best of our knowledge this is the first demonstration of using DL framework for parametrized shape learning for the delineation of deformable objects.

An evaluation of automatic coronary artery calcium scoring methods with cardiac CT using the orCaScore framework

PURPOSE

The amount of coronary artery calcification (CAC) is a strong and independent predictor of cardiovascular disease (CVD) events. In clinical practice, CAC is manually identified and automatically quantified in cardiac CT using commercially available software. This is a tedious and time-consuming process in large-scale studies. Therefore, a number of automatic methods that require no interaction and semiautomatic methods that require very limited interaction for the identification of CAC in cardiac CT have been proposed. Thus far, a comparison of their performance has been lacking. The objective of this study was to perform an independent evaluation of (semi)automatic methods for CAC scoring in cardiac CT using a publicly available standardized framework.

METHODS

Cardiac CT exams of 72 patients distributed over four CVD risk categories were provided for (semi)automatic CAC scoring. Each exam consisted of a noncontrast-enhanced calcium scoring CT (CSCT) and a corresponding coronary CT angiography (CCTA) scan. The exams were acquired in four different hospitals using state-of-the-art equipment from four major CT scanner vendors. The data were divided into 32 training exams and 40 test exams. A reference standard for CAC in CSCT was defined by consensus of two experts following a clinical protocol. The framework organizers evaluated the performance of (semi)automatic methods on test CSCT scans, per lesion, artery, and patient.

RESULTS

Five (semi)automatic methods were evaluated. Four methods used both CSCT and CCTA to identify CAC, and one method used only CSCT. The evaluated methods correctly detected between 52% and 94% of CAC lesions with positive predictive values between 65% and 96%. Lesions in distal coronary arteries were most commonly missed and aortic calcifications close to the coronary ostia were the most common false positive errors. The majority (between 88% and 98%) of correctly identified CAC lesions were assigned to the correct artery. Linearly weighted Cohen's kappa for patient CVD risk categorization by the evaluated methods ranged from 0.80 to 1.00.

CONCLUSIONS

A publicly available standardized framework for the evaluation of (semi)automatic methods for CAC identification in cardiac CT is described. An evaluation of five (semi)automatic methods within this framework shows that automatic per patient CVD risk categorization is feasible. CAC lesions at ambiguous locations such as the coronary ostia remain challenging, but their detection had limited impact on CVD risk determination.

Comprehensive Modeling and Visualization of Cardiac Anatomy and Physiology from CT Imaging and Computer Simulations

In clinical cardiology, both anatomy and physiology are needed to diagnose cardiac pathologies. CT imaging and computer simulations provide valuable and complementary data for this purpose. However, it remains challenging to gain useful information from the large amount of high-dimensional diverse data. The current tools are not adequately integrated to visualize anatomic and physiologic data from a complete yet focused perspective. We introduce a new computer-aided diagnosis framework, which allows for comprehensive modeling and visualization of cardiac anatomy and physiology from CT imaging data and computer simulations, with a primary focus on ischemic heart disease. The following visual information is presented: (1) Anatomy from CT imaging: geometric modeling and visualization of cardiac anatomy, including four heart chambers, left and right ventricular outflow tracts, and coronary arteries; (2) Function from CT imaging: motion modeling, strain calculation, and visualization of four heart chambers; (3) Physiology from CT imaging: quantification and visualization of myocardial perfusion and contextual integration with coronary artery anatomy; (4) Physiology from computer simulation: computation and visualization of hemodynamics (e.g., coronary blood velocity, pressure, shear stress, and fluid forces on the vessel wall). Substantially, feedback from cardiologists have confirmed the practical utility of integrating these features for the purpose of computer-aided diagnosis of ischemic heart disease.

Automated detection of coarctation of aorta in neonates from two-dimensional echocardiograms

Coarctation of aorta (CoA) is a critical congenital heart defect (CCHD) that requires accurate and immediate diagnosis and treatment. Current newborn screening methods to detect CoA lack both in sensitivity and specificity, and when suspected in a newborn, it must be confirmed using specialized imaging and expert diagnosis, both of which are usually unavailable at tertiary birthing centers. We explore the feasibility of applying machine learning methods to reliably determine the presence of this difficult-to-diagnose cardiac abnormality from ultrasound image data. We propose a framework that uses deep learning-based machine learning methods for fully automated detection of CoA from two-dimensional ultrasound clinical data acquired in the parasternal long axis view, the apical four chamber view, and the suprasternal notch view. On a validation set consisting of 26 CoA and 64 normal patients our algorithm achieved a total error rate of 12.9% (11.5% false-negative error and 13.6% false-positive error) when combining decisions of classifiers over three standard echocardiographic view planes. This compares favorably with published results that combine clinical assessments with pulse oximetry to detect CoA (71% sensitivity).

A Bottom-Up Approach for Pancreas Segmentation Using Cascaded Superpixels and (Deep) Image Patch Labeling

Robust organ segmentation is a prerequisite for computer-aided diagnosis, quantitative imaging analysis, pathology detection, and surgical assistance. For organs with high anatomical variability (e.g., the pancreas), previous segmentation approaches report low accuracies, compared with well-studied organs, such as the liver or heart. We present an automated bottom-up approach for pancreas segmentation in abdominal computed tomography (CT) scans. The method generates a hierarchical cascade of information propagation by classifying image patches at different resolutions and cascading (segments) superpixels. The system contains four steps: 1) decomposition of CT slice images into a set of disjoint boundary-preserving superpixels; 2) computation of pancreas class probability maps via dense patch labeling; 3) superpixel classification by pooling both intensity and probability features to form empirical statistics in cascaded random forest frameworks; and 4) simple connectivity based post-processing. Dense image patch labeling is conducted using two methods: efficient random forest classification on image histogram, location and texture features; and more expensive (but more accurate) deep convolutional neural network classification, on larger image windows (i.e., with more spatial contexts). Over-segmented 2-D CT slices by the simple linear iterative clustering approach are adopted through model/parameter calibration and labeled at the superpixel level for positive (pancreas) or negative (non-pancreas or background) classes. The proposed method is evaluated on a data set of 80 manually segmented CT volumes, using six-fold cross-validation. Its performance equals or surpasses other state-of-the-art methods (evaluated by "leave-one-patient-out"), with a dice coefficient of 70.7% and Jaccard index of 57.9%. In addition, the computational efficiency has improved significantly, requiring a mere 6 ~ 8 min per testing case, versus ≥ 10 h for other methods. The segmentation framework using deep patch labeling confidences is also more numerically stable, as reflected in the smaller performance metric standard deviations. Finally, we implement a multi-atlas label fusion (MALF) approach for pancreas segmentation using the same data set. Under six-fold cross-validation, our bottom-up segmentation method significantly outperforms its MALF counterpart: 70.7±13.0% versus 52.51±20.84% in dice coefficients.

Fast automated segmentation of multiple objects via spatially weighted shape learning

Active shape models (ASMs) have proved successful in automatic segmentation by using shape and appearance priors in a number of areas such as prostate segmentation, where accurate contouring is important in treatment planning for prostate cancer. The ASM approach however, is heavily reliant on a good initialisation for achieving high segmentation quality. This initialisation often requires algorithms with high computational complexity, such as three dimensional (3D) image registration. In this work, we present a fast, self-initialised ASM approach that simultaneously fits multiple objects hierarchically controlled by spatially weighted shape learning. Prominent objects are targeted initially and spatial weights are progressively adjusted so that the next (more difficult, less visible) object is simultaneously initialised using a series of weighted shape models. The scheme was validated and compared to a multi-atlas approach on 3D magnetic resonance (MR) images of 38 cancer patients and had the same (mean, median, inter-rater) Dice's similarity coefficients of (0.79, 0.81, 0.85), while having no registration error and a computational time of 12-15 min, nearly an order of magnitude faster than the multi-atlas approach.

Automatic multiorgan segmentation in CT images of the male pelvis using region-specific hierarchical appearance cluster models

PURPOSE

Accurate segmentation of pelvic organs in CT images is of great importance in external beam radiotherapy for prostate cancer. The aim of this studying is to develop a novel method for automatic, multiorgan segmentation of the male pelvis.

METHODS

The authors' segmentation method consists of several stages. First, a pretreatment includes parameterization, principal component analysis (PCA), and an established process of region-specific hierarchical appearance cluster (RSHAC) model which was executed on the training dataset. After the preprocessing, online automatic segmentation of new CT images is achieved by combining the RSHAC model with the PCA-based point distribution model. Fifty pelvic CT from eight prostate cancer patients were used as the training dataset. From another 20 prostate cancer patients, 210 CT images were used for independent validation of the segmentation method.

RESULTS

In the training dataset, 15 PCA modes were needed to represent 95% of shape variations of pelvic organs. When tested on the validation dataset, the authors' segmentation method had an average Dice similarity coefficient and mean absolute distance of 0.751 and 0.371 cm, 0.783 and 0.303 cm, 0.573 and 0.604 cm for prostate, bladder, and rectum, respectively. The automated segmentation process took on average 5 min on a personal computer equipped with Core 2 Duo CPU of 2.8 GHz and 8 GB RAM.

CONCLUSIONS

The authors have developed an efficient and reliable method for automatic segmentation of multiple organs in the male pelvis. This method should be useful for treatment planning and adaptive replanning for prostate cancer radiotherapy. With this method, the physicist can improve the work efficiency and stability.

Unsupervised Freeview Groupwise Cardiac Segmentation Using Synchronized Spectral Network

The diagnosis, comparative and population study of cardiac radiology data require heart segmentation on increasingly large amount of images from different modalities/chambers/patients under various imaging views. Most existing automatic cardiac segmentation methods are often limited to single image segmentation with regulated modality/region settings or well-cropped ROI areas, which is impossible for large datasets due to enormous device protocols and institutional differences. A pure data-driven unsupervised segmentation without regulated setting requirements is crucial in this scenario, and will significantly automate the manual work and adopt the various changes of modality, subject or view. In this paper, we propose a general unsupervised groupwise segmentation: a direct simultaneous segmentation for a group of multi-modality, multi-chamber, multi-subject ( M³) cardiac images from a freely chosen imaging view. The segmentation can directly perform not only on regulated two/four-chamber images, but also on non-regulated uncropped raw MR/CT scans. A new Synchronized Spectral Network (SSN) is developed for the simultaneous decomposing, synchronizing, and clustering the spectral features of free-view M³ cardiac images. The SSN-based groupwise analysis of image spectral bases immediately leads to groupwise segmentation of M³ freeview images. The segmentation is quantitatively evaluated by three datasets (MR and CT mixed) with more than 200 subjects. High dice metric ( ) is consistently achieved in validation. Our method provides a powerful tool for medical images under general imaging environment.

Abdomen and spinal cord segmentation with augmented active shape models

Active shape models (ASMs) have been widely used for extracting human anatomies in medical images given their capability for shape regularization of topology preservation. However, sensitivity to model initialization and local correspondence search often undermines their performances, especially around highly variable contexts in computed-tomography (CT) and magnetic resonance (MR) images. In this study, we propose an augmented ASM (AASM) by integrating the multiatlas label fusion (MALF) and level set (LS) techniques into the traditional ASM framework. Using AASM, landmark updates are optimized globally via a region-based LS evolution applied on the probability map generated from MALF. This augmentation effectively extends the searching range of correspondent landmarks while reducing sensitivity to the image contexts and improves the segmentation robustness. We propose the AASM framework as a two-dimensional segmentation technique targeting structures with one axis of regularity. We apply AASM approach to abdomen CT and spinal cord (SC) MR segmentation challenges. On 20 CT scans, the AASM segmentation of the whole abdominal wall enables the subcutaneous/visceral fat measurement, with high correlation to the measurement derived from manual segmentation. On 28 3T MR scans, AASM yields better performances than other state-of-the-art approaches in segmenting white/gray matter in SC.

Evaluation and comparison of 3D intervertebral disc localization and segmentation methods for 3D T2 MR data: A grand challenge

The evaluation of changes in Intervertebral Discs (IVDs) with 3D Magnetic Resonance (MR) Imaging (MRI) can be of interest for many clinical applications. This paper presents the evaluation of both IVD localization and IVD segmentation methods submitted to the Automatic 3D MRI IVD Localization and Segmentation challenge, held at the 2015 International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI2015) with an on-site competition. With the construction of a manually annotated reference data set composed of 25 3D T2-weighted MR images acquired from two different studies and the establishment of a standard validation framework, quantitative evaluation was performed to compare the results of methods submitted to the challenge. Experimental results show that overall the best localization method achieves a mean localization distance of 0.8 mm and the best segmentation method achieves a mean Dice of 91.8%, a mean average absolute distance of 1.1 mm and a mean Hausdorff distance of 4.3 mm, respectively. The strengths and drawbacks of each method are discussed, which provides insights into the performance of different IVD localization and segmentation methods.

Deep Learning Guided Partitioned Shape Model for Anterior Visual Pathway Segmentation

Analysis of cranial nerve systems, such as the anterior visual pathway (AVP), from MRI sequences is challenging due to their thin long architecture, structural variations along the path, and low contrast with adjacent anatomic structures. Segmentation of a pathologic AVP (e.g., with low-grade gliomas) poses additional challenges. In this work, we propose a fully automated partitioned shape model segmentation mechanism for AVP steered by multiple MRI sequences and deep learning features. Employing deep learning feature representation, this framework presents a joint partitioned statistical shape model able to deal with healthy and pathological AVP. The deep learning assistance is particularly useful in the poor contrast regions, such as optic tracts and pathological areas. Our main contributions are: 1) a fast and robust shape localization method using conditional space deep learning, 2) a volumetric multiscale curvelet transform-based intensity normalization method for robust statistical model, and 3) optimally partitioned statistical shape and appearance models based on regional shape variations for greater local flexibility. Our method was evaluated on MRI sequences obtained from 165 pediatric subjects. A mean Dice similarity coefficient of 0.779 was obtained for the segmentation of the entire AVP (optic nerve only =0.791 ) using the leave-one-out validation. Results demonstrated that the proposed localized shape and sparse appearance-based learning approach significantly outperforms current state-of-the-art segmentation approaches and is as robust as the manual segmentation.

Accurate Segmentation of CT Male Pelvic Organs via Regression-Based Deformable Models and Multi-Task Random Forests

Segmenting male pelvic organs from CT images is a prerequisite for prostate cancer radiotherapy. The efficacy of radiation treatment highly depends on segmentation accuracy. However, accurate segmentation of male pelvic organs is challenging due to low tissue contrast of CT images, as well as large variations of shape and appearance of the pelvic organs. Among existing segmentation methods, deformable models are the most popular, as shape prior can be easily incorporated to regularize the segmentation. Nonetheless, the sensitivity to initialization often limits their performance, especially for segmenting organs with large shape variations. In this paper, we propose a novel approach to guide deformable models, thus making them robust against arbitrary initializations. Specifically, we learn a displacement regressor, which predicts 3D displacement from any image voxel to the target organ boundary based on the local patch appearance. This regressor provides a non-local external force for each vertex of deformable model, thus overcoming the initialization problem suffered by the traditional deformable models. To learn a reliable displacement regressor, two strategies are particularly proposed. 1) A multi-task random forest is proposed to learn the displacement regressor jointly with the organ classifier; 2) an auto-context model is used to iteratively enforce structural information during voxel-wise prediction. Extensive experiments on 313 planning CT scans of 313 patients show that our method achieves better results than alternative classification or regression based methods, and also several other existing methods in CT pelvic organ segmentation.

Marginal Space Deep Learning: Efficient Architecture for Volumetric Image Parsing

Robust and fast solutions for anatomical object detection and segmentation support the entire clinical workflow from diagnosis, patient stratification, therapy planning, intervention and follow-up. Current state-of-the-art techniques for parsing volumetric medical image data are typically based on machine learning methods that exploit large annotated image databases. Two main challenges need to be addressed, these are the efficiency in scanning high-dimensional parametric spaces and the need for representative image features which require significant efforts of manual engineering. We propose a pipeline for object detection and segmentation in the context of volumetric image parsing, solving a two-step learning problem: anatomical pose estimation and boundary delineation. For this task we introduce Marginal Space Deep Learning (MSDL), a novel framework exploiting both the strengths of efficient object parametrization in hierarchical marginal spaces and the automated feature design of Deep Learning (DL) network architectures. In the 3D context, the application of deep learning systems is limited by the very high complexity of the parametrization. More specifically 9 parameters are necessary to describe a restricted affine transformation in 3D, resulting in a prohibitive amount of billions of scanning hypotheses. The mechanism of marginal space learning provides excellent run-time performance by learning classifiers in clustered, high-probability regions in spaces of gradually increasing dimensionality. To further increase computational efficiency and robustness, in our system we learn sparse adaptive data sampling patterns that automatically capture the structure of the input. Given the object localization, we propose a DL-based active shape model to estimate the non-rigid object boundary. Experimental results are presented on the aortic valve in ultrasound using an extensive dataset of 2891 volumes from 869 patients, showing significant improvements of up to 45.2% over the state-of-the-art. To our knowledge, this is the first successful demonstration of the DL potential to detection and segmentation in full 3D data with parametrized representations.

A self-taught artificial agent for multi-physics computational model personalization

Personalization is the process of fitting a model to patient data, a critical step towards application of multi-physics computational models in clinical practice. Designing robust personalization algorithms is often a tedious, time-consuming, model- and data-specific process. We propose to use artificial intelligence concepts to learn this task, inspired by how human experts manually perform it. The problem is reformulated in terms of reinforcement learning. In an off-line phase, Vito, our self-taught artificial agent, learns a representative decision process model through exploration of the computational model: it learns how the model behaves under change of parameters. The agent then automatically learns an optimal strategy for on-line personalization. The algorithm is model-independent; applying it to a new model requires only adjusting few hyper-parameters of the agent and defining the observations to match. The full knowledge of the model itself is not required. Vito was tested in a synthetic scenario, showing that it could learn how to optimize cost functions generically. Then Vito was applied to the inverse problem of cardiac electrophysiology and the personalization of a whole-body circulation model. The obtained results suggested that Vito could achieve equivalent, if not better goodness of fit than standard methods, while being more robust (up to 11% higher success rates) and with faster (up to seven times) convergence rate. Our artificial intelligence approach could thus make personalization algorithms generalizable and self-adaptable to any patient and any model.

Semiautomated Global Quantification of Left Ventricular Myocardial Perfusion at Stress Dynamic CT:: Diagnostic Accuracy for Detection of Territorial Myocardial Perfusion Deficits Compared to Visual Assessment

RATIONALE AND OBJECTIVES

To evaluate the diagnostic accuracy of semiautomated global quantification of left ventricular myocardial perfusion derived from stress dynamic computed tomography myocardial perfusion imaging (CTMPI) for detection of territorial perfusion deficits (PD).

MATERIALS AND METHODS

Dynamic CTMPI datasets of 71 patients were analyzed using semiautomated volume-based software to calculate global myocardial blood flow (MBF), myocardial blood volume, and volume transfer constant. Optimal cutoff values to assess the diagnostic accuracy of these parameters for detection of one- to three-vessel territories with PD in comparison to visual analysis were calculated.

RESULTS

Nonsignificant differences (P = 0.694) were found for average global MBF in patients without PD and single-territorial PD. Significant differences were found for mean global MBF in patients with PD in two (P < 0.0058) and three territories (P < 0.0003). Calculated optimal thresholds for global MBF and myocardial blood volume resulted in a sensitivity, specificity, and negative predictive value of 100% for detection of three-vessel territory PD. For detection of ≥2 territories with PD, global MBF was superior to other parameters (sensitivity 81.3%, specificity 90.9%, and negative predictive value 94.3%).

CONCLUSIONS

Semiautomated global quantification of left ventricular MBF during stress dynamic CTMPI shows high diagnostic accuracy for detection of ≥2 vessel territories with PD, facilitating identification of patients with multi-territorial myocardial PD.

Noninvasive hemodynamic assessment, treatment outcome prediction and follow-up of aortic coarctation from MR imaging

PURPOSE

Coarctation of the aorta (CoA) is a congenital heart disease characterized by an abnormal narrowing of the proximal descending aorta. Severity of this pathology is quantified by the blood pressure drop (△P) across the stenotic coarctation lesion. In order to evaluate the physiological significance of the preoperative coarctation and to assess the postoperative results, the hemodynamic analysis is routinely performed by measuring the △P across the coarctation site via invasive cardiac catheterization. The focus of this work is to present an alternative, noninvasive measurement of blood pressure drop △P through the introduction of a fast, image-based workflow for personalized computational modeling of the CoA hemodynamics.

METHODS

The authors propose an end-to-end system comprised of shape and computational models, their personalization setup using MR imaging, and a fast, noninvasive method based on computational fluid dynamics (CFD) to estimate the pre- and postoperative hemodynamics for coarctation patients. A virtual treatment method is investigated to assess the predictive power of our approach.

RESULTS

Automatic thoracic aorta segmentation was applied on a population of 212 3D MR volumes, with mean symmetric point-to-mesh error of 3.00 ± 1.58 mm and average computation time of 8 s. Through quantitative evaluation of 6 CoA patients, good agreement between computed blood pressure drop and catheter measurements is shown: average differences are 2.38 ± 0.82 mm Hg (pre-), 1.10 ± 0.63 mm Hg (postoperative), and 4.99 ± 3.00 mm Hg (virtual stenting), respectively.

CONCLUSIONS

The complete workflow is realized in a fast, mostly-automated system that is integrable in the clinical setting. To the best of our knowledge, this is the first time that three different settings (preoperative--severity assessment, poststenting--follow-up, and virtual stenting--treatment outcome prediction) of CoA are investigated on multiple subjects. We believe that in future-given wider clinical validation-our noninvasive in-silico method could replace invasive pressure catheterization for CoA.

Automatic localization of mitral valve orifice in three-dimensional left cardiac model

This study proposes a novel method to localize the mitral valve orifice in the three-dimensional left cardiac model reconstructed from Cardiac Magnetic Resonance (CMR) images. After acquiring both short axis and long axis CMR images, endocardium contours were delineated on all images while additional six points were marked to identify the mitral valve orifice on three long axis images. Contours from long axis images were registered to the short axis contours. The resulting registration parameters were stored. A three-dimensional surface model of the left cardiac structure was then reconstructed from the short axis contours. The six points representing mitral valve orifice in long axis images were projected onto the surface model using the registration parameters. A variational method was then applied to localize the mitral valve orifice on the surface model via minimizing the geodesic distance. Numerical examples show the robustness and effectiveness of the proposed method. Automatic location of mitral annulus orifice with the computational time per data set (22 frames) of five minutes would hold clinical potential as a real-time mitral valve assessment tool.

Collaborative regression-based anatomical landmark detection

Anatomical landmark detection plays an important role in medical image analysis, e.g. for registration, segmentation and quantitative analysis. Among the various existing methods for landmark detection, regression-based methods have recently attracted much attention due to their robustness and efficiency. In these methods, landmarks are localised through voting from all image voxels, which is completely different from the classification-based methods that use voxel-wise classification to detect landmarks. Despite their robustness, the accuracy of regression-based landmark detection methods is often limited due to (1) the inclusion of uninformative image voxels in the voting procedure, and (2) the lack of effective ways to incorporate inter-landmark spatial dependency into the detection step. In this paper, we propose a collaborative landmark detection framework to address these limitations. The concept of collaboration is reflected in two aspects. (1) Multi-resolution collaboration. A multi-resolution strategy is proposed to hierarchically localise landmarks by gradually excluding uninformative votes from faraway voxels. Moreover, for informative voxels near the landmark, a spherical sampling strategy is also designed at the training stage to improve their prediction accuracy. (2) Inter-landmark collaboration. A confidence-based landmark detection strategy is proposed to improve the detection accuracy of 'difficult-to-detect' landmarks by using spatial guidance from 'easy-to-detect' landmarks. To evaluate our method, we conducted experiments extensively on three datasets for detecting prostate landmarks and head & neck landmarks in computed tomography images, and also dental landmarks in cone beam computed tomography images. The results show the effectiveness of our collaborative landmark detection framework in improving landmark detection accuracy, compared to other state-of-the-art methods.

Variational Reconstruction of Left Cardiac Structure from CMR Images

Cardiovascular Disease (CVD), accounting for 17% of overall deaths in the USA, is the leading cause of death over the world. Advances in medical imaging techniques make the quantitative assessment of both the anatomy and function of heart possible. The cardiac modeling is an invariable prerequisite for quantitative analysis. In this study, a novel method is proposed to reconstruct the left cardiac structure from multi-planed cardiac magnetic resonance (CMR) images and contours. Routine CMR examination was performed to acquire both long axis and short axis images. Trained technologists delineated the endocardial contours. Multiple sets of two dimensional contours were projected into the three dimensional patient-based coordinate system and registered to each other. The union of the registered point sets was applied a variational surface reconstruction algorithm based on Delaunay triangulation and graph-cuts. The resulting triangulated surfaces were further post-processed. Quantitative evaluation on our method was performed via computing the overlapping ratio between the reconstructed model and the manually delineated long axis contours, which validates our method. We envisage that this method could be used by radiographers and cardiologists to diagnose and assess cardiac function in patients with diverse heart diseases.

Landmark constellation models for medical image content identification and localization

PURPOSE

Many medical imaging tasks require the detection and localization of anatomical landmarks, for example for the initialization of model-based segmentation or to detect anatomical regions present in an image. A large number of landmark and object localization methods have been described in the literature. The detection of single landmarks may be insufficient to achieve robust localization across a variety of imaging settings and subjects. Furthermore, methods like the generalized Hough transform yield the most likely location of an object, but not an indication whether or not the landmark was actually present in the image.

METHODS

For these reasons, we developed a simple and computationally efficient method combining localization results from multiple landmarks to achieve robust localization and to compute a localization confidence measure. For each anatomical region, we train a constellation model indicating the mean relative locations and location variability of a set of landmarks. This model is registered to the landmarks detected in a test image via point-based registration, using closed-form solutions. Three different outlier suppression schemes are compared, two using iterative re-weighting based on the residual landmark registration errors and the third being a variant of RANSAC. The mean weighted residual registration error serves as a confidence measure to distinguish true from false localization results. The method is optimized and evaluated on synthetic data, evaluating both the localization accuracy and the ability to classify good from bad registration results based on the residual registration error.

RESULTS

Two application examples are presented: the identification of the imaged anatomical region in trauma CT scans and the initialization of model-based segmentation for C-arm CT scans with different target regions. The identification of the target region with the presented method was in 96 % of the cases correct.

CONCLUSION

The presented method is a simple solution for combining multiple landmark localization results. With appropriate parameters, outlier suppression clearly improves the localization performance over model registration without outlier suppression. The optimum choice of method and parameters depends on the expected level of noise and outliers in the application at hand, as well as on the focus on localization, classification, or both. The method allows detecting and localizing anatomical fields of view in medical images and is well suited to support a wide range of applications comprising image content identification, anatomical navigation and visualization, or initializing the pose of organ shape models.

Estimation of Large-Scale Organ Motion in B-Mode Ultrasound Image Sequences: A Survey

Reviewed here are methods developed for following (i.e., tracking) structures in medical B-mode ultrasound time sequences during large-scale motion. The resulting motion estimation problem and its key components are defined. The main tracking approaches are described, and their strengths and weaknesses are discussed. Existing motion estimation methods, tested on multiple in vivo sequences, are categorized with respect to their clinical applications, namely, cardiac, respiratory and muscular motion. A large number of works in this field had to be discarded as thorough validation of the results was missing. The remaining relevant works identified indicate the possibility of reaching an average tracking accuracy up to 1-2 mm. Real-time performance can be achieved using several methods. Yet only very few of these have progressed to clinical practice. The latest trends include incorporation of complementary and prior information. Advances are expected from common evaluation databases and 4-D ultrasound scanning technologies.

Image-Based Personalization of Cardiac Anatomy for Coupled Electromechanical Modeling

Computational models of cardiac electromechanics (EM) are increasingly being applied to clinical problems, with patient-specific models being generated from high fidelity imaging and used to simulate patient physiology, pathophysiology and response to treatment. Current structured meshes are limited in their ability to fully represent the detailed anatomical data available from clinical images and capture complex and varied anatomy with limited geometric accuracy. In this paper, we review the state of the art in image-based personalization of cardiac anatomy for biophysically detailed, strongly coupled EM modeling, and present our own tools for the automatic building of anatomically and structurally accurate patient-specific models. Our method relies on using high resolution unstructured meshes for discretizing both physics, electrophysiology and mechanics, in combination with efficient, strongly scalable solvers necessary to deal with the computational load imposed by the large number of degrees of freedom of these meshes. These tools permit automated anatomical model generation and strongly coupled EM simulations at an unprecedented level of anatomical and biophysical detail.

Automatic 3-D segmentation of endocardial border of the left ventricle from ultrasound images

The segmentation of the left ventricle (LV) is an important task to assess the cardiac function in ultrasound images of the heart. This paper presents a novel methodology for the segmentation of the LV in three-dimensional (3-D) echocardiographic images based on the probabilistic data association filter (PDAF). The proposed methodology begins by initializing a 3-D deformable model either semiautomatically, with user input, or automatically, and it comprises the following feature hierarchical approach: 1) edge detection in the vicinity of the surface (low-level features); 2) edge grouping to obtain potential LV surface patches (mid-level features); and 3) patch filtering using a shape-PDAF framework (high-level features). This method provides good performance accuracy in 20 echocardiographic volumes, and compares favorably with the state-of-the-art segmentation methodologies proposed in the recent literature.

A Framework for Automated Spine and Vertebrae Interpolation-Based Detection and Model-Based Segmentation

Automated and semi-automated detection and segmentation of spinal and vertebral structures from computed tomography (CT) images is a challenging task due to a relatively high degree of anatomical complexity, presence of unclear boundaries and articulation of vertebrae with each other, as well as due to insufficient image spatial resolution, partial volume effects, presence of image artifacts, intensity variations and low signal-to-noise ratio. In this paper, we describe a novel framework for automated spine and vertebrae detection and segmentation from 3-D CT images. A novel optimization technique based on interpolation theory is applied to detect the location of the whole spine in the 3-D image and, using the obtained location of the whole spine, to further detect the location of individual vertebrae within the spinal column. The obtained vertebra detection results represent a robust and accurate initialization for the subsequent segmentation of individual vertebrae, which is performed by an improved shape-constrained deformable model approach. The framework was evaluated on two publicly available CT spine image databases of 50 lumbar and 170 thoracolumbar vertebrae. Quantitative comparison against corresponding reference vertebra segmentations yielded an overall mean centroid-to-centroid distance of 1.1 mm and Dice coefficient of 83.6% for vertebra detection, and an overall mean symmetric surface distance of 0.3 mm and Dice coefficient of 94.6% for vertebra segmentation. The results indicate that by applying the proposed automated detection and segmentation framework, vertebrae can be successfully detected and accurately segmented in 3-D from CT spine images.

Localization and Segmentation of 3D Intervertebral Discs in MR Images by Data Driven Estimation

This paper addresses the problem of fully-automatic localization and segmentation of 3D intervertebral discs (IVDs) from MR images. Our method contains two steps, where we first localize the center of each IVD, and then segment IVDs by classifying image pixels around each disc center as foreground (disc) or background. The disc localization is done by estimating the image displacements from a set of randomly sampled 3D image patches to the disc center. The image displacements are estimated by jointly optimizing the training and test displacement values in a data-driven way, where we take into consideration both the training data and the geometric constraint on the test image. After the disc centers are localized, we segment the discs by classifying image pixels around disc centers as background or foreground. The classification is done in a similar data-driven approach as we used for localization, but in this segmentation case we are aiming to estimate the foreground/background probability of each pixel instead of the image displacements. In addition, an extra neighborhood smooth constraint is introduced to enforce the local smoothness of the label field. Our method is validated on 3D T2-weighted turbo spin echo MR images of 35 patients from two different studies. Experiments show that compared to state of the art, our method achieves better or comparable results. Specifically, we achieve for localization a mean error of 1.6-2.0 mm, and for segmentation a mean Dice metric of 85%-88% and a mean surface distance of 1.3-1.4 mm.

Towards Personalized Cardiology: Multi-Scale Modeling of the Failing Heart

BACKGROUND

Despite modern pharmacotherapy and advanced implantable cardiac devices, overall prognosis and quality of life of HF patients remain poor. This is in part due to insufficient patient stratification and lack of individualized therapy planning, resulting in less effective treatments and a significant number of non-responders.

METHODS AND RESULTS

State-of-the-art clinical phenotyping was acquired, including magnetic resonance imaging (MRI) and biomarker assessment. An individualized, multi-scale model of heart function covering cardiac anatomy, electrophysiology, biomechanics and hemodynamics was estimated using a robust framework. The model was computed on n=46 HF patients, showing for the first time that advanced multi-scale models can be fitted consistently on large cohorts. Novel multi-scale parameters derived from the model of all cases were analyzed and compared against clinical parameters, cardiac imaging, lab tests and survival scores to evaluate the explicative power of the model and its potential for better patient stratification. Model validation was pursued by comparing clinical parameters that were not used in the fitting process against model parameters.

CONCLUSION

This paper illustrates how advanced multi-scale models can complement cardiovascular imaging and how they could be applied in patient care. Based on obtained results, it becomes conceivable that, after thorough validation, such heart failure models could be applied for patient management and therapy planning in the future, as we illustrate in one patient of our cohort who received CRT-D implantation.

Automatic quantification of aortic regurgitation using 3D full volume color doppler echocardiography: a validation study with cardiac magnetic resonance imaging

Recent advances in real-time three-dimensional (3D) echocardiography provide the automated measurement of mitral inflow and aortic stroke volume without the need to assume the geometry of the heart. The aim of this study is to explore the ability of 3D full volume color Doppler echocardiography (FVCDE) to quantify aortic regurgitation (AR). Thirty-two patients with more than a moderate degree of AR were enrolled. AR volume was measured by (1) two-dimensional-CDE, using the proximal isovelocity surface area (PISA) and (2) real-time 3D-FVCDE with (3) phase-contrast cardiac magnetic resonance imaging (PC-CMR) as the reference method. Automated AR quantification using 3D-FVCDE was feasible in 30 of the 32 patients. 2D-PISA underestimated the AR volume compared to 3D-FVCDE and PC-CMR (38.6 ± 9.9 mL by 2D-PISA; 49.5 ± 10.2 mL by 3D-FVCDE; 52.3 ± 12.6 mL by PC-CMR). The AR volume assessed by 3D-FVCDE showed better correlation and agreement with PC-CMR (r = 0.93, p < 0.001, 2SD: 9.5 mL) than did 2D-PISA (r = 0.76, p < 0.001, 2SD: 15.7 mL). When used to classify AR severity, 3D-FVCDE agreed better with PC-CMR (k = 0.94) than did 2D-PISA (k = 0.53). In patients with eccentric jets, only 30% were correctly graded by 2D-PISA. Conversely, almost all patients with eccentric jets (86.7%) were correctly graded by 3D-FVCDE. In patients with multiple jets, only 3 out of 10 were correctly graded by 2D-PISA, while 3D-FVCDE correctly graded 9 out of 10 of these patients. Automated quantification of AR using the 3D-FVCDE method is clinically feasible and more accurate than the current 2D-based method. AR quantification by 2D-PISA significantly misclassified AR grade in patients with eccentric or multiple jets. This study demonstrates that 3D-FVCDE is a valuable tool to accurately measure AR volume regardless of AR characteristics.

Automatic cardiac segmentation using semantic information from random forests

We propose a fully automated method for segmenting the cardiac right ventricle (RV) from magnetic resonance (MR) images. Given a MR test image, it is first oversegmented into superpixels and each superpixel is analyzed to detect the presence of RV regions using random forest (RF) classifiers. The superpixels containing RV regions constitute the region of interest (ROI) which is used to segment the actual RV. Probability maps are generated for each ROI pixel using a second set of RF classifiers which give the probabilities of each pixel belonging to RV or background. The negative log-likelihood of these maps are used as penalty costs in a graph cut segmentation framework. Low-level features like intensity statistics, texture anisotropy and curvature asymmetry, and high level context features are used at different stages. Smoothness constraints are imposed based on semantic information (importance of each feature to the classification task) derived from the second set of learned RF classifiers. Experimental results show that compared to conventional method our algorithm achieves superior performance due to the inclusion of semantic knowledge and context information.

Benchmark for Algorithms Segmenting the Left Atrium From 3D CT and MRI Datasets

Knowledge of left atrial (LA) anatomy is important for atrial fibrillation ablation guidance, fibrosis quantification and biophysical modelling. Segmentation of the LA from Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) images is a complex problem. This manuscript presents a benchmark to evaluate algorithms that address LA segmentation. The datasets, ground truth and evaluation code have been made publicly available through the http://www.cardiacatlas.org website. This manuscript also reports the results of the Left Atrial Segmentation Challenge (LASC) carried out at the STACOM'13 workshop, in conjunction with MICCAI'13. Thirty CT and 30 MRI datasets were provided to participants for segmentation. Each participant segmented the LA including a short part of the LA appendage trunk and proximal sections of the pulmonary veins (PVs). We present results for nine algorithms for CT and eight algorithms for MRI. Results showed that methodologies combining statistical models with region growing approaches were the most appropriate to handle the proposed task. The ground truth and automatic segmentations were standardised to reduce the influence of inconsistently defined regions (e.g., mitral plane, PVs end points, LA appendage). This standardisation framework, which is a contribution of this work, can be used to label and further analyse anatomical regions of the LA. By performing the standardisation directly on the left atrial surface, we can process multiple input data, including meshes exported from different electroanatomical mapping systems.

Computer Vision Techniques for Transcatheter Intervention

Minimally invasive transcatheter technologies have demonstrated substantial promise for the diagnosis and the treatment of cardiovascular diseases. For example, transcatheter aortic valve implantation is an alternative to aortic valve replacement for the treatment of severe aortic stenosis, and transcatheter atrial fibrillation ablation is widely used for the treatment and the cure of atrial fibrillation. In addition, catheter-based intravascular ultrasound and optical coherence tomography imaging of coronary arteries provides important information about the coronary lumen, wall, and plaque characteristics. Qualitative and quantitative analysis of these cross-sectional image data will be beneficial to the evaluation and the treatment of coronary artery diseases such as atherosclerosis. In all the phases (preoperative, intraoperative, and postoperative) during the transcatheter intervention procedure, computer vision techniques (e.g., image segmentation and motion tracking) have been largely applied in the field to accomplish tasks like annulus measurement, valve selection, catheter placement control, and vessel centerline extraction. This provides beneficial guidance for the clinicians in surgical planning, disease diagnosis, and treatment assessment. In this paper, we present a systematical review on these state-of-the-art methods. We aim to give a comprehensive overview for researchers in the area of computer vision on the subject of transcatheter intervention. Research in medical computing is multi-disciplinary due to its nature, and hence, it is important to understand the application domain, clinical background, and imaging modality, so that methods and quantitative measurements derived from analyzing the imaging data are appropriate and meaningful. We thus provide an overview on the background information of the transcatheter intervention procedures, as well as a review of the computer vision techniques and methodologies applied in this area.

Myocardial perfusion analysis in cardiac computed tomography angiographic images at rest

Cardiac computed tomography angiography (CTA) is a non-invasive method for anatomic evaluation of coronary artery stenoses. However, CTA is prone to artifacts that reduce the diagnostic accuracy to identify stenoses. Further, CTA does not allow for determination of the physiologic significance of the visualized stenoses. In this paper, we propose a new system to determine the physiologic manifestation of coronary stenoses by assessment of myocardial perfusion from typically acquired CTA images at rest. As a first step, we develop an automated segmentation method to delineate the left ventricle. Both endocardium and epicardium are compactly modeled with subdivision surfaces and coupled by explicit thickness representation. After initialization with five anatomical landmarks, the model is adapted to a target image by deformation increments including control vertex displacements and thickness variations guided by trained AdaBoost classifiers, and regularized by a prior of deformation increments from principal component analysis (PCA). The evaluation using a 5-fold cross-validation demonstrates the overall segmentation error to be 1.00 ± 0.39 mm for endocardium and 1.06 ± 0.43 mm for epicardium, with a boundary contour alignment error of 2.79 ± 0.52. Based on our LV model, two types of myocardial perfusion analyzes have been performed. One is a perfusion network analysis, which explores the correlation (as network edges) pattern of perfusion between all pairs of myocardial segments (as network nodes) defined in AHA 17-segment model. We find perfusion network display different patterns in the normal and disease groups, as divided by whether significant coronary stenosis is present in quantitative coronary angiography (QCA). The other analysis is a clinical validation assessment of the ability of the developed algorithm to predict whether a patient has significant coronary stenosis when referenced to an invasive QCA ground truth standard. By training three machine learning techniques using three features of normalized perfusion intensity, transmural perfusion ratio, and myocardial wall thickness, we demonstrate AdaBoost to be slightly better than Naive Bayes and Random Forest by the area under receiver operating characteristics (ROC) curve. For the AdaBoost algorithm, an optimal cut-point reveals an accuracy of 0.70, with sensitivity and specificity of 0.79 and 0.64, respectively. Our study shows perfusion analysis from CTA images acquired at rest is useful for providing physiologic information in diagnosis of obstructive coronary artery stenoses.